Ben Fletcher

Looking back to the blog entry about train station as a partially blind citizen, what can augmented reality technology do?

Everything is of greyness.  To the untrained sight, it is not immediately very clear where the taxi rank may be.

Augmented Reality

The same except that the view is now augmented with useful assistance.  A pink path that suggests where the taxi rank may be.  A yellow circle that suggests where the first taxi’s passenger door may be.

The same thing, but from a different standpoint.

Visor

Photo Credits: Leonard Low

This visor is not just a visor, but augmentation is projected onto the back of the visor, providing an augmented view for the wearer. The augmentation of the pink and the yellow objects can possibly one day be projected onto the visor. The intriguing technical challenges may include determining the visor’s idea of location and orientation to accurately augment the computed objects to non-computed objects such as the taxi rank and the taxi passenger door.

This picture shows very clearly where to not walk to, to avoid falling off the platform! What could be more useful is that no longer does the citizen need to listen out for announcement or keep an eye on the public display, it’s all written out on the platform.

Thanks

Ian Hughes for this entry’s inspiration.

I imagine myself arriving at a train station somewhere random in Sydney.  I am a citizen who is:

• deaf without speech (hearing impaired); and
• partially blind (visually impaired).

What to do? Here is a demonstration of a small Proof of Concept.

The top two-thirds of the screen shows the map of the station. The bottom one-third contains the text saying welcome to the station.

Text to Speech Technology

The text is also spoken using the phone’s built in technology which translates text into audio for the blind citizen or James Bond.

Navigation

I press the big button, and a menu is brought up.

This list shows just the Go to Ticket Office option, but of course it would contain options for other places such as Toilet, Barrier Line, Taxi Rank, and so on.  I scroll to the Go to Ticket Office option and select it.

I can navigate to the ticket office using a choice of the map with the route, the text, or the voice guidance.

Location Aware Technology

I can tell the phone where I am going manually or automatically: manually by using the phone’s joystick as I walk along; automatically by using location aware technology.

Location Aware Messaging Technology

As soon as I arrive at the ticket office, the phone, knowing that I’m at ticket office, informs me that the office is closed and that the barrier line is open. This is because of the location aware messaging technology, which uses my location information to provide me with more relevant information.

The phone then takes me to the barrier line.  Before I catch the train, I would like to go to the toilets.

Profile

The phone knows I want to go to the gentlemen, rather than ladies, because my profile is provided with the phone.  This profile includes my gender; it can also contain my disabilities and alert the staff of my needs as I enter the station.

Let’s say that I am actually female and I ended up in the ladies.  I want to take a beeline to the next train for London.

The phone picks up my new location as being in the ladies, rather than the gentlemen, and takes me to Platform 3 which has the next train to London.  This is calculated through messaging technology which should be linked to the railway’s information backends.

Rendezvous

The phone provides me with up to date information regarding the train delay and how to make a connection with the conductor if I need his assistance. This eliminates the need for a good eyesight to read the public display and/or hearing to be alerted through the speaker.

Development

This is a standalone application which runs on the Nokia N95 using Python (for Series 60) and MQTT (a lightweight messaging protocol).  I hope to develop the same application that is web-based and therefore can be used by a compatible mobile phone with no need for an application installation.

Contacts

I was inspired to write this when I heard through Andy Stanford-Clark that Glenn Wightwick, another inspiring colleague from Australia, produced something similar, and with thanks to Bharat Bedi and Dave Conway-Jones for some more inspiration behind this work.  Lastly, but by no means least,  the LAMA team who worked on very similar scenarios.

See Also

Location Aware Messaging for Accessibility (LAMA) – an Extreme Blue project.

MQTT – the lightweight messaging protocol used in this PoC.

Really Small Messaging Broker (RSMB) – the lightweight messaging broker used in this PoC which can also be downloaded.