How Microsoft are raising the bar with their visual impairment technology

Apple has been the mainstream leader in accessibility over the past few years, but with two new apps from Microsoft, the technology giants have been firmly cementing their position as a top accessibility contender.

Technology and innovation are travelling at a million miles an hour and it’s been a real pleasure holding on to this juggernaut! There is also a tangible surge in creating solutions for people living with vision loss, with lots of incredible standalone devices and smartphone applications constantly being created which can give and/or maintain the user’s independence.

Two such applications have come thick and fast from the technology giants Microsoft. I’m talking about Seeing AI and Soundscape.

Seeing AI

Seeing AI arrived in the UK Apple store in November 2017, and we’ve been big fans of it over here at Henshaws ever since. It’s a free app that uses AI (artificial intelligence) for a number of nifty purposes including reading text, recognising products, and describing objects. It’s combined what a number of other, and often paid for, apps do in one app – leading it to be what I refer to as ‘the daddy of all apps!’

In our tests, we’ve managed to get it to recognise people from an impressive 14 feet away, decipher handwriting from Christmas cards, and even identify people from photos. Check out our series of videos putting it to the test.

Soundscape

The follow up to Seeing AI was Microsoft’s ‘Soundscape’, which came out earlier this year. The app is designed to be a navigational tool for blind and partially sighted people and uses 3D audio to create a mental map of the world around the user.

It involves some pretty amazing tech but can be a bit overwhelming to understand at first- what is the drumbeat for? And how do beacons work? I give it an overview in our video here, and then we tested it out on the road in a follow-up video, which demonstrates how the app can enable someone to navigate around with greater confidence and independence.

I love these two applications – the people who created these solutions should be more than chuffed with their current form. I say ‘current’ because I know for a fact that they haven’t finished with Soundscape yet… they’ve already recently updated the app to have improved use in cars and on public transport, as well as adding the ability to place audio beacons on addresses and places of interest, and there’s still plenty more tweaks to come.

Now, what would happen if Seeing AI and Soundscape had a baby? What would that look like?

I’m always mooching around the world wide web for new stuff, and I recently came across this article and video detailing some new innovation from our friends at Microsoft called HoloLens.

Microsoft Hololens is (according to their website); the “first self-contained, holographic computer, enabling you to engage with your digital content and interact with holograms in the world around you” through a head-mounted display. There are a number of apps available to use with the device, including ‘Holotour’ which provides 3D personal tours around the world of iconic locations, and ‘Holoanatomy’, which enables you to study human anatomy with virtual body systems that you can interact with.

Now the HoloLens might not immediately seem like a useful tool for someone with sight loss, but researchers at the California Institute of Technology have created a guiding app for the device that effectively combines together the benefits of Seeing AI and Soundscape. The app maps out pathways with a virtual guide giving navigational voice commands and is designed to effectively “give sounds to all relevant objects in the environment” for the visually impaired user.

The aim is for the app to enable users to navigate unfamiliar areas, including multi-story buildings, on their first attempt, and there’s even a video showing it in action:

 

I don’t know about you but if this became available this will be a game changer – an ‘on the fly, in building navigation support’ – bonkers! I can’t wait to see what the next five years will bring with regards to new technology.

If you live in the Greater Manchester area, Henshaws digital enablement team offer assessments, training and groups for anyone with a visual impairment of any age or technical ability. The team can even offer training on the Seeing AI and Soundscape apps; get in touch on 0300 222 5555 or at digital.enablement@henshaws.org.uk for more information.

To keep about the latest updates from Microsoft accessibility news, follow the team’s Twitter at @MSFTaccess, and make sure you sign up to the Henshaws Knowledge Village emails for updates on our latest videos, blogs, etc.

Log in or register to download

Check out our other technology blogs

We can't do it without you

Henshaws rely on voluntary donations; our work just wouldn’t be possible without people like you. Your support empowers local people living with sight loss and a range of other disabilities to increase their independence, achieve their dreams, and go beyond expectations.

Donate now