AI in Action: A practical comparison of image description tools

AI technology is constantly evolving, offering exciting opportunities for independence and accessibility for people with sight loss. From everyday tasks at home to navigating new environments, these tools are breaking down barriers by providing visual information instantly.

In this blog, we’re exploring how AI can be used to describe images, giving you a hands-on look at how different tools compare.

We’ll test those specifically designed for people with sight loss like Be My AI, Access AI, Seeing AI and Picture Smart AI. We’ll also test ChatGPT, a leading AI tool that many people use and isn’t specifically for people with sight loss.

Each of these tools aims to provide detailed descriptions of images, but how do they measure up when faced with the same pictures? We’ll be uploading the same two images to each tool and sharing the exact description they generate, giving you a practical side-by-side comparison.

Whether you have sight loss yourself, support someone who might benefit from these tools, or are a content creator looking to make your work more accessible, this experiment will offer real insights into the strengths, differences, and potential of these technologies.

For each tool, we’ll give a brief overview before diving into the best part—seeing it in action. It’s worth mentioning that, because these tools generate descriptions using natural language, the same image uploaded multiple times may result in slightly different descriptions. For the purpose of this blog, we’ll upload an image once to each tool and share the exact description generated at that moment.

Without further ado, let’s get started!

Here are the two images featured in this blog: an infographic showcasing the positive outcomes of our Sight Loss Services and a photo from a drama workshop at our Arts and Crafts Centre.

Infographic with circular graphic showing positive feedback about Henshaws services. Icons representing information, support, wellbeing, and confidence accompany the statistics. An older man with a cane and cup of tea is pictured below the data.
A woman wearing a tiger hat sits smiling in a chair. A man is knelt beside her with a red puppet with yellow hair. The setting is a colourful activity room with props and containers of toys visible in the background.

ChatGPT

ChatGPT is an AI tool that can help with almost anything. This could be doing research, drafting content, or analysing data. It can generate image descriptions too.

ChatGPT is mostly accessible, both on the website and through the app. There are some unlabelled buttons in the chat interface, but the accessibility of the ChatGPT interface has come on leaps and bounds since launch.

How good is ChatGPT at describing images? For our test, we uploaded an image to the ChatGPT iOS app and sent the prompt ‘please describe this image’. We used a free ChatGPT account.

Putting ChatGPT to the test

Click here for the ChatGPT website
Download ChatGPT from the App Store
Download ChatGPT from the App Store

Be My AI

Be My AI is a powerful feature of the well-known Be My Eyes app, which has long connected people with visual impairments to sighted volunteers for assistance with visual tasks.

Be My AI enhances what the app can offer. By simply taking a picture (or sharing an image from another app), Be My AI uses ChatGPT to provide a detailed description. Even better, you can ask follow-up questions to uncover more details about the image.

How did it perform when we gave it a try? Let’s find out.

Putting Be My AI to the test

Click here for the Be My Eyes website
Download Be My Eyes from the App Store
Download Be My Eyes from the Play Store
Download Be My Eyes for Windows

Access AI

Access AI is a groundbreaking feature of the Aira Explorer app, the app known for connecting people with visual impairments to professionally trained visual interpreters.

Access AI now adds an extra layer of independence, providing AI-generated descriptions of images you capture through the app or share from other apps. You can even ask follow-up questions to get more details about the image.

Access AI comes with some advanced features, including options to adjust language and verbosity preferences, so descriptions are tailored to what suits you best. Moreover, Access AI saves your chat history for easy reference. Specifically, it can store up to the 50 most recent chats. If you ever need extra reassurance, Aira’s visual interpreters can verify the AI’s responses and make any necessary adjustments.

For our experiment, we’ve set both the verbosity and language preferences to ‘default’ to give a clear picture of how Access AI works right out of the box. Now, here goes with our test!

Putting Access AI to the test

Click here for the Aira website
Download Aira Explorer from the App Store
Download Aira Explorer from the Play Store

Seeing AI

Seeing AI is one of the most popular and well-regarded apps within the visually impaired community. This versatile app can read printed text, describe surroundings, detect colours, sense light, and much more.

Now, Seeing AI offers even more detail through even more advanced AI. After using the Describe Scene mode, or sharing a photo to Seeing AI from another app, you can activate the ‘more info’ button to receive an even richer, more detailed description. This feature adds another layer of insight, helping you gain a deeper understanding of what’s around you.

Putting Seeing AI to the test

Download Seeing AI from the App Store
Download Seeing AI from the Play Store

Picture Smart AI

Do you use the JAWS for Windows screen reader? JAWS offers a range of advanced features to make computer use more accessible for people who are blind, and one of those is Picture Smart AI.

Picture Smart AI harnesses the power of AI, specifically Claude and ChatGPT, to analyse and describe visual content. While it’s great for describing pictures (as the name suggests), it can also go beyond that—describing your active window or the entire screen.

With Picture Smart AI, you can get descriptions of image files, images within documents and on web pages, and more. Consequently, there are different keystrokes available so you can use the features that you need quickly and easily. You also have the flexibility to ask specific questions about an image—whether you want a particular detail before it’s analysed, or need extra information after receiving an initial description.

For our experiment, we’ll upload an image file through File Explorer. Picture Smart AI lets you generate both long and short descriptions. For our test, we’re going to generate a long description, to demonstrate all of its capabilities.

Putting Picture Smart AI to the test

Download JAWS

The pitfalls

This side-by-side comparison has hopefully given you a real insight into how amazing AI can be, especially with the level of detail that AI generates. However, we can’t ignore the pitfalls. AI isn’t perfect, and there are things you need to be mindful of and aware of.

Making assumptions

One of the downfalls of AI is its tendency to make assumptions about the social context of an image, which can sometimes be inaccurate.

AI tools are trained on large datasets, but these datasets don’t always reflect the full diversity of the real world. This can lead to underrepresentation of certain groups, and assumptions based on stereotypes or patterns.

This issue is emphasised in the descriptions generated for the image taken at our Arts And Crafts Centre. Several AI tools incorrectly assumed it was a setting for children, when in fact, our Arts and Crafts Centre is a creative space for adults. The AI oversimplified the context, likely interpreting the presence of art supplies and a puppet as indicators of a children’s environment. This highlights a limitation of AI. It can’t yet grasp the ‘full picture’ and often relies on incomplete or biased patterns to interpret what it sees.

Other things to be aware of

Below we outline some more things to be aware of when using AI to describe images.

Wrapping up

AI-powered image description tools are opening up exciting possibilities for people with sight loss, making visual information more accessible, and empowering people to engage more fully with the world around them. From our side-by-side comparisons, it’s clear that each tool has its own strengths and unique features, offering a range of ways to describe images in different contexts. Whether it’s the added detail of a follow-up question or the convenience of adjusting settings to suit your needs, these tools demonstrate just how far AI has come in supporting independence.

However, AI has its limitations. There are situations where you shouldn’t 100% rely on AI, but having AI tools in your toolbox can revolutionise your independence.

We hope this blog has given you practical insights into what each tool can offer, helping you find the ones that might best suit your needs—or those of someone you support. As AI technology continues to advance, so too will the possibilities for creating a more accessible world for all.

Henshaws graphic with pink and purple shapes

Learn how to get involved

From fundraising events to volunteering, there are plenty of ways to give to make a difference at Henshaws

Accreditations & Awards