Be My AI: Powerful visual assistance with AI

Want to see Be My AI working its magic? In the video below, you can see and hear how Be My AI works.

Introducing Be My AI

Have you ever felt frustrated by limited image descriptions from artificial intelligence (AI) apps, leaving you longing for more detailed descriptions or the ability to ask questions?

We understand how important it is to have access to thorough information, and that’s why we’re excited to introduce you to be My AI, a cutting edge solution to these very issues.

The best part? It is completely free, and seamlessly integrated into an app you may already be familiar with: Be My Eyes.

Be My AI is currently in open beta on both iOS and Android. This means that Be My AI is available for everyone who uses Be My Eyes on iOS and Android, while Be My Eyes continues to develop it and make it even better than it already is, based on user feedback.

Alice, our Digital Communications Officer, who is also totally blind and passionate about investigating new technology, had the privilege of testing Be My AI early in the closed beta phase, after registering her interest as soon as she could.

This blog is based on research, both online and through her first-hand experience of using it on her iPhone.

What is Be My AI?

Be My Eyes, the app created to connect visually impaired individuals to sighted volunteers, has introduced an innovative new feature: Be My AI.

Powered by GPT-4, an advanced AI model, Be My AI generates detailed image descriptions using human-like language.

It not only generates the descriptions, it also supports the ability to ask follow-up questions, so you can get descriptions tailored to your needs.

Don’t worry – if you still need support from one of the many Be My Eyes volunteers, they are still just a call away.

How do I use Be My AI?

When you open the Be My Eyes app, navigate to the Be My AI tab. Here, you will find a ‘take picture’ button, which when activated, will take a picture of what your rear camera is pointing at. Within moments, you’ll receive a description of that image. Amazing, right?

Your interactions with Be My AI are like a natural conversation, much like what you would get on any other messaging app on your device.

Once you have received the initial description, you have several options:

The Be My AI interface, showing the Take Picture button in the centre of the screen. Along the bottom are the five tabs found in the Be My Eyes app: Get Support, Be My AI, Community, Learn and Settings.

What about getting descriptions of pictures from other apps?

Be My AI goes beyond just generating descriptions of images you take within the Be My Eyes app. You can receive descriptions of photos from other apps too!

You could use Be My AI to sort through your photo library, get descriptions of photos sent to you in chats on apps like Messages and WhatsApp, and get descriptions of images you come across on social media.

If you’re in any app that supports photo sharing using the standard iOS share sheet, you’re all set. Once you’re in the share sheet, activate the ‘Describe with Be My Eyes’ option and Be My AI will jump into action, generating a detailed description for you.

Some apps, like Facebook, don’t allow you to share directly to Be My AI despite it being an option in the share sheet – don’t worry, there is a workaround for this: save the picture to your camera roll, and you can share it from there.

Just like when using Be My AI through taking pictures with the Be My Eyes app, you can ask follow-up questions to dig deeper if you’re after more specific information.

When you’re done, just press the Cancel button and you will be returned seamlessly back to the image so you can continue browsing where you left off.

When can I use Be My AI?

The uses for Be My AI are endless – here is just a small handful of examples:

It’s important to note that Be My AI currently only supports documents in picture format, for example JPG and PNG. While it can’t process other formats such as PDF, and it can’t yet describe videos, the possibilities with image-based content are vast.

Want an example of Be My AI in action? The photo below was taken at our wonderful Arts And Crafts Centre in Knaresborough, of one of our fantastic art makers in our pottery workshop. The alt text is the description generated by Be My AI.

The image shows a man with short hair and glasses, smiling broadly. He is wearing a black jacket over a blue apron and blue gloves. He is holding a large, star-shaped piece of pottery in his hands, which he appears to be crafting or decorating with clay. The setting looks like a pottery workshop with various pottery tools and supplies in the background, including shelves filled with pottery pieces and a bulletin board with papers. There's also a first aid box on the wall. The atmosphere seems cheerful and creative.

How do I use Be My AI for support with Microsoft products?

Be My AI is not only available as a standalone feature, it is also available specifically for support with Microsoft products and applications through the Microsoft Disability Answer Desk. How does it work might you ask?

Be My Eyes has a feature called Specialised Help, which enables you to connect to representatives from a range of organisations for support with their products and services. One company offering this is Microsoft, and you now have the option to connect to Microsoft’s Disability Answer Desk using Be My AI.

Don’t worry, the option to call a Microsoft representative hasn’t gone away!

What should I know before using Be My AI?

Be My AI uses advanced artificial intelligence to generate image descriptions.

It’s important to note that while it strives for accuracy, it may not get it right 100% of the time.

When it comes to taking photos, Be My AI doesn’t provide real-time guidance, meaning you may not know exactly what you have taken a picture of until you receive a description. If you’re aiming for a specific image, it may take a few attempts. While Be My AI can offer guidance for taking photos as part of a generated description or answer to a follow-up question, this isn’t instant, so it may take a few attempts to refine your photos for what you are trying to get a description of.

For your safety and the best user experience, Be My Eyes recommends that you do not use Be My AI for the following tasks:

  • Interpreting personal and sensitive information
  • Accessing allergen information
  • Reading the contents of medicine packages
  • Assisting with mobility – You could use Be My AI to check the boards at your local bus stop, but it is not a substitute for a mobility aid, or orientation and mobility skills.

What does Alice think of Be My AI?

As soon as I heard about Be My AI, I immediately got myself on the waitlist, and was counting down the days until I could finally use it.
As soon as I could, I immediately put it to work.
If there’s one thing I love most about Be My AI, it’s the quality of the descriptions. I’ve never had an app give me such detail.
For example, I shared a photo of my cat – Be My AI not only told me it was a photo of a cat, it also told me what my cat looked like, and about what she was lying on.
I also shared a photo of an invitation to an Apple event that had been posted on social media, and Be My AI described all the content including text, the Apple logo, and the placement of the content on the invitation, for example what was at the top and what was at the bottom.
Other apps I use, like Seeing AI, have introduced the ability to generate detailed descriptions, but the descriptions don’t feel as natural as those generated by Be My AI.
Be My AI will now always be my first choice for image descriptions, unless I need to ask for assistance from a person.

Alice, Henshaws Digital Communications Officer and Be My AI user

What do we think at Henshaws?

We’re all about enabling our fantastic service users to achieve their ambitions and go beyond expectations here at Henshaws, and we think Be My AI is another tool that will enable them to do just that.

Be My AI is groundbreaking and life-changing technology. Be My Eyes is very dedicated to making it the best it can be, to enable people to get the most out of it and enjoy its benefits with ease.

How can I watch Be My AI in action?

Want to see Be My AI working its magic? In the video below, you can see and hear how Be My AI works, with even more examples of image descriptions. It also gives the main points of this blog.

Henshaws graphic with pink and purple shapes

Learn how to get involved

From fundraising events to volunteering, there are plenty of ways to give to make a difference at Henshaws

Accreditations & Awards