An interesting rabbit hole

Yesterday, I got an interesting request from Peter Beens.

Is there an AI tool that can generate good alt-text for images? It's a difficult skill! @dougpete, have you come across anything in your travels?

— Peter Beens Mastodon: @pbeens@mstdn.ca 🇺🇦 (@pbeens) March 29, 2023

It’s an important concept. By using alt-text, browsers can interpret what’s in a picture for those who might have issues seeing it. Using alt-text is something that all good developers should be using. I’ll be honest here; I don’t do it nearly regularly enough and I’ve never had any questions as to why I don’t use it.

On the surface, it seemed to be a simple request.

I did what I thought should be the correct answer. I opened Bing search and the AI chat.

If there ever was going to be a softball lobbed up, it would have been that image. You can see it above on my blog. I was out at the Tip at Point Pelee one day and took that picture. It’s still one of my favourites.

I would have been happy with alt-text like “water” or “wavy water” or, if it was really good and noticed that the waves were coming from both directions, “wave at the Tip of Point Pelee”. Instead, it basically threw its hands up and said “I dunno”. It correctly identified that there was no alt-text attached to the image. That does bring up the question – if you’re just going to scrape that information from the page, you’re not really all that helpful.

So, I tried another Chat program. This one is from You.com. I sent it the same image link and got an “I don’t understand” in the first paragraph. However, the second paragraph got a bit of a Wow from me.

Now, the picture wasn’t from Pelee Island, but being close should count for something, shouldn’t it?

I did a whole lot of research and the leads all take me to Microsoft’s Azure’s Computer Vision API as best of breed when it comes to this. There’s a site from Sarah Drasner – https://codepen.io/sdras/pen/jawPGa that gives access to it. I played around with it with mixed success.

I’m guessing that we’re still early in the game for something as sophisticated as this request, Peter. It would seem to me to be logical for Edge’s implementation of Bing Chat to have access to the Azure resource.

So, to answer your question, I had limited success. I’ll be interested in following your work with this.

Advertisement