translate pictures with firefox translations

[Jeppie]

Expand firefox translations with a option to translate words on a picture.

[Jon]

Thanks for submitting an idea to the Mozilla Connect community! Your idea is now open to votes (aka kudos) and comments.

[firecat]

It's a good idea. This is what I would like to see when Mozilla implies the introduction of AI in Firefox.

[ffffff]

I like the idea, but I think the description is really vague right now. "Translate words on a picture" could mean anything from running unreliable OCR on an image, passing the result through the translator and then putting it on a simple popup, up to something more sophisticated like Project Naptha's translation feature. Tentative kudos because I think it'd be amazing if Mozilla ever managed to get the more important stuff out of the way (like adding more highly used languages to translate) and then proceeded to tackle ideas like this one.

[Mozilla2048]

Good idea, but in my opinion it will be very difficult to implement. Such a tool would first have to analyze the image using some kind of deep learning AI model, detect text zones (in all different languages if the image contains more than one), convert them to plain text, including newlines and special characters, translate the words within the context, and put them back in the image without altering the background part or changing the font. Well, not impossible - Google translate has done something similar.