topic Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback! in Discussions

Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Dimi — Thu, 21 Nov 2024 15:04:19 GMT

Hi everyone!

The Firefox credential management team has been working on a Proof of Concept (PoC) that allows users to capture a photo of their credit card using Firefox on mobile, and then have the card information automatically filled on their Firefox desktop. The idea is to provide a smooth autofill experience for those who prefer not to store their credit card information on their computer or haven’t yet saved their card details in Firefox.

It's important to note that in this idea, the credit card information will not be stored on either the mobile phone or the desktop unless users explicitly choose to save it.

If you would like to see how it works, here is a demo video!

We’re interested in hearing what you think about this idea! Specifically, we’d love to know:

Do you like this idea?
Do you have any privacy or security concerns?
Any comments or suggestions on what you also want to see in this feature? For example, passport autofill etc

Your feedback will help us understand whether this idea is valuable and if this is an area we should focus on. Thanks in advance for sharing your thoughts!

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Erwan — Mon, 25 Nov 2024 16:06:15 GMT

I will never use a feature like this.

    Do you like this idea?
    = No
    Do you have privacy or security concerns?
    = Of course !

I don't see how typing in your PINs manually is a problem, so I'll continue.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

okay_okay — Mon, 25 Nov 2024 21:04:24 GMT

I like the convenience of it and that users will have the choice of storing credit card information or not. It looks like the user will still have to manually enter the security code, correct? @Dimi

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Dimi — Tue, 26 Nov 2024 07:51:09 GMT

Yes, that’s correct. Firefox doesn’t fill in the security code, so users will still need to enter it manually.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

lackey — Wed, 27 Nov 2024 18:21:45 GMT

I'm all for catering towards niches of users, but this one has me scratching my head. Who exactly is this for?

They care about privacy enough to not store credit cards on their desktop browser. Fair enough.

But on the other hand, they would need to have a desktop, a phone, a Firefox account connecting the two, and a willingness to photograph their own credit card with their phone.

I feel like a person who doesn't want to store credit card details on a desktop browser, might have issues with photographing it on a phone through a mobile app, synchronizing it to a desktop app, etc. And I feel like someone who simply wants to keep their credit card details on hand would rather type them in manually, instead of scrambling for both their credit card and their phone at the same time.

But that's just my impression.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Dimi — Thu, 28 Nov 2024 08:58:05 GMT

Hi Lackey,
Thank you for sharing your thoughts with us!
You’re right - whether there are users who prefer not to save their credit card in a browser but feel safer and find it convenient if we do it this way is exactly the question. This is what we’re trying to understand. Ultimately, while there are differences in the underlying implementation, what truly matters is how users feel. Your feedback helps us evaluate these assumptions more thoughtfully, so again, thank you for the valuable feedback!

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Erwan — Mon, 02 Dec 2024 11:35:50 GMT

Will the produced photo be encrypted in our phone, on the transfer network and in the computer? otherwise, how can we ensure that the photograph of our credit card is not freely accessible by spyware or fraudulent network surveillance?

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Dimi — Tue, 03 Dec 2024 14:57:28 GMT

Hi @Erwan, Thank you for your question! Here's the idea:
The photo will not be saved on the mobile phone. The extracted credit card information will be securely transmitted to the desktop via an end-to-end encrypted communication channel, similar to the mechanism used by Firefox Sync (see "Is it secure?" section).

The data will be decrypted by Firefox on desktop and use it for autofill. The credit card information will not be saved on the desktop unless the user explicitly chooses to save it via the capture doorhanger UI.
I hope this answers the question!

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

myspace — Tue, 10 Dec 2024 17:45:22 GMT

@Dimi Thanks for sharing this demo (hope it becomes a regular thing on Connect)! It might be nice in a pinch, but I don't think I would use this feature.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Dimi — Thu, 12 Dec 2024 09:16:48 GMT

Hi @myspace, Thank you for the feedback! I also hope we’ll have the opportunity to share more demos on Connect.
As for "I don't think I would use this feature", may I ask the reason? Is it due to security or privacy concerns, being too complicated, or would you simply prefer to save your data in Firefox? Thank you!

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

RexPouros — Tue, 24 Dec 2024 06:02:09 GMT

alright

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

daniel3 — Tue, 31 Dec 2024 20:53:33 GMT

This concept is highly practical and enhances convenience by allowing users to transfer credit card details from mobile to desktop seamlessly. However, strong privacy measures, such as encrypted data transmission and temporary storage, are crucial. Expanding the feature to include passport and ID autofill, multi-language support, and accessibility options would make it even more versatile. It’s a promising idea with great potential to improve user experience while ensuring security and trust.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

TwoNames — Fri, 10 Jan 2025 16:33:52 GMT

I am a Veteran of some high age and like some of the auto-fill features of normal public info, i.e. name and address. The VA is in the long process of making sure veterans have secure access to their accounts and supports such as MyHealth.org. They are going with a system named ID.me, which appears to me to be a very large inclusive arena for more than just the VA but many other government agencies. Over the past year of so I have attempted to follow the directions and use of this "revised" system but it has failed me in being so convoluted as to be useless and possibly even far easier to be hacked by anyone interested in getting into not only veterans' accounts but Social Security and others. As much as I would like to believe those working on this ID.me or whatever (I forget the actual acronym at this moment) they do not appear to have the expertise of you folks at Mozilla! And that's the pity of some government agencies. And beginning Jan 20 it will get unbelievably worse!

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Dimi — Tue, 14 Jan 2025 09:12:15 GMT

Thank you for the feedback and for the suggestion to include passport and ID autofill. This is really valuable information for us!

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Donnybhoy — Wed, 15 Jan 2025 15:08:10 GMT

It doesn't work. It used to, but it doesn't now

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Point-To-Ponder — Sun, 20 Apr 2025 16:51:59 GMT

As they say, Once the Jeanie is out of the Bottle. Yet Similar to Taking a picture of a Check for Direct Deposit. This is the world we live in, adapting is part of it. Better to be ahead of the curve as a Business then follow behind it.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

Point-To-Ponder — Sun, 20 Apr 2025 17:16:45 GMT

Hybrid concept; Might Have a use for Business who Check Id's for entry (such as Bars/Pubs). Example Take a picture of a face then of the I.D. allowing OCR of ID info and I.D. picture to compare with a AI Probability match output. Additionally approve or reject feedback on Date of Birth when compared against user age preset on Entry. Would be Great tool to create a Business customer Data Base. Uses are endless

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

ITScrapBuyer — Sat, 31 Jan 2026 19:03:18 GMT

Trusted Dead Stock Buyers in UAE including in Dubai,Sharjah and all States.We provide a eco-friendly way to sale your old,damaged and unused IT Equipments at very fair rates,with fast Pickup Service. https://itscrapsbuyer.com/

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

redolent — Fri, 27 Mar 2026 11:55:23 GMT

This is a really interesting concept—especially for users who don’t want to store sensitive card details locally.

One challenge I’ve noticed with similar OCR-based autofill systems is accuracy across different lighting conditions and card designs. Are you planning any fallback validation or manual correction flow?

Also, I recently explored how OCR and autofill workflows are evolving in web environments, including security and UX considerations—might be useful here: redolent-me.com

Would love to hear more about how Firefox plans to handle edge cases in real-world usage.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

redolent — Fri, 27 Mar 2026 11:56:29 GMT

This is a really interesting concept—especially for users who don’t want to store sensitive card details locally.

Also, I recently explored how OCR and autofill workflows are evolving in web environments, including security and UX considerations—might be useful here: redolent

Would love to hear more about how Firefox plans to handle edge cases in real-world usage.

Re: Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

RahimSoft — Sat, 28 Mar 2026 00:05:10 GMT

This is a really exciting direction OCR-based autofill has huge potential to simplify workflows and save time, especially for users who deal with a lot of forms or scanned documents.

From the demo perspective, it would be great to see how well it handles different formats and handwriting styles, as well as how accurate it is with low-quality images. Maybe adding a quick “confidence score” or allowing users to easily review/edit extracted fields before submission could make it even more reliable.

Also, thinking ahead, integrations with common file types (PDFs, screenshots, mobile captures) and privacy controls would be key for wider adoption.

Overall, this feels like a feature that could genuinely improve user experience if polished well—great work on pushing this forward!

On a side note, it’s always interesting to see how different platforms are improving accessibility and efficiency just like how people rely on trusted resources such as a Everett Truck Accident Lawyer when dealing with complex situations, having reliable autofill tools can make everyday digital tasks much easier and more approachable.