Autofill with OCR: Watch Proof-of-Concept DEMO and Share Your Feedback!

[Dimi]

Hi everyone!

The Firefox credential management team has been working on a Proof of Concept (PoC) that allows users to capture a photo of their credit card using Firefox on mobile, and then have the card information automatically filled on their Firefox desktop. The idea is to provide a smooth autofill experience for those who prefer not to store their credit card information on their computer or haven’t yet saved their card details in Firefox.

It's important to note that in this idea, the credit card information will not be stored on either the mobile phone or the desktop unless users explicitly choose to save it.

If you would like to see how it works, here is a demo video!

We’re interested in hearing what you think about this idea! Specifically, we’d love to know:

1. Do you like this idea?
2. Do you have any privacy or security concerns?
3. Any comments or suggestions on what you also want to see in this feature? For example, passport autofill etc

Your feedback will help us understand whether this idea is valuable and if this is an area we should focus on. Thanks in advance for sharing your thoughts!

[Erwan]

I will never use a feature like this.
 
    Do you like this idea?
    = No
    Do you have privacy or security concerns?
    = Of course !
 
I don't see how typing in your PINs manually is a problem, so I'll continue.

[okay_okay]

I like the convenience of it and that users will have the choice of storing credit card information or not. It looks like the user will still have to manually enter the security code, correct? @Dimi 

[Dimi]

Yes, that’s correct. Firefox doesn’t fill in the security code, so users will still need to enter it manually.

[lackey]

I'm all for catering towards niches of users, but this one has me scratching my head. Who exactly is this for? 

They care about privacy enough to not store credit cards on their desktop browser. Fair enough. 

But on the other hand, they would need to have a desktop, a phone, a Firefox account connecting the two, and a willingness to photograph their own credit card with their phone. 

I feel like a person who doesn't want to store credit card details on a desktop browser, might have issues with photographing it on a phone through a mobile app, synchronizing it to a desktop app, etc. And I feel like someone who simply wants to keep their credit card details on hand would rather type them in manually, instead of scrambling for both their credit card and their phone at the same time. 

But that's just my impression. 

[Dimi]

Hi Lackey,
Thank you for sharing your thoughts with us!
You’re right - whether there are users who prefer not to save their credit card in a browser but feel safer and find it convenient if we do it this way is exactly the question. This is what we’re trying to understand. Ultimately, while there are differences in the underlying implementation, what truly matters is how users feel. Your feedback helps us evaluate these assumptions more thoughtfully, so again, thank you for the valuable feedback!

[Erwan]

Will the produced photo be encrypted in our phone, on the transfer network and in the computer? otherwise, how can we ensure that the photograph of our credit card is not freely accessible by spyware or fraudulent network surveillance?

[Dimi]

Hi @Erwan, Thank you for your question! Here's the idea:
The photo will not be saved on the mobile phone. The extracted credit card information will be securely transmitted to the desktop via an end-to-end encrypted communication channel, similar to the mechanism used by Firefox Sync (see "Is it secure?" section).

The data will be decrypted by Firefox on desktop and use it for autofill. The credit card information will not be saved on the desktop unless the user explicitly chooses to save it via the capture doorhanger UI.
I hope this answers the question!

[myspace]

@Dimi Thanks for sharing this demo (hope it becomes a regular thing on Connect)! It might be nice in a pinch, but I don't think I would use this feature. 

[Dimi]

Hi @myspace, Thank you for the feedback! I also hope we’ll have the opportunity to share more demos on Connect.
As for "I don't think I would use this feature", may I ask the reason? Is it due to security or privacy concerns, being too complicated, or would you simply prefer to save your data in Firefox? Thank you!