Share your feedback on the AI services experiment in Nightly

[asafko]

Hi folks, 

In the next few days, we will start the Nightly experiment which provides easy access to AI services from the sidebar. This functionality is entirely optional, and it’s there to see if it’s a helpful addition to Firefox. It is not built into any core functionality and needs to be turned on by you to see it. 

If you want to try the experiment, activate it via Nightly Settings > Firefox Labs (please see full instructions here). 

We’d love to hear your feedback once you try out the feature, and we’re open to all your ideas and thoughts, whether it’s small tweaks to the current experience or big, creative suggestions that could boost your productivity and make accessing your favorite tools and services in Firefox even easier.

Thanks so much for helping us improve Firefox!

[Mardak]

Have you noticed a decrease in performance when the feature is disabled or even enabled but not open?

[mega]

Obviously it'd take even longer to build.

[facni]

It would be nice if we were able to do right-click over the chatbot's panel to, for example, copy, paste and translate text or take screenshots.

[Mardak]

Thanks for the feedback. It seems like we should be able to add the right-click menu to the sidebar at least with copy and paste as those should already work with keyboard shortcuts, e.g., ctrl-c / cmd-c. Others like translate might need some additional work to get the appropriate text from the sidebar content.

[AdventureGamer]

NO. No, no, no, no, NO. NO AI in the last bastions of privacy on the internet!!! What the hell!!!!!!!!

[Emmic]

when are we going to get a post like this on the new toolbar and url box thing that's just come to nightly? because I have many thoughts about that for firefox to pretend to care about....

[KellyClowers]

The local PDF image alt text generation is an example of a good and useful "AI" feature.
This "AI chatbot" stuff is a an example of a bad and stupid "AI" feature. Not many people want this, those that do can install an extension or something. The LLMs use too many resources, and for at best very little benefit. Most of the time there is no or even negative benefit. This is not something Mozilla should be using or promoting. Granted, it is a fad that will be over in a couple years anyway most likely, but even in the mean time you shouldn't be supporting it.

[ffffff]

Yes! I was so happy when I saw the alt text generation plans, because it was the first time any tech organization I follow announced a use of AI that not only was I not disappointed with, but was actually excited for. It showed that there are plenty yet unexplored and helpful use cases of AI that don't involve the endorsement and promotion of unethical big tech companies and their products.

[Mardak]

Would you expect Mozilla to do something similar to Common Voice and image-to-text training data but for LLMs?

[ffffff]

I don't know, Mardak. While the idea sounds beautiful in principle, as I mentioned in another comment, that feels dangerously close to a moonshot. Big data seems to be the name of the game, so I doubt Mozilla could ever hope to catch up to the big players who crawled all the internet. Expecting that feels unfair and akin to setting both parties up for disappointment, along with a hefty waste of resources.

[Mardak]

Are you saying a local LLM would be good or bad? Something optimized to use low resources for securely handling private data could have more benefit than existing chatbots for some users.

[Honeebunny]

Booooo!! I expected better from Firefox, but you're just proving that companies as a whole don't have or care about ethics. Disgusting.

[MarryAurora12]

best

[meias]

Absolutely not. There was a post from Proton on mastodon recently that said something to the effect of AI is inevitable. I take problem with such a stance as it really does not have to be inevitable. On the contrary, the only inevitable thing about AI is that it will inevitably cause harm. Stealing information to train the model, causing massive amounts of environmental damage due to the resources needed to power it. This is not the path we need to go down.

I understand Mozilla is trying it's best to stay relevant, and to bring people back to Firefox. But we shouldn't be trying to stoop to the levels of the people we are purporting to be better than. A true genuine focus on the mission to serve people's needs, not the needs of corporate investors, is how we achieve that.

[cannoness]

I have no use for this feature; I just ran away from Chrome and back to Firefox recently over AI/privacy and it looks like I'll be leaving again for a browser not falling into the hype. I am saying this as someone who writes AI Software professionally, not a "change resistant user." Consider your audience.

[Mardak]

What aspect of privacy are you concerned about? Mozilla isn't collecting your prompts or selection to train a model.

[JHeuser]

All LLMs either rely on privacy violations, plagiarism, or both.

I notice you ask a lot of people to give recommendations for privacy preserving LLMs. I want you to name one that is privacy preserving, doesn't plagiarize (meaning it has a public list of all training data, and it's all used with permission), and doesn't use tremendous amounts of energy. If one exists, it should be the only one firefox allows, as that's the only moral and ethical way to go about this.

[tribbleglorp]

Are there any even left? Obviously Chrome and Edge are out, Opera seems to have it in spades if the youtube sponcon is anything to go by, and let's not talk about Brave. My beloved Fox was the last one standing as far as I can tell. 😞

[draconicrose]

Vivaldi.

[andwhyisit]

Librewolf might be worth looking at.

[wutongtaiwan]

You can learn about Brave's AI features.After entering the search term in the address bar, the AI option will appear,After clicking, a chatbot on the side will appear to reply to the content related to the search term,I recommend using Mozilla's own AI,I don't trust other companies' AI,I believe Mozilla can better protect privacy,And this AI should have no chat limit、No login required、No geographical restrictions,Chat history is only stored on the local computer

[Mardak]

I believe that feature isn't necessarily on search engine result page as it can generally include the page contents to the chatbot. Are you wanting the Firefox feature to automatically share the page with the chatbot instead of requiring text selection?

[wutongtaiwan]

Maybe you can give a button to choose whether or not you want to share the page. I think Brave's AI is more helpful and useful to me

[wutongtaiwan]

I'm willing to share the page with the chatbot, but there may be some privacy-conscious people who will object, so a toggle can be provided in the browser settings for the user to decide whether to enable it or not, and not to enable it by default.

[Mardak]

Would a different interaction, e.g., a dedicated "share page" button, separate from the current text-selection interaction work? Instead of a toggle, someone could decide not to make use of a 1-click "summarize" or "help me understand" on a search result page.

A toggle could also be useful such as part of another action, e.g., share the page with chatbot when opening reader view, so it's still one interaction after enabling?

[wutongtaiwan]

I feel like it's a little better to separate the share page button from the text selection interaction

[ffffff]

If Mozilla commits to going ahead with this feature unchanged despite significant localized backlash in this tiny discussion thread, you'd better start planning and preparing both the technical and the average user targeted PR for the release wayyy ahead.

Privacy-Preserving Attribution is arguably good, yet tech media still made a circus out of it, with countless users denouncing "Mozilla's Evil Fall to Advertising Revenue", completely missing the point of the feature and rarely taking the time to read a single paragraph on what it actually does. It's unfortunate, but some things really didn't go well with PPA's release.

This feature is arguably bad. Straight-up, no arguing, simply bad for many people. I don't know how one could spin this well—maybe there isn't even a good strategy, just a least bad one—but Mozilla must lay the groundwork, somehow. Right now, I feel like I can see the writing on the wall, except instead of writing it's a stain and the wall is actually Mozilla's reputation.

Again, I hope you folks figure something out, and it all goes well, because I'm not terribly excited about going into another space with numerous angry users who may or may not have read past the title of an article, to try and add nuance to the conversation and explain what Mozilla is actually doing and why. I know nobody asked me to do this, but I feel like I have to, sometimes, because it really ought to be Mozilla's job, yet the ball keeps getting dropped.

Maybe the majority of users don't care, so it seems advantageous frustrating a couple thousand to please a million, or something. I don't know if I agree with that choice, but it's your prerogative, I suppose.

[Mardak]

We had multiple blog posts as part of landing the initial exploration of the feature:

• A Values-First Approach to AI and Product
• Choose how you want to navigate the web with Firefox

This only landed in Nightly 129 less than a month ago, and we can continue to default the feature off as we make improvements. Do you have suggestions of what needs to be added to the feature before the release or the messaging?

[mega]

I suggest deleting the feature and apologizing publicly.

[Kwonunn]

Seconded.

[MageBunkshelf]

I agree on the strongest possible terms

[MageBunkshelf]

Not only do I remember that first blog post, I also remember the overwhelmingly negative response to it. People were saying this was a bad idea way back in June. Do Mozilla not read feedback? Please read the room.

[aminought]

@MardakHello! I have a suggestion. Can you add page url into context here? Some AI chats can answer more accurate if you provide this information.

[Mardak]

Sure, we can add more to the context, so any other values you think might be useful? Just checking, are you expecting the default prompts to include the url or that you want custom prompts to be able to include url and other values?

[aminought]

I think that providing title, url and selection is enough for correct answer. I've configured and actually use two custom prompts for my needs and in my language:
1. Summarize: only title, but I would prefer page url instead.
2. Explain: only selection.

IMO, custom prompts and custom providers are unnecessary.

[Mardak]

The latest Nightly 130 (20240726152430) includes a url in the context for prompt inclusion or targeting. It's not used in any prompts by default yet, but we'll look for feedback on how it might be used for prompts.

[wutongtaiwan]

Please forgive me for saying something here that is not related to this topic. I hope Mozilla will drop the feature of privacy attribution, or turn it off by default, otherwise Mozilla could be subject to a hefty fine if Europeans sue Mozilla for violating the GDPR

[Allan-L]

Olá, no final dos prompts padrões poderia colocar a linguagem que esta sendo utilizada na interface do firefox para que os chatbots respondam na linguagem do usuario: EX: <prompt>... reply in "<current ff language>"

EX 2: em portuguese-pt-br configured in settings

<prompt>... reply in Portuguese-PT-BR

[Mardak]

The current plan is to have localized builds use translated prompts, e.g., `Estou na página "%tabTitle%" com "%selection|12000%" selecionado. Por favor, resuma a seleção usando linguagem precisa e concisa. Use cabeçalhos e listas com marcadores no resumo, para torná-lo escaneável. Mantenha o significado e a precisão factual.` (Here I used Firefox Translate, but our usual community process will take care of it when we expose these prompts to localization.)

In the meantime, could you try changing the prompts from about:config `browser.ml.chat.prompt.prefix` and/or browser.ml.chat.prompts.0 to see if translated prompts or "reply in Portuguese pt-BR" works better for you?

[Allan-L]

I understand the plan to have variants in each language; however, I believe language models work better in the native language of their training. For example, when prompts are made in English, chatbots interpret the content better. Therefore, in my humble opinion, I would leave the body of the prompt in English, making it unnecessary to create prompts in all languages. This accelerates development and makes chatbots understand the prompt better. If I’m not mistaken, Google’s chatbot, for instance, only creates images when English is used. I tested it here, and as you can see from my screenshots, it worked perfectly by adding at the end for the bot to respond in the desired language. Congratulations and thank you for your attention.

Additionally, I would like to suggest another idea: in this submenu, an option could be added to open the chatbots without any prompt, such as:

• [Ask ChatGPT]
  • Open ChatGPT Panel
  • Summarize
  • Etc...
  •  

The other day, I was taking a test in the browser, and I had to keep switching to another tab to access the chat. If there was a side-by-side panel for asking questions, it would have helped me a lot.

One more question/idea: Have you considered having an option to send a screenshot of the open page to chatbots that support interpreting images?