Share your feedback on the AI services experiment in Nightly

[asafko]

Hi folks, 

In the next few days, we will start the Nightly experiment which provides easy access to AI services from the sidebar. This functionality is entirely optional, and it’s there to see if it’s a helpful addition to Firefox. It is not built into any core functionality and needs to be turned on by you to see it. 

If you want to try the experiment, activate it via Nightly Settings > Firefox Labs (please see full instructions here). 

We’d love to hear your feedback once you try out the feature, and we’re open to all your ideas and thoughts, whether it’s small tweaks to the current experience or big, creative suggestions that could boost your productivity and make accessing your favorite tools and services in Firefox even easier.

Thanks so much for helping us improve Firefox!

[Mardak]

Any suggestions on what's a nice UI for adding custom prompts? The current way with about:config preferences is more for advanced testers, but it could work for you in the meantime. Or if not, what prompts were you wanting to add and perhaps we can add it to the default list.

[Mardak]

The latest Nightly 130 (20240726152430) has an initial exploration of custom prompts that you can activate from about:config with `browser.ml.chat.shortcuts` (to get the text selection popup) and `browser.ml.chat.shortcuts.custom` (to get the input box).

Is this something like what you were thinking?

[advpod]

No.

Stop adding dumb trendy features. Spend engineering effort on integrating features that have nothing to do with browsing the web into a web browser. If I wanted to use a chatbot, I would install a chatbot app onto my computer. Features that have nothing to do with browsing the web have no place being in a web browser.

[Mardak]

How many people install separate chatbot apps vs access from a web browser? It seems like many people who do use chatbots are already using a web browser and likely manually copy/pasting information from tab to tab.

Your point of non-browser usage is still interesting, so could you provide some more concrete examples?

[Kwonunn]

Then just make it possible for extensions to add sidebar tabs and offer this as an extension! Instead of genuinely adding something useful to Firefox you've just opened Pandora's Box.

[IBBoard]

---

@Mardak wrote:

Your point of non-browser usage is still interesting, so could you provide some more concrete examples?

---

Let's flip it around. Can you provide some concrete examples of where adding this to Firefox is necessary and beneficial over having a separate app or an extension or (heaven forbid) just visiting the chatbot's website directly?

So far there appears to be absolutely nothing compelling about including this. A big company like Mozilla must have a Product Manager. Someone must have seen a user story and said "Yes! This is a killer one that we must spend engineering effort on!". But the huge, overwhelming majority of responses here seem to show that it's a user story that's out of touch with its users.

So what was that user story? What makes it so important? What value is this supposed to unlock that simpler, more ethical, more privacy-preserving and less environmentally damaging approaches can't do?

[eladio77]

Amazing feature. I find the funtion, but when i have active the vertical tabs i can not see the Boton "Chat IA", only i can see it without "Vertical Tabs" active. Can you add the shortcut access to the Vertical Tab menu?

[Mardak]

If you're referring to the new "sidebar.revamp" (with "sidebar.verticalTabs"), you should now have the AI chatbot icon available as shortcut access in today's Nightly.

 

[mcortt]

Very nice! But please add a keyboard shortcut to open it as well.

Edit: I didn't see where you had replied to my earlier suggestion. Please disregard, sorry!

[eladio77]

Thanks..  is working ...

[aminought]

@MardakHello! I have a suggestion. Can you add page url into context here? Some AI chats can answer more accurate if you provide this information.

[Mardak]

Sure, we can add more to the context, so any other values you think might be useful? Just checking, are you expecting the default prompts to include the url or that you want custom prompts to be able to include url and other values?

[aminought]

I think that providing title, url and selection is enough for correct answer. I've configured and actually use two custom prompts for my needs and in my language:
1. Summarize: only title, but I would prefer page url instead.
2. Explain: only selection.

IMO, custom prompts and custom providers are unnecessary.

[Mardak]

The latest Nightly 130 (20240726152430) includes a url in the context for prompt inclusion or targeting. It's not used in any prompts by default yet, but we'll look for feedback on how it might be used for prompts.

[Allan-L]

Olá, no final dos prompts padrões poderia colocar a linguagem que esta sendo utilizada na interface do firefox para que os chatbots respondam na linguagem do usuario: EX: <prompt>... reply in "<current ff language>"

EX 2: em portuguese-pt-br configured in settings

<prompt>... reply in Portuguese-PT-BR

[Mardak]

The current plan is to have localized builds use translated prompts, e.g., `Estou na página "%tabTitle%" com "%selection|12000%" selecionado. Por favor, resuma a seleção usando linguagem precisa e concisa. Use cabeçalhos e listas com marcadores no resumo, para torná-lo escaneável. Mantenha o significado e a precisão factual.` (Here I used Firefox Translate, but our usual community process will take care of it when we expose these prompts to localization.)

In the meantime, could you try changing the prompts from about:config `browser.ml.chat.prompt.prefix` and/or browser.ml.chat.prompts.0 to see if translated prompts or "reply in Portuguese pt-BR" works better for you?

[Allan-L]

I understand the plan to have variants in each language; however, I believe language models work better in the native language of their training. For example, when prompts are made in English, chatbots interpret the content better. Therefore, in my humble opinion, I would leave the body of the prompt in English, making it unnecessary to create prompts in all languages. This accelerates development and makes chatbots understand the prompt better. If I’m not mistaken, Google’s chatbot, for instance, only creates images when English is used. I tested it here, and as you can see from my screenshots, it worked perfectly by adding at the end for the bot to respond in the desired language. Congratulations and thank you for your attention.

Additionally, I would like to suggest another idea: in this submenu, an option could be added to open the chatbots without any prompt, such as:

• [Ask ChatGPT]
  • Open ChatGPT Panel
  • Summarize
  • Etc...
  •  

The other day, I was taking a test in the browser, and I had to keep switching to another tab to access the chat. If there was a side-by-side panel for asking questions, it would have helped me a lot.

One more question/idea: Have you considered having an option to send a screenshot of the open page to chatbots that support interpreting images?

[Mardak]

Indeed, there might be some trouble with various LLMs supporting the 100+ Firefox locales. Those using these localized Firefox might not like seeing English show up in the chatbot, but perhaps this is something we can test to see if the quality of chatbot interpreting content is better.

If you want to pass in the selected text without a prompt, you could create a new `browser.ml.chat.prompts.empty` string pref and put in a space:

Or if you're just wanting a fresh chatbot to ask general questions without passing in the selection, you can have the chatbot icon available with the new Sidebar experimental feature also in Firefox Labs or add the sidebar icon to your toolbar to toggle it open/close.

[Allan-L]

Hello, I enabled the "browser.ml.chat.shortcuts" option, which displays a floating menu when selecting text or performing a long click. Is this option a test for future versions? If so, corrections are needed to ensure this menu only appears when text is selected. Currently, I am using Ubuntu with the Nightly build, and my mouse's scroll wheel is faulty, so I use the scroll bars everywhere. The problem is that this menu keeps appearing constantly, whether I'm using the scroll bar or simply performing a long click without moving the mouse. I believe the best option would be for it to appear only when text is selected on the page. Thank you!

[Mardak]

Thanks for reporting. Could you try the latest Nightly 131 (20240808093537) to see if the shortcuts stop showing up with the scrollbar click or scrollwheel click? The long-press behavior should be off for now, but we'll keep this in mind as we add selection-less behaviors.

[Allan-L]

Yeah, is perfect now, you are very fast

[Allan-L]

i believe the long press option is a good idea for touchscreen to perform page action, ex: back, forward, send or share the page, send to printer etc... because in mobile theses options make more sense to longe press a page or a blank area of the page, in desktop our already have the right click menu with much more options

[Dan3000]

Please allow the usage of containers in the Sidebar. I don't want to be logged in to one of the AI API providers without isolating it to some specific container.

[Mardak]

Currently the AI Chatbot in the sidebar reuses the same regular tab container, so if you want to also use a different account for the same provider, you can open that in a different container. Do you have a particular provider you're wanting to use with multiple accounts? I currently have a personal and work ChatGPT account that I use Firefox containers to access both at the same time, but I do currently need to keep in mind which one I log into in a container vs regular tab.

Or are you not using multiple accounts and dislike how the feature reuses your logged-in state from regular tabs? At least for ChatGPT in the US, this could be useful for using the logged-out experience.

[phonics8226]

I'm not sure if I understand, are you saying that the sidebar uses the container of the tab from which it was launched?

[lei]

I think "regular tab container" means the default "no container" tab.

The Firefox Multi-Account Containers extension has an option to always open a site in a container, but it does not seem to work in the sidebar at the moment.

[mcc]

I demand a version of Firefox that does not have this feature. It makes no difference to me that the feature can be disabled, or even whether it is disabled by default. The presence of the code to run this feature on my computer is unacceptable. If you can't provide me a version of Firefox that has the AI chatbot feature *removed* (not just disabled), or in which the AI chatbot feature is an uninstallable plugin, then I am going to switch to LibreWolf or whatever fork of Firefox will remove the AI features for me. 

I. Do. Not. Consent. To AI software being installed on my computer. Period. It wasn't okay when Microsoft forced an AI client onto my computer, and it's not okay when you do it. I switched from SwiftKey to GBoard to remove the AI client, I am switching from Windows to Linux to avoid the Windows 11 AI client, I will switch from Firefox to [whatever I have to] to remove the AI client.

You should not have done this, and I cannot support you as a company until you stop.

[PleaseMozilla]

I believe LibreWolf (Firefox fork that focus' on being more private by default) might not have this feature, not entirely sure but I know I will probably look into it!

[tfunken]

As of right now, this feature is also present in Librewolf and Zen Browser.
Not sure if they will remove it in the future.

Kinda wish we had a "Unmozillad Firefox", similar to the Ungoogled Chrome.
But sad that we need that in the first place 😥

[yayroos]

librewolf came to a decision on removing it today, it'll be gone soon i expect

[dpaek91]

NO. NO NO NO NO NO NO NO! JUST NO.

AI is literally ruining the internet and is just an absolute waste of the resources it takes to keep it running.

JUST NO.

[mysteriousjelly]

Why tho, it'd help answering questions while reading articles. Short, crisp and quick on the same page. Why's everyone so against it

[missylafleur]

The biggest one is that AI doesn't -know- anything. It just takes common search results and makes a guess.  Many times those guesses are wrong.

AI is also actively taking jobs from artists and other creators. It's cheaper but does a horrible job.

There are also issues with how much power consumption it takes to develop and run AI. Many of these data centers use as much energy as a city!

AI in its current state has bad performance, is bad for creatives, and is bad for the environment.

[angengea]

My #1 concern when it comes to integrating AI into something like a browser is that I fundamentally do not trust any company to be responsible when it comes to sucking up huge quantities of user data and protecting user privacy. Almost every genAI model was trained on stolen data or data used without the permission/awareness of its creator. And the companies making these models are untrustworthy when it comes to disclosing where they get their training data and how they're using user data they collect. We've already seen incredibly invasive **bleep** like Microsoft's Recall that will feed every single thing you do or type on your computer into their AI model training pipeline. Every prompt or query you enter into an AI chatbot becomes data for it to learn from.

Not to mention the detrimental effect of massive AI datacenters on our climate. For example.

[SleepyCatten]

No. Just no. We don't want "AI" (machine learning). We don't want NFTs, crypto, or anything else that the "tech bros" think are cool. We don't want this: we want QOL things like the ability to allow us to enable dark mode (via Dark Reader) on the add-ons page! (Seriously! How do you STILL NOT have a dark mode option on that page and why can't you AT LEAST give Dark Reader or similar vetted add-ons access to it?!) Focus on bug fixes and QOL issues please!!!

[jahtnamas]

slightly offtopic but there is a way to enable dark reader on "restricted" mozilla pages, check the instructions from the dev: https://github.com/darkreader/darkreader/blob/main/README.md#enable-dark-reader-on-restricted-pages-...

[SleepyCatten]

Oh my gosh! Thank you SO much 🥰

[rolingmetal]

You are being ironic? Looking at the comments, no one seems to want this 🙂

[Lydiaa]

You're starting to get huge traction and you're doing this?? Thought you were smarter than this.  shame

[lackey]

"Last edited on 9-15-2024 (6 minutes after creation) by [admin] Jon"