cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Share your feedback on the AI services experiment in Nightly

asafko
Employee
Employee

Hi folks, 

In the next few days, we will start the Nightly experiment which provides easy access to AI services from the sidebar. This functionality is entirely optional, and it’s there to see if it’s a helpful addition to Firefox. It is not built into any core functionality and needs to be turned on by you to see it. 

If you want to try the experiment, activate it via Nightly Settings > Firefox Labs (please see full instructions here). 

We’d love to hear your feedback once you try out the feature, and we’re open to all your ideas and thoughts, whether it’s small tweaks to the current experience or big, creative suggestions that could boost your productivity and make accessing your favorite tools and services in Firefox even easier.

Thanks so much for helping us improve Firefox!

99 REPLIES 99

DenJohn
Making moves

NO

Mte90
Making moves

The announcement is not clear that is not started yet.

Also it isn't clear if it will require a personal token for this services and what settings are available.

Try checking Nightly for updates. It should be available with 2024-06-25 builds with settings dropdown choices of 4 providers. No personal token is necessary as the chatbot will reuse your usual tab's cookies if any, e.g., ChatGPT doesn't require a login and can reuse your login or subscription for GPT-4o "omni."

nightly 2024-06-25.png

If you're comfortable with changing about:config, you can set "browser.ml.chat.provider" to any URL to open it in the sidebar, and if that page accepts passing in prompts with ?q=, the context menu functionality will work too. You can create your own prompts by adding a new String pref, e.g., "browser.ml.chat.prompts.a" set to "Tell me more"

custom prompt.png

I can confirm that works 🙂

The first thing that I notice that there isn't a way to close the vertical window...

The only way I found is to press Customize Sidebar that has an X so I can close the AI sidebar.

Thanks for the feedback. You should be able to close it in today's Nightly with the new AI chatbot icon.

marianoguerra
Making moves

will this be exposed for js developers? chrome canary experiments with gemini seem to expose a window.ai object you can use to use the capabilities from code.

 

if so please try to talk to each other and standardize an MVP 🙂

This current chatbot experiment does not make use of LLM / chat completion-type APIs, but there's a path to building a more integrated Firefox chatbot powered by local or remote inference APIs that could be exposed to developers both for the web and add-ons.

magentapuppy
Making moves

yeah keep adding this useless and privacy invasive stuff that *all* the big corps are doing, rather than adding features that actually differentiate it, particularly as a less 'BS' browser option 🙄

We included chatbot providers like HuggingChat which says: "We endorse Privacy by Design. As such, your conversations are private to you and will not be shared with anyone, including model authors, for any purpose, including for research or model training purposes."

Additionally, if you don't want to use a remotely hosted chatbot, you could use one that's on-device so data doesn't leave your computer while getting the benefits of optional AI.

AI has myriad ethical and environmental issues behind it. The first two options offered by Mozilla are created by some of the most invasive, unethical, environmentally destructive companies in the AI space. Google and Microsoft are not known for being private, not even a little bit.

The general issues behind AI, and the fact companies like HuFace need to scrape data without consent or permission, makes me question their promise to keep the data I offer them private.

These new changes appear antithetical to the Mozilla Manifesto.

I believe Mozilla should cease their attempts to integrate AI into the browser, and to reevaluate the ethical implications of promoting these products by these companies.

Unfortunately I do not count scraping people's data without consent of any kind "private by design"

And wow, it must take an awful person to only consider the 'end user' as the one deserving of privacy. Like, just, wow.

And how long will that last? How long has that *ever* lasted? Remember Don't Be Evil? Gone. We're not stupid, and I think y'all have forgotten some crucially classic blunders in tech history. If you let this in, it will be a death knell for Mozilla as a brand and Firefox as a browser. What's your plan to divest from this technology *when*, not if, they're bought out by a larger company, one that doesn't have the same respect for privacy you do? Are you prepared to make the necessary sacrifice and risk pissing off the less aware among us who don't realize the dangers of this digital gentrification? They're colonizing Mozilla, man. Don't sell the farm! Please. Pretty please don't break our hearts and lives like this. It's *going* to snowball out of your control. This pattern has played out again and again without fail since the dawn of the internet. It started with freedom and connectivity, and it ends with oppression, surveillance, and false loyalty.

Mark my words.

sure, that's a nice privacy policy for the chat itself. what about all the stuff the AI is scraping from the internet to feed itself? what about the privacy issues surrounding that?

-0
Making moves

Why does this need to be shipped with Firefox? Can't this be an extension? I want less AI, not more. 😞

This is currently a first integration of AI with plans for more optional functionality that don't require a sidebar interface, e.g., suggesting tabs that are relevant to your current activity or name for a tab group. Getting to more seamless integrations would likely be trickier as an extension and can be implemented with minimal impact for those who don't want it.

You're carrying water for the colonization of the digital space, slapping every single Firefox user in the face.

Mozilla was supposed to be better than that. How actually dare you.

duskord
Making moves

Can you also provide Claude?

You can configure a custom provider to Anthropic Claude for easy access to chat while on another tab. Currently the context menu passing in a prompt doesn't work, but we'll work with them for better compatibility.

claude sidebar.png

The latest Nightly 130 (20240712095045) includes Anthropic Claude in the list and supports passing in prompts with the context menu when logged in.

anthropic claude.png

I want kimi chat.This is chinese chat bot

TS25714
Making moves

This will absolutely need a second sidebar, the same way Edge already has (left side for tabs, right side for copilot etc, for example). Right now we would have to decide between Sidebery and similar (even the upcoming efforts to bring vertical tabs natively to FX) which is not worth it IMHO. Vertical tabs > shortcut to chatbot.

These Nightly builds have the new sidebar experience with vertical tabs in development along with the chatbot experiment. We're still working on integrating these features, but you can have both active today.

vertical tabs sidebar.png

Sidebery > native vertical tabs. Also:
1. Each sidebar extension should have their own width. It is very inconvenient to look at the result of the AI Chatbot in a small panel in which I usually have Sidebery.
2. Some sidebar extensions, like AI Chatbot, should be able to open in the overlay so that I can return to Sidebery without additional actions.
3. A setting is needed to select the language in which the chatbot should respond.

Thank you!

Iron56716
Making moves

I honestly like the AI Chatbot integration. I'm hoping Google AI Studio will be added officially.

It'd be even better if the sidebar could get official web page integrations like Edge or Floorp, That feature is very handy.

Curious, is there a particular functionality you like about Google AI Studio? The additional settings for Gemini model choice? Model tuning? Here's using it as a custom provider, but I had to manually copy/paste the prompt:

ai studio sidebar.png

There's quite a lot of things I like about Google AI Studio. Firstly, the ability to completely turn off all the safety filters, it's much easier to chat with the AI without them. Secondly, like you mentioned - the Gemini Model choice. I can change the model to any different one right in the conversation without the need to open a new chat. And of course, there's some extra perks like Custom Instructions, Model tuning, etc. Last but not least, AI Studio allows the use of Gemini 1.5 Pro without the subscription (unlike the regular Gemini websites. Although, the AI Studio version of 1.5 Pro does have a 50 turn rate limit I believe).

mcortt
Making moves

AI in sidebar is great. However, please give us some sort of keyboard shortcut to open it.

Does it need to be a keyboard shortcut or would a dedicated button to click be good for now? There's a lot of existing keyboard shortcuts, so having one for each sidebar might be too many, but could a keyboard shortcut to toggle / reopen the last opened sidebar work too?

The main goal of a web browser is to browse the web, not browse the browser UI, which is what I feel a specific button accomplishes. The button would work (the sidebar toggle button is what I currently use), but there is no keyboard shortcut to toggle it currently that I am aware of. Personally, I would prefer a keyboard shortcut for the AI chat specifically, but I am also of the thought that most things should have keyboard shortcuts, or at least be able to be assigned a keyboard shortcut (a la extension shortcuts). Though, I understand where that could be a lot of work.

My preferences in order:

  • Assignable keyboard shortcut to AI, synced tabs, etc
  • Keyboard shortcut for sidebar toggle
  • Button to open it

For now I would settle and be happy with a keyboard shortcut to toggle the sidebar to its last state.

 

BiORNADE
Making moves

You can put ChatGPT or any AI Chatting service into a sidebar, which makes the features useless and sort of a filler but... Actually, the only time that AI Chatbots in a browser was a big deal was when Microsoft Edge had BingAI, which was a free version of GPT 4.0 FE (with some improved enhancements), but it didn't last much until MS got Copilot (BingAI) in almost everywhere, so you can use it on Windows 7 now (sarcastically saying)

To test the waters with sidebar, using a known website (from time to time) is a nice idea, so the Firefox Team are able to detect the issue easier and deliver a fix earlier... But a native AI feature that works just like a sidebars is not that of an idea

But here's a nice thing; you can actually access everything just like watching a gecko video and commenting on this very forum using a simple technique.
Some more contrast in the lines, some more tests, some polish from the community, the final testing, then the final product. The technique could also extend to PWAs (the most Kudoed question still on review)

Looking ahead to an answer

Kind Regards,
BiORNADE

I'm guessing you used Google Gemini to navigate the sidebar to another page such as this Connect discussion? You could more directly do that by setting a custom provider url, but that's not the main use case of this feature.

It almost works like a Sidebar, so you can just access anything with little to no issues.

Wouldn't make sense to have ChatBot option when Sidebar gets complete, but maybe some little additions that you can make like:

[to provide ChatBOT Features] -> IF<TitleGPT is the only website marked as CHATBOT (1)> or IF<TitleGPT is open out of all marked ChatBots (2+)> == (Chatbot actions shown for TitleGPT only)
<Otherwise, if there are no Chatbots added on the sidebar> == [all features of Chatbots must be disabled]

Atleast for me, adding Chatbot (and an option to mark as) to a sidebar feature instead of a separate thing, would be better. Just don't make it so bloated, but just do some, like, simple stuff

Frisk
Making moves

Hello!

With introduction of even more AI services to Firefox I wanted to express that to me it does seem like Firefox is missing with development of features. This sentiment is echoed by a lot of people in my social bubble of technologists, ethicists and other people with same priorities as what one could think are values which Firefox was built on.

Those concerns are in my opinion very valid. The machine learning models have shown to be unreliable - just some of the recent examples from AI products made by large corporations: pointing users to eat pizza with glue, providing false information about just about anything. There are a number of ethical issues yet to be resolved with usage of AI, from its intense usage of computing resources that adds up to electric grid demands. Through privacy and possible copyright violations of datasets that power the models. To an entire bag of other issues monitored by excellent resources such as AIAAIC repository

AI/Machine learning is an amazing field with many likely applications, and yet, its recent rise to fame is characterized by failures and issues in many implementations. Personally I often don't even see whether the application of AI is truthfully necessary, in many cases a human would do a more trustworthy and fast task of information gathering than a large machine learning model.

When we compare current state of "AI", does it reflect what Firefox stands for? Does it reflect Mozilla's principals?

Let's compare.

  • Principle 2

    The internet is a global public resource that must remain open and accessible.

LLMs are known for being a black box. Depending on our definition of "open and accessible", LLMs can be a very free resource or a completely inaccessible black box of math.

  • Principle 4

    Individuals’ security and privacy on the internet are fundamental and must not be treated as optional.

It's clear that LLMs pose a privacy risk to Internet users. LLMs pose risk in at least two ways - because the data they are trained on sometimes contains private information due to negligent training process. In this case users of a learned model can possibly access private information. The second risk is of course usage of 3rd party services that may use information to infringe on privacy of users. While Mozilla in blog assures that "we are committed to following the principles of user choice, agency, and privacy as we bring AI-powered enhancements to Firefox", it's unclear how supporting such services as "ChatGPT, Google Gemini, HuggingChat, and Le Chat Mistral" helps protect Firefox user privacy. Giving users choice should not compromise their safety and privacy.

  • Principle 10

    Magnifying the public benefit aspects of the internet is an important goal, worthy of time, attention and commitment.

In my opinion in the process of designing AI functionalities on top of Firefox there was no evaluation on how those functionalities can benefit the public. There are a number of issues as mentioned above with the LLMs, they can be dangerous and work in detriment to users. Investing and supporting in technology of this type may lead to terrible consequences with little actual benefit.

In addition Mozilla claims:

  • We are committed to an internet that elevates critical thinking, reasoned argument, shared knowledge, and verifiable facts.

I argue that AI models are the opposite to that. AI output is not verifiable. They are working against sharing knowledge by making seemingly accurate information that turn out to be false.

I ask Mozilla to reevaluate impact of AI considering all of those points. I ask on behalf of myself as well as many users that I see on Fediverse being greatly worried and frustrated with AI changes added on top of Firefox. There is certainly a lot of potential greatness that could be done with AI, but those steps must be taken responsibly.

Thanks for the thoughtful feedback that many might have trouble writing up. Specifically about open and accessible, we've built the Firefox feature with transparency and interoperability with any webpage. This encourages diversity in the AI ecosystem, so perhaps competition and innovation will help make these systems less of a black box and improve on many other aspects like privacy as well.

Not everyone has your skill nor background to participate in these discussions. There's potential for these free and readily available AI chatbots to help many others universally and inclusively share their ideas and concerns. Indeed, there's a lot of potential greatness in these tools as well as responsibility in how they get used.

And this answer is a perfect example for meaningless marketing-speech and not taking serious responsibility. By honoring the commenters competency and repeating his/her arguments you are exactly saying nothing.
What will you really do to take care of those issues?
When will you do it?
How will you keep us updated about the consequences?
Us - the users and advocates of your product - we are the multiplicators: explaining, promoting and defending it - if you allow us to. The latest developments with AI and ad-tracking is chilling my motivation extremely to continue with this...

narazamsa
Making moves

If its on device only and no data collection is going to happen. Also only complete opensource models. Then yes that would be good to have.

Are there any particular open source models that you would recommend for running on device? You might be able to set it as a custom provider to chat directly from Firefox.

ilikefirefox99
Making moves

I love the features being implemented, BUT... as many point out, a second sidebar  is necessary. The idea should be same as all other browsers. You have a vertical panel with your vertical tabs in the left, and you have the sidebar (history, bookmarks, etc, etc, etc) on the right.

Trying to create a single sidebar that also has your vertical tabs on it, it's a very messy concept and a terrible poor implementation in my view. I understand that it might be tricky to implement this feature, but honestly, it's very very very necessary.

No need to reinvent the wheel, Vivaldi, and especially Microsoft Edge, already show  you the best concept:

ilikefirefox99_0-1719369058805.png

Thank you.