topic Re: Share your feedback on the AI services experiment in Nightly in Firefox Labs

Share your feedback on the AI services experiment in Nightly

asafko — Fri, 18 Oct 2024 21:19:13 GMT

Hi folks,

In the next few days, we will start the Nightly experiment which provides easy access to AI services from the sidebar. This functionality is entirely optional, and it’s there to see if it’s a helpful addition to Firefox. It is not built into any core functionality and needs to be turned on by you to see it.

If you want to try the experiment, activate it via Nightly Settings > Firefox Labs (please see full instructions here).

We’d love to hear your feedback once you try out the feature, and we’re open to all your ideas and thoughts, whether it’s small tweaks to the current experience or big, creative suggestions that could boost your productivity and make accessing your favorite tools and services in Firefox even easier.

Thanks so much for helping us improve Firefox!

Re: Share your feedback on the AI services experiment in Nightly

DenJohn — Mon, 24 Jun 2024 23:42:58 GMT

Re: Share your feedback on the AI services experiment in Nightly

Mte90 — Tue, 25 Jun 2024 08:49:29 GMT

The announcement is not clear that is not started yet.

Also it isn't clear if it will require a personal token for this services and what settings are available.

Re: Share your feedback on the AI services experiment in Nightly

marianoguerra — Tue, 25 Jun 2024 11:22:29 GMT

will this be exposed for js developers? chrome canary experiments with gemini seem to expose a window.ai object you can use to use the capabilities from code.

if so please try to talk to each other and standardize an MVP 🙂

Re: Share your feedback on the AI services experiment in Nightly

magentapuppy — Tue, 25 Jun 2024 12:34:49 GMT

yeah keep adding this useless and privacy invasive stuff that *all* the big corps are doing, rather than adding features that actually differentiate it, particularly as a less 'BS' browser option 🙄

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 12:43:53 GMT

Try checking Nightly for updates. It should be available with 2024-06-25 builds with settings dropdown choices of 4 providers. No personal token is necessary as the chatbot will reuse your usual tab's cookies if any, e.g., ChatGPT doesn't require a login and can reuse your login or subscription for GPT-4o "omni."

If you're comfortable with changing about:config, you can set "browser.ml.chat.provider" to any URL to open it in the sidebar, and if that page accepts passing in prompts with ?q=, the context menu functionality will work too. You can create your own prompts by adding a new String pref, e.g., "browser.ml.chat.prompts.a" set to "Tell me more"

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 12:49:56 GMT

This current chatbot experiment does not make use of LLM / chat completion-type APIs, but there's a path to building a more integrated Firefox chatbot powered by local or remote inference APIs that could be exposed to developers both for the web and add-ons.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 14:03:48 GMT

We included chatbot providers like HuggingChat which says: "We endorse Privacy by Design. As such, your conversations are private to you and will not be shared with anyone, including model authors, for any purpose, including for research or model training purposes."

Additionally, if you don't want to use a remotely hosted chatbot, you could use one that's on-device so data doesn't leave your computer while getting the benefits of optional AI.

Re: Share your feedback on the AI services experiment in Nightly

-0 — Tue, 25 Jun 2024 14:19:51 GMT

Why does this need to be shipped with Firefox? Can't this be an extension? I want less AI, not more. 😞

Re: Share your feedback on the AI services experiment in Nightly

duskord — Tue, 25 Jun 2024 14:22:58 GMT

Can you also provide Claude?

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 14:50:11 GMT

You can configure a custom provider to Anthropic Claude for easy access to chat while on another tab. Currently the context menu passing in a prompt doesn't work, but we'll work with them for better compatibility.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 15:03:39 GMT

This is currently a first integration of AI with plans for more optional functionality that don't require a sidebar interface, e.g., suggesting tabs that are relevant to your current activity or name for a tab group. Getting to more seamless integrations would likely be trickier as an extension and can be implemented with minimal impact for those who don't want it.

Re: Share your feedback on the AI services experiment in Nightly

TS25714 — Tue, 25 Jun 2024 15:32:35 GMT

This will absolutely need a second sidebar, the same way Edge already has (left side for tabs, right side for copilot etc, for example). Right now we would have to decide between Sidebery and similar (even the upcoming efforts to bring vertical tabs natively to FX) which is not worth it IMHO. Vertical tabs > shortcut to chatbot.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 16:00:00 GMT

These Nightly builds have the new sidebar experience with vertical tabs in development along with the chatbot experiment. We're still working on integrating these features, but you can have both active today.

Re: Share your feedback on the AI services experiment in Nightly

Mte90 — Tue, 25 Jun 2024 16:09:17 GMT

I can confirm that works 🙂

Re: Share your feedback on the AI services experiment in Nightly

Mte90 — Tue, 25 Jun 2024 16:41:15 GMT

The first thing that I notice that there isn't a way to close the vertical window...

The only way I found is to press Customize Sidebar that has an X so I can close the AI sidebar.

Re: Share your feedback on the AI services experiment in Nightly

Iron56716 — Tue, 25 Jun 2024 17:32:49 GMT

I honestly like the AI Chatbot integration. I'm hoping Google AI Studio will be added officially.

It'd be even better if the sidebar could get official web page integrations like Edge or Floorp, That feature is very handy.

Re: Share your feedback on the AI services experiment in Nightly

mcortt — Tue, 25 Jun 2024 19:31:36 GMT

AI in sidebar is great. However, please give us some sort of keyboard shortcut to open it.

Re: Share your feedback on the AI services experiment in Nightly

BiORNADE — Tue, 25 Jun 2024 22:00:31 GMT

You can put ChatGPT or any AI Chatting service into a sidebar, which makes the features useless and sort of a filler but... Actually, the only time that AI Chatbots in a browser was a big deal was when Microsoft Edge had BingAI, which was a free version of GPT 4.0 FE (with some improved enhancements), but it didn't last much until MS got Copilot (BingAI) in almost everywhere, so you can use it on Windows 7 now (sarcastically saying)

To test the waters with sidebar, using a known website (from time to time) is a nice idea, so the Firefox Team are able to detect the issue easier and deliver a fix earlier... But a native AI feature that works just like a sidebars is not that of an idea

But here's a nice thing; you can actually access everything just like watching a gecko video and commenting on this very forum using a simple technique.
Some more contrast in the lines, some more tests, some polish from the community, the final testing, then the final product. The technique could also extend to PWAs (the most Kudoed question still on review)

Looking ahead to an answer

Kind Regards,
BiORNADE

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Tue, 25 Jun 2024 20:47:41 GMT

Curious, is there a particular functionality you like about Google AI Studio? The additional settings for Gemini model choice? Model tuning? Here's using it as a custom provider, but I had to manually copy/paste the prompt:

Re: Share your feedback on the AI services experiment in Nightly

Iron56716 — Tue, 25 Jun 2024 21:50:58 GMT

There's quite a lot of things I like about Google AI Studio. Firstly, the ability to completely turn off all the safety filters, it's much easier to chat with the AI without them. Secondly, like you mentioned - the Gemini Model choice. I can change the model to any different one right in the conversation without the need to open a new chat. And of course, there's some extra perks like Custom Instructions, Model tuning, etc. Last but not least, AI Studio allows the use of Gemini 1.5 Pro without the subscription (unlike the regular Gemini websites. Although, the AI Studio version of 1.5 Pro does have a 50 turn rate limit I believe).

Re: Share your feedback on the AI services experiment in Nightly

Frisk — Wed, 26 Jun 2024 00:28:50 GMT

Hello!

With introduction of even more AI services to Firefox I wanted to express that to me it does seem like Firefox is missing with development of features. This sentiment is echoed by a lot of people in my social bubble of technologists, ethicists and other people with same priorities as what one could think are values which Firefox was built on.

Those concerns are in my opinion very valid. The machine learning models have shown to be unreliable - just some of the recent examples from AI products made by large corporations: pointing users to eat pizza with glue, providing false information about just about anything. There are a number of ethical issues yet to be resolved with usage of AI, from its intense usage of computing resources that adds up to electric grid demands. Through privacy and possible copyright violations of datasets that power the models. To an entire bag of other issues monitored by excellent resources such as AIAAIC repository.

AI/Machine learning is an amazing field with many likely applications, and yet, its recent rise to fame is characterized by failures and issues in many implementations. Personally I often don't even see whether the application of AI is truthfully necessary, in many cases a human would do a more trustworthy and fast task of information gathering than a large machine learning model.

When we compare current state of "AI", does it reflect what Firefox stands for? Does it reflect Mozilla's principals?

Let's compare.

Principle 2
The internet is a global public resource that must remain open and accessible.

LLMs are known for being a black box. Depending on our definition of "open and accessible", LLMs can be a very free resource or a completely inaccessible black box of math.

Principle 4
Individuals’ security and privacy on the internet are fundamental and must not be treated as optional.

It's clear that LLMs pose a privacy risk to Internet users. LLMs pose risk in at least two ways - because the data they are trained on sometimes contains private information due to negligent training process. In this case users of a learned model can possibly access private information. The second risk is of course usage of 3rd party services that may use information to infringe on privacy of users. While Mozilla in blog assures that "we are committed to following the principles of user choice, agency, and privacy as we bring AI-powered enhancements to Firefox", it's unclear how supporting such services as "ChatGPT, Google Gemini, HuggingChat, and Le Chat Mistral" helps protect Firefox user privacy. Giving users choice should not compromise their safety and privacy.

Principle 10
Magnifying the public benefit aspects of the internet is an important goal, worthy of time, attention and commitment.

In my opinion in the process of designing AI functionalities on top of Firefox there was no evaluation on how those functionalities can benefit the public. There are a number of issues as mentioned above with the LLMs, they can be dangerous and work in detriment to users. Investing and supporting in technology of this type may lead to terrible consequences with little actual benefit.

In addition Mozilla claims:

We are committed to an internet that elevates critical thinking, reasoned argument, shared knowledge, and verifiable facts.

I argue that AI models are the opposite to that. AI output is not verifiable. They are working against sharing knowledge by making seemingly accurate information that turn out to be false.

I ask Mozilla to reevaluate impact of AI considering all of those points. I ask on behalf of myself as well as many users that I see on Fediverse being greatly worried and frustrated with AI changes added on top of Firefox. There is certainly a lot of potential greatness that could be done with AI, but those steps must be taken responsibly.

Re: Share your feedback on the AI services experiment in Nightly

aminought — Wed, 26 Jun 2024 01:44:12 GMT

Sidebery > native vertical tabs. Also:
1. Each sidebar extension should have their own width. It is very inconvenient to look at the result of the AI Chatbot in a small panel in which I usually have Sidebery.
2. Some sidebar extensions, like AI Chatbot, should be able to open in the overlay so that I can return to Sidebery without additional actions.
3. A setting is needed to select the language in which the chatbot should respond.

Thank you!

Re: Share your feedback on the AI services experiment in Nightly

narazamsa — Wed, 26 Jun 2024 02:23:37 GMT

If its on device only and no data collection is going to happen. Also only complete opensource models. Then yes that would be good to have.

Re: Share your feedback on the AI services experiment in Nightly

ilikefirefox99 — Wed, 26 Jun 2024 02:31:25 GMT

I love the features being implemented, BUT... as many point out, a second sidebar is necessary. The idea should be same as all other browsers. You have a vertical panel with your vertical tabs in the left, and you have the sidebar (history, bookmarks, etc, etc, etc) on the right.

Trying to create a single sidebar that also has your vertical tabs on it, it's a very messy concept and a terrible poor implementation in my view. I understand that it might be tricky to implement this feature, but honestly, it's very very very necessary.

No need to reinvent the wheel, Vivaldi, and especially Microsoft Edge, already show you the best concept:

Thank you.

Re: Share your feedback on the AI services experiment in Nightly

ksetlak — Wed, 26 Jun 2024 10:11:41 GMT

I have one observation: I'm logged in to Google on my work laptop and my work Google account is the primary one, while the private one is the secondary. However, I only use Gemini with my private account. I can't change the account I'm using in the side-docked window, because when I attempt to do that, a new tab is opened in my main Firefox window. Would be lovely if this was configurable and `authuser=1` could be added to the params in the URL used to open the side pane.

Re: Share your feedback on the AI services experiment in Nightly

0x4d6165 — Wed, 26 Jun 2024 16:29:32 GMT

Actually listen to your userbase for *once* when we say we do NOT want this! Be a good web browser stop trying to be edge.

Re: Share your feedback on the AI services experiment in Nightly

0x4d6165 — Wed, 26 Jun 2024 16:30:54 GMT

if you're gonna use AI crap anyway why not just use edge lol

Re: Share your feedback on the AI services experiment in Nightly

Jon — Wed, 26 Jun 2024 17:50:13 GMT

Thanks for the feedback! For those interested, there's another idea thread for a second sidebar here:

Re: Share your feedback on the AI services experiment in Nightly

Reycko — Wed, 26 Jun 2024 22:32:39 GMT

Would be nice to have a nice UI to add custom prompts if that's not in the works already

Re: Share your feedback on the AI services experiment in Nightly

aminought — Wed, 26 Jun 2024 22:40:01 GMT

You can! Just edit "browser.ml.chat.prompt.prefix" and "browser.ml.chat.prompts.*" prefs. Also, you can add additional prompts like "Answer Questions" if you add "browser.ml.chat.prompts.3", "browser.ml.chat.prompts.4", etc.

Re: Share your feedback on the AI services experiment in Nightly

myspace — Thu, 27 Jun 2024 12:48:32 GMT

Thanks going to try it out now!

Re: Share your feedback on the AI services experiment in Nightly

FastBrowser2022 — Thu, 27 Jun 2024 15:34:11 GMT

I tried using the right click > Simplify text option with Gemini. It worked the first time, but the 2nd time Gemini did a bot check. It otherwise worked fine after completing it.

I also tested it with Chat-GPT 4o, this worked for summarizing the same text and simplifying content.

However I noticed there is no context menu option to simply highlight text and re-write text in an unsubmitted text input field

Re: Share your feedback on the AI services experiment in Nightly

lackey — Thu, 27 Jun 2024 23:49:48 GMT

AI has myriad ethical and environmental issues behind it. The first two options offered by Mozilla are created by some of the most invasive, unethical, environmentally destructive companies in the AI space. Google and Microsoft are not known for being private, not even a little bit.

The general issues behind AI, and the fact companies like HuFace need to scrape data without consent or permission, makes me question their promise to keep the data I offer them private.

These new changes appear antithetical to the Mozilla Manifesto.

I believe Mozilla should cease their attempts to integrate AI into the browser, and to reevaluate the ethical implications of promoting these products by these companies.

Re: Share your feedback on the AI services experiment in Nightly

lackey — Thu, 18 Jul 2024 00:47:01 GMT

Considering the privacy problems, corporate centralization, and environmental issues caused by the companies represented in Mozilla's AI chatbot integration, I have some potential recommendations, ranked from most to least palatable.

Entirely discontinue this experiment and stop pursuing AI.
Cannibalize the work done for integrating these 4 proprietary websites into Firefox, and use it for allowing users to add any website of their choice to the sidebar (like Floorp, Edge, or Vivaldi).
Continue developing AI, but as a browser extension that is not shipped with Firefox.
Continue developing AI, and ship it as a browser extension that does come with Firefox, but can be fully erased by users if they so choose.

Re: Share your feedback on the AI services experiment in Nightly

magentapuppy — Fri, 28 Jun 2024 14:26:11 GMT

Unfortunately I do not count scraping people's data without consent of any kind "private by design"

And wow, it must take an awful person to only consider the 'end user' as the one deserving of privacy. Like, just, wow.

Re: Share your feedback on the AI services experiment in Nightly

advpod — Fri, 28 Jun 2024 15:27:50 GMT

No.

Stop adding dumb trendy features. Spend engineering effort on integrating features that have nothing to do with browsing the web into a web browser. If I wanted to use a chatbot, I would install a chatbot app onto my computer. Features that have nothing to do with browsing the web have no place being in a web browser.

Re: Share your feedback on the AI services experiment in Nightly

eladio77 — Sun, 30 Jun 2024 06:54:30 GMT

Amazing feature. I find the funtion, but when i have active the vertical tabs i can not see the Boton "Chat IA", only i can see it without "Vertical Tabs" active. Can you add the shortcut access to the Vertical Tab menu?

Re: Share your feedback on the AI services experiment in Nightly

nclm — Sun, 30 Jun 2024 15:27:54 GMT

Please Mozilla, don’t get lost. AI chatbots is a very silly feature, and it makes it hard to take you seriously. I know investors love it these days when tech companies integrate anything “AI” in their product, but you’re Mozilla, we all expect you to behave differently and for the good of the users and the web. There is no demand for AI chatbots, only unsolicited offer from the tech world, please don’t be part of this! There is nothing ethical about about offering a feature that is consistently giving misinformation based on stolen data processed on energy and water wasting machines. It would be lovely if you could focus your ressources on actually improving the browser and on communication campaigns to get more users, not on jumping on the latest fad. Thank you ❤️

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Tue, 02 Jul 2024 18:19:41 GMT

I want the AI chatbot feature, it's free, unlimited and no account login, and there are no geographical restrictions, and I don't want to be blocked because I'm in China

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Tue, 02 Jul 2024 18:26:26 GMT

It would be even better if AI could be used for search, enabling an open-source AI model locally, and when searching, the AI aggregates the search results and answers them in the form of a human conversation, just like a chatbot

Re: Share your feedback on the AI services experiment in Nightly

piradata — Wed, 03 Jul 2024 15:49:49 GMT

Loved the feature, I always have a open tab just for chatgpt, so its a big win, and its nice that it can be disabled or opt in/out for people that don't like it.

What I miss is just a way to simply open the chat on the sidebar, the only way I found is to right click and chose one option in 'Summarize', 'Simplify Language' or 'Quiz Me', but neither of these options can be used to just open a plain clear new chat, and if I close the sidebar, there is no way to open it again on the previous conversation.

What i would suggest, is simply a button on the bookmark toolbar, to simply open close the chat, that would suffice already 🙂

Re: Share your feedback on the AI services experiment in Nightly

piradata — Wed, 03 Jul 2024 16:01:21 GMT

these two last ones are not the same? would be just a new extension on the mozilla store, like adguard or multiaccount containers, simply tab groups, etc

Re: Share your feedback on the AI services experiment in Nightly

francois79 — Fri, 05 Jul 2024 05:22:25 GMT

Hey

Thanks for the work done.

I do not understand why you plug only onto private llm services where some of them are using private llm

My ideal is to be able to connect to llm model server such as https://github.com/ollama/ollama
Or https://github.com/vllm-project/vllm
Best through proxies https://github.com/BerriAI/litellm

Is it possible?

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 06:31:06 GMT

If you're referring to the new "sidebar.revamp" (with "sidebar.verticalTabs"), you should now have the AI chatbot icon available as shortcut access in today's Nightly.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 06:47:56 GMT

A quick test of this text input field for replying 📝 seems to work for selecting the text and passing it to the chatbot 🤖. However the current sidebar integration requires copy/pasting the result back to the original text input field 📋.

What were you wanting to do with this if it was more seamless? 🤔

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 06:53:18 GMT

Any suggestions on what's a nice UI for adding custom prompts? The current way with about:config preferences is more for advanced testers, but it could work for you in the meantime. Or if not, what prompts were you wanting to add and perhaps we can add it to the default list.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:10:43 GMT

Google does support multiple active accounts where in your case https://gemini.google.com/u/0/ is probably your primary work account and https://gemini.google.com/u/1/ is your private account. Currently, the sidebar defaults to the first account logged in, so if you could sign in first with your private account, that should get it to work.

Firefox also supports Container Tabs, which is useful for sites that don't normally support multiple accounts. If you logged in to your work Google account with a "Work" container while your private account is logged in to a regular tab (not container), this might work for you too.

Alternatively, you could specify a custom provider for the desired Gemini url to get that in the sidebar for simple chatting, but currently Gemini doesn't support passing in prompts when customized this way.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:15:01 GMT

Thanks for the feedback. You should be able to close it in today's Nightly with the new AI chatbot icon.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:20:36 GMT

Are you able to download and run a https://llamafile.ai as that would be free, unlimited and no account login. It runs on your computer, so you don't need internet access to chat, but it could be slow depending on your hardware.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:26:04 GMT

Are you suggesting the chatbot to look at the search result page content or would you expect Firefox/chatbot visit some linked pages for you? There are AI services that do this like https://perplexity.ai that you could configure as a custom provider and also supports passing in your custom prompts.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:36:29 GMT

You can Customize toolbar to add a sidebar toggle button, and the chatbot will open a plain new chat. If you are logged in to some chat providers, you should be able to access previous conversations as well. Potentially Firefox could restore the last conversation, but others might prefer a clear chat, so maybe we could restore when reopening as well as provide a button to start new?

The improved sidebar also has a button to open/close the AI chatbot (as well as other sidebar tools).

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:43:04 GMT

You can configure a custom provider to any url including those running on a local server such as your own computer. At a glance, it looks like there's various web UIs for ollama that can be separately installed, so you could at least have a private chatbot in the sidebar, but I'm not sure if any of them currently accepts passing in prompts. https://llamafile.ai is another llm model server that has a built in web chat interface that accepts ?q=prompt urls.

Potentially Firefox can directly interface with these services to render a custom Firefox chatbot or power non-chatbot experiences instead of currently requiring a web chat interface.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 07:48:48 GMT

Are there any particular open source models that you would recommend for running on device? You might be able to set it as a custom provider to chat directly from Firefox.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 08:01:19 GMT

Does it need to be a keyboard shortcut or would a dedicated button to click be good for now? There's a lot of existing keyboard shortcuts, so having one for each sidebar might be too many, but could a keyboard shortcut to toggle / reopen the last opened sidebar work too?

Re: Share your feedback on the AI services experiment in Nightly

mcortt — Fri, 05 Jul 2024 08:25:51 GMT

Very nice! But please add a keyboard shortcut to open it as well.

Edit: I didn't see where you had replied to my earlier suggestion. Please disregard, sorry!

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 05 Jul 2024 08:04:07 GMT

I'm guessing you used Google Gemini to navigate the sidebar to another page such as this Connect discussion? You could more directly do that by setting a custom provider url, but that's not the main use case of this feature.

Re: Share your feedback on the AI services experiment in Nightly

mcortt — Fri, 05 Jul 2024 08:20:07 GMT

The main goal of a web browser is to browse the web, not browse the browser UI, which is what I feel a specific button accomplishes. The button would work (the sidebar toggle button is what I currently use), but there is no keyboard shortcut to toggle it currently that I am aware of. Personally, I would prefer a keyboard shortcut for the AI chat specifically, but I am also of the thought that most things should have keyboard shortcuts, or at least be able to be assigned a keyboard shortcut (a la extension shortcuts). Though, I understand where that could be a lot of work.

My preferences in order:

Assignable keyboard shortcut to AI, synced tabs, etc
Keyboard shortcut for sidebar toggle
Button to open it

For now I would settle and be happy with a keyboard shortcut to toggle the sidebar to its last state.

Re: Share your feedback on the AI services experiment in Nightly

SpidFightFR-Dev — Sat, 06 Jul 2024 08:38:03 GMT

Hey guys, i just tried to nightly version, with AI stuff.

Honestly it's good, but it would better fit the standards of privacy Mozilla displays if it was possible to download and run models (E.G: 7b llama3 or Mixtral models) from the web browser (considering the pc running firefox is powerful enough). In a similar fashion of GPT4All.

It would allow for a less-powerfull yet completely offline model execution.

Just food for thoughts, it's similar to the "localhost" approach with locallama, it's not very practical on windows for example, it would require a docker instance running on wsl to make something clean.

Peace - Spid. ✌️

Re: Share your feedback on the AI services experiment in Nightly

eladio77 — Sat, 06 Jul 2024 12:55:20 GMT

Thanks.. is working ...

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Sat, 06 Jul 2024 18:26:22 GMT

I want the chatbot to look at the content of the search results page, summarize it based on the content, and answer in human language

Re: Share your feedback on the AI services experiment in Nightly

BiORNADE — Sat, 06 Jul 2024 22:16:54 GMT

It almost works like a Sidebar, so you can just access anything with little to no issues.

Wouldn't make sense to have ChatBot option when Sidebar gets complete, but maybe some little additions that you can make like:

[to provide ChatBOT Features] -> IF<TitleGPT is the only website marked as CHATBOT (1)> or IF<TitleGPT is open out of all marked ChatBots (2+)> == (Chatbot actions shown for TitleGPT only)
<Otherwise, if there are no Chatbots added on the sidebar> == [all features of Chatbots must be disabled]

Atleast for me, adding Chatbot (and an option to mark as) to a sidebar feature instead of a separate thing, would be better. Just don't make it so bloated, but just do some, like, simple stuff

Re: Share your feedback on the AI services experiment in Nightly

Anonymous — Fri, 12 Jul 2024 17:55:10 GMT

This whole thing flies in the face of the Mozilla Manifesto.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 17:33:37 GMT

The latest Nightly 130 (20240712095045) includes Anthropic Claude in the list and supports passing in prompts with the context menu when logged in.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 17:50:06 GMT

Something like this?

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 18:00:46 GMT

I believe GPT4All exposes an inference http API but not a chatbot page that could be shown in the Firefox sidebar. Currently Firefox is rendering (potentially local) server-provided html in the sidebar, but we could build our own Firefox chat interface to show the results of inference -- potentially handled directly within Firefox similar to translations and alt-text or a server of your choice such as GPT4All running your preferred model.

Are there any particular use cases that you think must be handled with a local model?

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 18:14:22 GMT

Are there privacy-focused models that you would recommend? This chatbot feature is compatible with locally running llamafile with models that can be trained on data you find acceptable. Even if the quality of these models aren't comparable yet, hopefully we can help others try them out and contribute to these efforts so that we can get them to a quality level to include by default in Firefox.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 18:29:06 GMT

Have you found any uses of AI chatbots for your own situation? It sounds like you correctly don't trust the responses, and others with similar concerns can still find value with these tools such as entertainment.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 19:21:34 GMT

How many people install separate chatbot apps vs access from a web browser? It seems like many people who do use chatbots are already using a web browser and likely manually copy/pasting information from tab to tab.

Your point of non-browser usage is still interesting, so could you provide some more concrete examples?

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 19:37:43 GMT

It sounds like the primary concern is based on the initial list of companies providing the chatbots. Are there other companies that have better privacy, corporate, environmental aspects that should be included? We've since added Anthropic and will add more choices that people want.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Fri, 12 Jul 2024 20:37:22 GMT

Thanks for the thoughtful feedback that many might have trouble writing up. Specifically about open and accessible, we've built the Firefox feature with transparency and interoperability with any webpage. This encourages diversity in the AI ecosystem, so perhaps competition and innovation will help make these systems less of a black box and improve on many other aspects like privacy as well.

Not everyone has your skill nor background to participate in these discussions. There's potential for these free and readily available AI chatbots to help many others universally and inclusively share their ideas and concerns. Indeed, there's a lot of potential greatness in these tools as well as responsibility in how they get used.

Re: Share your feedback on the AI services experiment in Nightly

ffffff — Sat, 13 Jul 2024 00:31:06 GMT

I believe the bigger issue to people isn't a perceived lack of ability to use models that are:

Locally running
Open source
Trained on data you find acceptable

Rather, the core issue is that it seems Mozilla is promoting and incentivizing people to run models that are:

Running on computers they don't own and have no control over
Closed, proprietary, and in the hands of companies with horrible reputations
Trained on data that people often find unacceptable or believe was unethically sourced

Furthermore, these models also:

Consistently fail at delivering information in a safe and trustworthy manner
Save data you provide them to use in furthering their objectives at the cost of user privacy

Even the Mozilla manifesto—I know it's not a rigid set of internal rules or anything, but just for the sake of argument—boldly states:

4. Individuals’ security and privacy on the internet are fundamental and must not be treated as optional.

Throwing unaware users at OpenAI doesn't seem like a good way of protecting their privacy.

Then there are additional concerns, such as the tight coupling with the browser (due to reasonable limitations or not is beside the point), or the fact that Mozilla is, in a manner, endorsing and enabling these companies and their practices.

So it's understandable that this feature just isn't right to a lot of people.

I understand Firefox must remain a competitive browser if it wants to regain market share. I am aware that many are implementing similar features, and nobody wants to fall behind. I know that many users are, in fact, quite excited about this feature. Maybe the developers are, as well. Hell, even I admit I'd love to have a somehow technomagically ethically-sourced and ran LLM helping me around the web.

However, I imagine this is a very sensitive topic for a significant amount of people. There is a lot of frustration with current AI trends and big tech, especially among the most tech-aware users. I think it's safe to assume that if this reaches stable as it is, there will be much valid complaining and criticism towards Mozilla. Either you already knew this, or you didn't think far enough ahead.

Sadly, I'm not sure what's the right answer here. I sincerely hope you folks manage to figure it out.

But asking...

@Mardak wrote:
Are there privacy-focused models that you would recommend?

...with all respect, seems very tone-deaf considering all this.

Re: Share your feedback on the AI services experiment in Nightly

Mardak — Sat, 13 Jul 2024 01:28:44 GMT

I asked about privacy-focused models because there was a specific comment about training data before it was edited and made anonymous.

Re: Share your feedback on the AI services experiment in Nightly

ffffff — Sat, 13 Jul 2024 01:51:55 GMT

I see, I apologize for that, Mardak. Seems I assumed much.

Despite that, I hope the rest of my comment still provided some value. I was going to post something like it regardless, and just happened to read your response on my way there. The main point, of promotion, is my main concern with the feature.

Re: Share your feedback on the AI services experiment in Nightly

thomy2000 — Sat, 13 Jul 2024 10:37:24 GMT

Please stop destroying Firefox, pretty please.

Also, this is already possible. It's called tabs and copy paste. Those who want to use plagiarism machines can do it that way.

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Sat, 13 Jul 2024 12:48:51 GMT

yes

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Sat, 13 Jul 2024 12:56:17 GMT

Firefox can now have multiple chatbots, but none of them can be used in China, so I recommend adding chatbots from China. For example, ERNIE bot, the Chinese name of this bot is called “文心一言”. And the other one is Kimi Chat. Their official website: Kimi Chat • SmartAI 文心一言 (baidu.com)

Re: Share your feedback on the AI services experiment in Nightly

danyeaw — Sat, 13 Jul 2024 13:02:48 GMT

As a user, I want a great performant web browser, I don't want LLM features integrated at all. If Firefox wants to work on this, it should only be offered as an extension.

Re: Share your feedback on the AI services experiment in Nightly

aminought — Sat, 13 Jul 2024 15:31:53 GMT

People are also plagiarism machines. You and me. So what? Mozilla must go to such experiments, otherwise it will be left out. This is progress, and the people who prevent it are no better for me than green activists: they do nothing themselves and do not let others do it.

Re: Share your feedback on the AI services experiment in Nightly

thomy2000 — Sat, 13 Jul 2024 16:58:22 GMT

🐑

Re: Share your feedback on the AI services experiment in Nightly

facni — Sat, 13 Jul 2024 19:18:38 GMT

It would be nice if we were able to do right-click over the chatbot's panel to, for example, copy, paste and translate text or take screenshots.

Re: Share your feedback on the AI services experiment in Nightly

AdventureGamer — Sun, 14 Jul 2024 16:32:45 GMT

NO. No, no, no, no, NO. NO AI in the last bastions of privacy on the internet!!! What the hell!!!!!!!!

Re: Share your feedback on the AI services experiment in Nightly

AdventureGamer — Sun, 14 Jul 2024 16:43:55 GMT

What is entertaining about weakening the privacy standards of one of the only browsers worthy of respect in the whole internet? I love you Firefox. Please, please, please don't ruin what you have. You'll force me off my only & favorite browser. To be frank, if Firefox becomes Chrome with a coat of paint, I'll bail I swear to god - and I won't be the only one. What will investors say then? You're the last bastion of integrity in the whole game. Don't throw that away...

Re: Share your feedback on the AI services experiment in Nightly

AdventureGamer — Sun, 14 Jul 2024 16:55:07 GMT

And how long will that last? How long has that *ever* lasted? Remember Don't Be Evil? Gone. We're not stupid, and I think y'all have forgotten some crucially classic blunders in tech history. If you let this in, it will be a death knell for Mozilla as a brand and Firefox as a browser. What's your plan to divest from this technology *when*, not if, they're bought out by a larger company, one that doesn't have the same respect for privacy you do? Are you prepared to make the necessary sacrifice and risk pissing off the less aware among us who don't realize the dangers of this digital gentrification? They're colonizing Mozilla, man. Don't sell the farm! Please. Pretty please don't break our hearts and lives like this. It's *going* to snowball out of your control. This pattern has played out again and again without fail since the dawn of the internet. It started with freedom and connectivity, and it ends with oppression, surveillance, and false loyalty.

Mark my words.

Re: Share your feedback on the AI services experiment in Nightly

AdventureGamer — Sun, 14 Jul 2024 17:01:09 GMT

You're carrying water for the colonization of the digital space, slapping every single Firefox user in the face.

Mozilla was supposed to be better than that. How actually dare you.

Re: Share your feedback on the AI services experiment in Nightly

AdventureGamer — Sun, 14 Jul 2024 17:03:37 GMT

#FireEdge

Re: Share your feedback on the AI services experiment in Nightly

Emmic — Mon, 15 Jul 2024 04:55:38 GMT

when are we going to get a post like this on the new toolbar and url box thing that's just come to nightly? because I have many thoughts about that for firefox to pretend to care about....

Re: Share your feedback on the AI services experiment in Nightly

Emmic — Mon, 15 Jul 2024 04:57:40 GMT

sure, that's a nice privacy policy for the chat itself. what about all the stuff the AI is scraping from the internet to feed itself? what about the privacy issues surrounding that?

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Mon, 15 Jul 2024 07:21:30 GMT

I want kimi chat.This is chinese chat bot

Re: Share your feedback on the AI services experiment in Nightly

KellyClowers — Mon, 15 Jul 2024 21:55:15 GMT

The local PDF image alt text generation is an example of a good and useful "AI" feature.
This "AI chatbot" stuff is a an example of a bad and stupid "AI" feature. Not many people want this, those that do can install an extension or something. The LLMs use too many resources, and for at best very little benefit. Most of the time there is no or even negative benefit. This is not something Mozilla should be using or promoting. Granted, it is a fad that will be over in a couple years anyway most likely, but even in the mean time you shouldn't be supporting it.

Re: Share your feedback on the AI services experiment in Nightly

ffffff — Mon, 15 Jul 2024 22:52:58 GMT

Yes! I was so happy when I saw the alt text generation plans, because it was the first time any tech organization I follow announced a use of AI that not only was I not disappointed with, but was actually excited for. It showed that there are plenty yet unexplored and helpful use cases of AI that don't involve the endorsement and promotion of unethical big tech companies and their products.

Re: Share your feedback on the AI services experiment in Nightly

Honeebunny — Tue, 16 Jul 2024 02:34:49 GMT

Booooo!! I expected better from Firefox, but you're just proving that companies as a whole don't have or care about ethics. Disgusting.

Re: Share your feedback on the AI services experiment in Nightly

MarryAurora12 — Tue, 16 Jul 2024 06:49:17 GMT

best

Re: Share your feedback on the AI services experiment in Nightly

phreaknerd — Tue, 16 Jul 2024 09:32:30 GMT

And this answer is a perfect example for meaningless marketing-speech and not taking serious responsibility. By honoring the commenters competency and repeating his/her arguments you are exactly saying nothing.
What will you really do to take care of those issues?
When will you do it?
How will you keep us updated about the consequences?
Us - the users and advocates of your product - we are the multiplicators: explaining, promoting and defending it - if you allow us to. The latest developments with AI and ad-tracking is chilling my motivation extremely to continue with this...

Re: Share your feedback on the AI services experiment in Nightly

meias — Wed, 17 Jul 2024 15:01:08 GMT

Absolutely not. There was a post from Proton on mastodon recently that said something to the effect of AI is inevitable. I take problem with such a stance as it really does not have to be inevitable. On the contrary, the only inevitable thing about AI is that it will inevitably cause harm. Stealing information to train the model, causing massive amounts of environmental damage due to the resources needed to power it. This is not the path we need to go down.

I understand Mozilla is trying it's best to stay relevant, and to bring people back to Firefox. But we shouldn't be trying to stoop to the levels of the people we are purporting to be better than. A true genuine focus on the mission to serve people's needs, not the needs of corporate investors, is how we achieve that.

Re: Share your feedback on the AI services experiment in Nightly

lackey — Thu, 18 Jul 2024 00:45:06 GMT

My greatest concern is that Mozilla is engaging in this at all, wasting precious employee time and resources on a feature that users didn't request. I realize you probably had nothing to do with this, but I hope whoever made that decision gets to see the community outcry.

I think others have expressed my lesser concerns better, such as saying that LLM models abandoned privacy for the people whose data they scrape, which goes against the Mozilla manifesto (that privacy must not be treated as optional, in a way that is not limited to the users of a particular browser). Sure, users can download a pre-compiled language model, but the sourcing of that model also needs to be ethical, which is something that Mozilla has realized and attempted to do in the past with their Common Voice project.

Mozilla is at its best when it's creating trends, not following them. Microsoft and Google are going to win at the big data game. They always have. And that's all AI is.

Re: Share your feedback on the AI services experiment in Nightly

lackey — Thu, 18 Jul 2024 00:48:17 GMT

I thought maybe if they really wanted to bundle this into the browser, that it should simply be an extension that can be deleted. The same way Pocket and Fakespot were previously just extensions.

Re: Share your feedback on the AI services experiment in Nightly

ffffff — Thu, 18 Jul 2024 03:33:46 GMT

Not a big fan either, but in fairness, working directly with the browser is probably quite the shortcut and may allow access to functionality that extensions aren't able to use, for better or for worse, leading to a more unified design and user experience on the whole. I like the translations being built-in instead of an extension. If I liked the AI stuff, I'd probably prefer it this way as well.

Re: Share your feedback on the AI services experiment in Nightly

cannoness — Thu, 18 Jul 2024 04:23:52 GMT

I have no use for this feature; I just ran away from Chrome and back to Firefox recently over AI/privacy and it looks like I'll be leaving again for a browser not falling into the hype. I am saying this as someone who writes AI Software professionally, not a "change resistant user." Consider your audience.

Re: Share your feedback on the AI services experiment in Nightly

wutongtaiwan — Thu, 18 Jul 2024 06:24:41 GMT

You can learn about Brave's AI features.After entering the search term in the address bar, the AI option will appear,After clicking, a chatbot on the side will appear to reply to the content related to the search term,I recommend using Mozilla's own AI,I don't trust other companies' AI,I believe Mozilla can better protect privacy,And this AI should have no chat limit、No login required、No geographical restrictions,Chat history is only stored on the local computer

topic Re: Share your feedback on the AI services experiment in Nightly in Firefox Labs

Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Principle 2

Principle 4

Principle 10

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly

Re: Share your feedback on the AI services experiment in Nightly