Video upscaling in browser video players using AMD FidelityFX or similar technology

[tonebacas]

Would be nice to be able to use image upscaling on video players, as an optional feature you could toggle for any individual video player.

[Selinth]

Yes please for RTX VSR!

[Arexodius]

@BethofDeath 

"Needing newer hardware from a specific vendor for the feature feels pretty opposed to Mozilla's philosophy towards internet accessibility."

I don't think anyone would go buy an Nvidia GPU exclusively to have FSR in Firefox just because Mozilla would add support for it, so the "needing newer hardware to use feature X" part is a bit of a moot point I think. If you have an Nvidia GPU, then you have one. If you don't have one, then you don't have one, and that's fine and that's the end of it. Firefox won't suddenly become non-inclusive just because of adding support for FSR. I'd rather argue that not having it would sooner be non-inclusive towards Nvidia GPU users. And since it's an already existing technology available on other browsers to people who happen to use Nvidia GPUs, there's no good reason (from a practical standpoint) for Mozilla not to support it. Whether there are good reasons for it is another matter. I think there are, but that's me.

From an ethical standpoint, arguments could be made for or against using/supporting AI technology and I think that would be an endless discussion, and this is not the place for that. The technology exists for various reasons, and using the technology or not using it should be up to each individual. It's also up to Mozilla to decide if they think AI technology is ethically viable or not. That's not something that will or even should be influenced in a discussion like this.

Moreover, just as a bit of general advice, you can't use the "carbon footprint" rhetoric as a universally accepted moral code in a public space. Please, be aware that man-made climate change is not a universal truth. The global scientific community is not in consensus over the matter (consensus is contrary to good science). Many experts have strong and differing opinions on the matter. It is a politically charged topic and governments and corporations are using it for influence and monetary gain. It's not an open and shut case. The science isn't "settled" as they say. Just realize that many disagree with the the concept as a whole and that you can't make arguments towards it (or use links that do so) in a public space without disregarding a whole lot of people. Don't be like the politicians.

Forums like this are not in any way meant for deciding whether it's right or wrong in an ethical sense to implement a feature. Arguments like that don't really belong here. However you did make a lot of arguments that purely relates to the efficacy of the technology, and that's always appreciated I think. I still personally think that it's less important how effective a particular technology is, and more important whether it's supported at all or not, if only to prevent Firefox users from feeling cheated out of something just because they use Firefox. But like you said, using the feature or not needs to be optional if it does become implemented. Some control over how it works would also be great if that's possible.

Lastly, I just want to affirm your efforts to be conscientious. That's a really good thing in and of itself, but one can't be dogmatic about it (especially in public) and there are more and less appropriate spaces for that.

Edit: Got my acronyms confused there. Meant VSR, not FSR 😅

[piterk]

@BethofDeath 

Hmm nvidia technology seems to work much better than simple sharpening exactly thanks to AI. AI will be spreading and will be used in everything, no matter if you worry about ecology or not. You should not be the one to care about how they power their servers. If they can afford it they probably have to use renewable sources to some degree as pure coal electricity is too pricey these days. Or they just move this cost to users, by increasing gpu cards 😉

Anyway - this is theoretical digression. Truth is that firefox will be like internet explorer if they will not introduce it - it will be final nail to this browser's coffin...

[BethofDeath]

@ArexodiusI feel like you're very confused. FSR (FidelityFX Super Resolution) is the open-source AMD version, which runs on everything, and is the solution I favor.

Firefox devs can barely get hardware-accelerated video decoding and whitelisting working, and you want to add support for proprietary code features? Like it or not, this is in opposition to Mozilla's philosophy and has been as long as Firefox has existed. Even h264 support was only added under protest, and because it was forced into the HTML5 standard.

Also, please keep your reality-denial politics off the Ideas pages, it has zero place here; or anywhere else for that matter, people are dying from the effects of climate change already.

@piterkGamer's Nexus on Ye Olde Spyware Vendor's Video Distribution Platform just did a video on nVidia's solution, and it was only significantly noticeable in motion on text. Otherwise, it was found that freeze-framing it was needed to spot a real difference. As a general-purpose thing that's always on and working... it's a waste. Let someone make a plugin if they want to justify their expenditure, imo, don't put the maintenance load on the dev team for something not everyone will be able to use.

It's not supported on AMD or Intel, it's not supported at all on Linux or the BSDs yet (if ever), and chasing a bad actor's badly-thought-out new toy is a terrible idea on the face of it.

And finally: regardless of anyone's... opinions... on my being concerned about "ecology", power consumption is a very short-and-medium-term practical problem in Europe, with the massively-inflated energy prices right now; implementing "RTX Super Video Resolution" and having it on by default is what would put the nail in the coffin for Firefox.

Constructive Suggestion: Mozilla should instead have a toolbar appear like it does for enabling and installing DRM support (yuck) to offer to enable AMD FidelityFX Super Resolution, the first time a video playing is detected as being a lower resolution than the viewport and/or display device is detected, with a "What is this and why isn't it default?" link to a page explaining the caveats.

Perhaps even have a radio button option to automatically detect whether the user is on battery or AC, and disable the extra processing if on battery. Nobody likes having the battery run out faster, after all, and this simple option allows everyone to have the optimal experience that applies to them. ...just don't make the setting/support visible to the Web itself. Fingerprinting help, that.

[Fairplex]

Please don't add this proprietary technology to Firefox or if you plan to add it, well make it OFF by default. We don't want Firefox using +200 watt of gpu power for this over hyped feature.

[axy_david]

I think it's better to use amd's solution not gpu locked not closed source. That being said it should be off by default most ppl use 1080p screens still.

[axy_david]

Tired of blurry images? Let's start using fidelityFX or relevant AI to upscale our images!

Let's end the blurry images, once and for all!

This becomes noticeable when using 4k but also 1080p sometimes as well, and let's not even talk about mobiles,  especially with PNGs.

[Zaphire]

This is a must if Edge and Chrome both have it.

[hwamil]

For individuals the electricity usage may go up, but if you think about how this could veer content creators and providers to use less network bandwidth and storage, the overall infrastructure will therefore produce less waste and use less resources in the long term.

[Star_Dust]

Lets talk practicality of these technologies, for a general user i don't think this is much helpful. Since most of the ISPs offer unlimited data plans (at least in my country)  i don't see point of saving some network bandwidth at cost of GPU usage. However for videos which are around 15-20 years ago whose resolution don't exceed 360p/480p this is extremely helpful.

Overall my opinion is this tech will be good to have it but useless in 90% of the general use cases for consumers(not content creators/commercial usage)