Add Firefox Support for RTX Video Super Resolution

[navek]

Please, work with nVidia to add support for RTX Video Super Resolution in Firefox. Nearly 80% of internet bandwidth today is streaming video, and 90% of that content streams at 1080p or lower. Many of us use displays with resolutions higher than 1080p, and Firefox uses basic upscaling techniques, which result in final images that are soft or blurry. PC users also sit close to their screens, which makes the quality of upscaling even more noticeable, and is a good argument for a higher level of processing and refinement.

I prefer to use Firefox for everything I do, but I now feel torn because the difference is so noticeable, and I stream a great deal of video. Again, please, reach out to nVidia and work with them to implement support for RTX VSR. It really makes a difference, and would eliminate any inclination I might have to use a different browser for streaming video.

[Redline]

Oh apparently there's a new entry in about:config that needs to be changed, and there's RTX HDR support now as well

gfx.webrender.overlay-vp-super-resolution
gfx.webrender.overlay-vp-auto-hdr

[Dorraj]

Driver version 551.86, FF Dev version 125.0b4, still works, it's just still very selective about what videos it wants to work on. Of course Chrome works on all videos.

To anyone else trying this: Remember to check the setting in FF config, and remember to try multiple different videos.

[Coolsnow]

First enable in about:config:
gfx.webrender.overlay-vp-auto-hdr
gfx.webrender.overlay-vp-super-resolution
gfx.webrender.super-resolution.nvidia

This simply won't work because of how firefox switches between gpu API's.

In Nvidia 3d settings, make sure Firefox.exe is on Nvidia, make sure OpenGl GPU is set to your RTX capable gpu.
Now you need to tell windows to give right Gpu endpoints, Nvidia settings page only handles request to nvidia api.

The most critical part is Windows GPU settings.
Open windows GPU settings, open the location of your firefox and select firefox.exe and set it high performance.

Don't forget to enable Super Resolution in Nvidia panel.

I don't like how windows handles GPU api's, It is what it is. And firefox seems to get low-power gpu endpoints by default. On the other hand, chromium seems to request all of your gpu.

[twcpu]

Firefox 125.0.2
Super Resolution only work if I set monitor refresh rate to 60 FPS. if I set refresh rate higher the Super Resolution just won't kick in.

[B1GN05E]

there's a Limitation at this function.

1. at least in current RTX 2xxx~4xxx core, there's a capability limitation on they're tensor core quantity.
nvenc/nvdec and they're VA unit quantity is as example but nothing much issue on capability limitations.
cause they're unlock the session limit in after certain version of driver.
also even if it reach the limit of hw performance, it just go slows, not the boom error or hang the app itself.

2.problem happened in here.
was old time, without enough capability on VA fixed unit in GPU, force Video Accelations are quite a mess all around not only in webbrowser, just a single video player were puke tons of errors.
times is come around once again for Frame Upscaling, Interpolating, Generating by ML and accelator all again.

3.all the image service and data provider handle VP8/VP9/AV1 contained in WebM as 'Much Compressed version of Animation GIF file' but in a same time, they handle those as Typical Video File
cause it's Word as Video file, everyone handle them as Video File, not only a web browser, just everyone.

4.at the same time, RTX SR and RTX HDR is designed and purpose on 'Additional Processing on Video'
not for the tons of GIF and Autoplay Short Videos on WebBrowser.
as example, they're provide early support on certain video website like Youtubes
(that usually play only 1 or 2 videos, proper limit on autoplay in single page, no need to processing to preview videos on backgrounds)
also they're Super Resolution function is only activated in FullScreen/CurrentActiveTab/CurrentFocusedVideo, and then Proper viewpoint resolution that bigger then video itself.

5.when they're RTX SR and RTX HDR override in multiful video play session on video players, it reach the performance limit soon, then just get bottleneck on tensor hw then it slows video and color get mess.
(for me, it was RTX 3070 with 50 of random resolutions medium time videos)
it might be app hang when try to close or slow the videocard itself, but at least there's no Crash at this point.

6.when firefox or edge kind web browser try to access tons of video play website, or community that has tons of autoplay webm as 'replace of GIF image' to spamming meme in comment,
it quite a bottleneck and slows the browsing either you got iGPU or Expensive dGPU.
still, it's not the cause the application crash or driver crash, or entire system hang.
but if you're turn RTX SR/RTX HDR on, access video website except youtube kind managed one,
it try to VideoAccelate, SDRtoHDR on every short loop meme and comment emoji as Video File.
suddenly go to mess that crash the entire tab, hang the browser, crash the nvidia driver and get panic the windows.

(ex: nvlddmkm \Device\Video3 \Device\00000126 \Device\0000010f \Device\0000012f )

7. we discuss only in Current Active function that handle (non 3d game) Video, nvidia RTX in Driver.
cause 3D was relatively free, Video is usually got Legal and Technical limitations (before bring A.I. things)
but Intel and AMD seems attempt simmilar approach in UpScaling, FrameInterpolation, FrameGenerating as part of Video Processing.
also Microsoft try to 'traffic control' those as API in part of they're OS.
already Windows AutoHDR intergrated and handle them as part of OS function,
DirectX SR give a flag to suggest possible option and developer choose them.

8.most of website just turn auto convert GIF as WebM(VP8/VP9/AV1) to transition, for cost reasons.
WebBrowser need to recognize and divide the 'Video as animation GIF' or 'just typical Video as Video'
cause whole driver and webbrowser just handle them same as just Video.
they bring tons of Decode call for VA, it bring power trouble on mobile devices but at least Okay State.
but when those 'ShortLoop' start, it can cause bottleneck and gpu driver panic sometime,
and after then using SR and HDR things processing on those videos, it can easily issue on handling Tensor Unit, get trouble with WebBrowser's Render(might be cause of HDR?)

9.i think this issue must be discuss between WebRender and MediaAccelation Engineer with GPU driver Provider.
as other peoples in here discussed, there's too many different user enviorment variables in this issues not just each Vendor Performance and Device RefreshRate things.
need to test more on various cases. before every hw vendor and os provider bring fragmentized thing to get spread these chaos.