We're now working on filling the remaining gaps so this can become an actual reality on the #gnome (and generally #linux / #fdo / #gnu) desktop. And what can I say - things actually work out quite nicely! While on modern Intel or AMD systems the effect is mostly about lower resource consumption, on some low-end hardware there are visible differences on what you can play fluently.

Take for example these examples of a #PinebookPro (rk3399 / v4l2 stateless) playing 4k content or the #RaspberryPi 4b (v4l2 stateful / m2m) playing 1080p@60fps (the limit of the hardware decoder) on a 2560x1440 screen. My videos probably don't fully capture the effect, but especially in the later case the difference is pretty visible). This is using the gtk video player demo with some extra patches and #gstreamer.

My hope is that we can ship some initial support for all of this in #gnome46 - if possible with the default video player. Be it Totem, ported to #gtk4, or some alternative. The goal is of course that as many apps as possible can make use of it without much hassle - be it video streaming apps, video chat apps like @dino or Fractal, #fediverse clients like Tuba or camera apps like #gnomecamera / Snapshot (for a battery friendly viewfinder - important on #linuxmobile). And of course browsers.

Making this all work is a long process and when talking to people that have been working in the field longer than me, I often get the feedback to be not too optimistic regarding time frames. As always when pushing some limits there will be driver bugs, missing APIs and loose ends all over the place. So far every device I tried needed at least some fixes in #mesa, the kernel or both. And when things break, we have to be extra careful to not impact stability.

That's why we're very fortunate that #gnome recently got funding from the #SovereignTechFund. One of the sponsored projects is to implement GL robustness in #gnomeshell / Mutter, so if the driver stumbles when trying to import some unusual buffer you won't lose your session.

My personal vision with all of this is to see #wayland desktop technologies not only catching up with what other OSs offer, but becoming leading players - just like what other FLOSS projects already archived (or are in the process of becoming) in their areas. I'm thinking of #mesa, #pipewire, #gstreamer, #systemd, the kernel of course, and many others.

Similarly, with the new free desktop accessibility architecture I started prototyping last week, my goal is for accessibility on free desktops to leap ahead of the proprietary platforms in some ways, not just catch up. See https://blogs.gnome.org/a11y/2023/10/27/a-new-accessibility-architecture-for-modern-free-desktops/