You are here

Planet GNOME

Subscribe to Feed Planet GNOME
Planet GNOME - https://planet.gnome.org/
Përditësimi: 7 orë 46 min më parë

Alexander Mikhaylenko: GNOME and gestures, Part 2: HdyLeaflet

Dje, 15/09/2019 - 5:01md

This is part 2 of a mini-series. Part 1.

Shortly after the WebKit gesture was merged, I started experimenting with making this gesture more widely available. The first step was porting it to Vala and decoupling from WebKit. Since I wrote this part of the gesture tracker from scratch anyway, it was simple and straightforward. The resulting playground project was also used as a convenient place to quickly iterate on the WebKit gesture itself. Later I also reimplemented rendering to match the WebKit one. Here’s how it looked at various points of time:

Other than that, I used the swipe tracker to make a few more demos for Tobias Bernard:

Check out his GUADEC talk showcasing the second demo! :)

At the same time, I started integrating it into libhandy by supporting back/forward swipe in HdyLeaflet. And there I hit four problems:

1. Transitions and visible-child

A folded HdyLeaflet, just like GtkStack, shows one of its children at any given moment, even during child transitions. The second visible child during transitions is just a screenshot. But which child is “real” and which is a screenshot? Turns out the real child is the destination one, meaning the widget switches its visible child when the animation starts. It isn’t a problem if the animation is quick and time-based, but becomes very noticeable with a gesture. Additionally, it means that starting and cancelling a gesture switches the visible child two time.

One solution would be only switching the visible child at the end of the animation (or not at all if it was canceled). The problem is that it’s a major behavior change: applications that listen to visible-child to know when to update the widgets, or sync the property between two leaflets will break.

Another solution would be to draw both children during transitions, but it still means that visible-child changes two times if the gesture was canceled. The problem here is similar: applications wouldn’t expect the other child to still be drawn, but at least it’s just a visual breakage. And it still means that starting and canceling the gesture would mean two visible-child changes.

The second solution may sound better, and yet the current WIP code uses the first one.

2. Visuals

Leaflet had many issues in this area, such as over transition not making sense spatially and bottom widget being visible through the top widget. Additionally, Adrien liked the drop shadow and dimming in WebKit and the demo and wanted to have it in leaflet as well. :)

The first issue was solved by splitting the transition into over and under and clipping the bottom child. Similarly, I implemented shadow and dimming, though it’s pending on those transition types for mode transitions being merged first, so that the shadow can also be added to those, so that it’s consistent.

I’m also not happy with how the dimming and shadow are implemented, neither here nor in WebKit: it’s custom drawing with hardcoded values. Ideally, this needs to be controlled from CSS somehow. GTK itself uses gadgets for things like this (for example, the overshoot effect in GtkScrolledWindow), but that API is private. Having dimming and drop shadow widgets is an overkill, at least until GTK 4 arrives and makes GtkWidget instantiable. Maybe foreign drawing could work…

3. Syncing animation

Often, GTK applications have two leaflets: one in the window’s content area and one in titlebar. Their visible child is always changed at the same time, so it looks like they are one tall leaflet spanning both titlebar and content. This still needs to work with the gesture. And while it’s easy to make nice-looking throwaway demos that do this, syncing actual HdyLeaflets has to be a proper API.

Initially I suggested what I thought was a nice solution with having swipe tracker as a public object and connecting multiple widgets to it. Benjamin Otte and other people immediately pointed out many problems with it, so I researched how other platforms do it. The answer is simple: most platforms don’t. :)

Android has a rather silly way to sync multiple widgets together, but it’s rarely needed, as app bars are just widgets, so they can be packed into a ViewPager without a need to sync two pagers together.

Another constraint is that the solution must not expose animation progress as a write-able property, so it must not be possible to set this value to something arbitrary and get the transition stuck.

4. Interaction with GtkScrolledWindow

GTK event propagation works in two phases: capture and bubble. Widgets can connect to event signal and receive events on bubble phase. Then they return a value to either stop the event or propagate it further. More recently, GTK added various event controllers that allow choosing the phase where they run. With GTK_PHASE_CAPTURE it’s possible to handle events on the capture phase… But their signals don’t support fine-grained stopping/propagation, i.e. don’t have return values (they do in GTK4 though).
All in all, it means that there’s no way to get an event on capture phase and stop it atbitrarily…

Except there is, it’s private and it’s used by GtkScrolledWindow. This widget captures scroll events and stops some of them. For example, if the scrolled window has a vertical scrollbar, but not horizontal, it stops vertical scrolling events and propagates horizontal scrolling. This is harmless, but it also always stops events with is_stop set to TRUE, meaning a leaflet containing a GtkScrolledWindow will get stuck at the end of the gesture. So every single way of receiving events fails in a different and exciting way.

This last issue made me hate life and put the project on a long hiatus.

More demos

A while later while doing another demo (more on that in the next post) I discovered the horrible workaround: the private function for capturing events in GTK has a very simple implementation, so it’s easy to set this handler manually. And of course, with this workaround it just works. This solves the issue #4.

For the issue #3 I made a crude solution similar to already existing HdyHeaderGroup: HdySwipeable and HdySwipeGroup. It’s an RFC at this point, so criticism is welcome.

That allowed me to make a fully working (though still buggy) prototype of swipeable leaflet:

(Yes, that’s a bug there on 0:21)

Another visible problem in the video is that HdyHeaderGroup showing and hiding buttons doesn’t really work with the gesture. One possible solution here would be to show all buttons on all headerbars when folded, but that would once again involve an API break.

The (still very messy) code is here. Even though it’s not ready yet, Shortwave app already makes use of it:

This also uncovered a crash when a leaflet is created in unfolded state. Oops.

Thanks Felix Haecker for testing, and once again Tobias Bernard for feedback and suggestions while iterating on it.

Michael Meeks: 2019-09-14 Saturday.

Sht, 14/09/2019 - 11:00md
  • Up early, bid more 'byes; coach with Tor & Kendy to Malaga, worked in the airport on estimation & planning, as well as tile joining.
  • There is surely some irony embedded in the outrage at RMS' suggestion that accusation inflation exists and can be damaging. The escalating gap between what he is accused of, and what (as far as can be told) he actually wrote looks like a great example of exactly that. Argument by outrage, with a spirit of vindictive intolerance seems to be the fashionable bigotry of the day. Luckily I don't look to RMS for ethical statements on sexuality to try to live by, but try: this instead. In general, promiscuity seems ethically troubling, as well as a meagre subsitute for a healthy marriage filled with love & mutual respect. However, I'm still rather interested in RMS' views on software freedom - despite being more persuaded by Gerv's Theological Cultural Analysis of the Free Software movement.

Matthew Garrett: It's time to talk about post-RMS Free Software

Sht, 14/09/2019 - 1:57md
Richard Stallman has once again managed to demonstrate incredible insensitivity[1]. There's an argument that in a pure technical universe this is irrelevant and we should instead only consider what he does in free software[2], but free software isn't a purely technical topic - the GNU Manifesto is nakedly political, and while free software may result in better technical outcomes it is fundamentally focused on individual freedom and will compromise on technical excellence if otherwise the result would be any compromise on those freedoms. And in a political movement, there is no way that we can ignore the behaviour and beliefs of that movement's leader. Stallman is driving away our natural allies. It's inappropriate for him to continue as the figurehead for free software.

But I'm not calling for Stallman to be replaced. If the history of social movements has taught us anything, it's that tying a movement to a single individual is a recipe for disaster. The FSF needs a president, but there's no need for that person to be a leader - instead, we need to foster an environment where any member of the community can feel empowered to speak up about the importance of free software. A decentralised movement about returning freedoms to individuals can't also be about elevating a single individual to near-magical status. Heroes will always end up letting us down. We fix that by removing the need for heroes in the first place, not attempting to find increasingly perfect heroes.

Stallman was never going to save us. We need to take responsibility for saving ourselves. Let's talk about how we do that.

[1] There will doubtless be people who will leap to his defense with the assertion that he's neurodivergent and all of these cases are consequences of that.

(A) I am unaware of a formal diagnosis of that, and I am unqualified to make one myself. I suspect that basically everyone making that argument is similarly unqualified.
(B) I've spent a lot of time working with him to help him understand why various positions he holds are harmful. I've reached the conclusion that it's not that he's unable to understand, he's just unwilling to change his mind.

[2] This argument is, obviously, bullshit

comments

Michael Meeks: 2019-09-13 Friday.

Pre, 13/09/2019 - 11:00md
  • Lots of conference hallway track conversations. Fine tour of the beautiful Almeria fortress & dramatized reading by our team. Tapas & bed late having bid 'bye to many friends & colleagues. Packed luggage.

Richard Hughes: GNOME Firmware 3.34.0 Release

Pre, 13/09/2019 - 3:12md

This morning I tagged the newest fwupd release, 1.3.1. There are a lot of new things in this release and a whole lot of polishing, so I encourage you to read the release notes if this kind of thing interests you.

Anyway, to the point of this post. With the new fwupd 1.3.1 you can now build just the libfwupd library, which makes it easy to build GNOME Firmware (old name: gnome-firmware-updater) in Flathub. I tagged the first official release 3.34.0 to celebrate the recent GNOME release, and to indicate that it’s ready for use by end users. I guess it’s important to note this is just a random app hacked together by 3 engineers and not something lovelingly designed by the official design team. All UX mistakes are my own :)

GNOME Firmware is designed to be a not-installed-by-default power-user tool to investigate, upgrade, downgrade and re-install firmware.
GNOME Software will continue to be used for updates as before. Vendor helpdesks can ask users to install GNOME Firmware rather than getting them to look at command line output.

We need to polish up GNOME Firmware going forwards, and add the last few features we need. If this interests you, please send email and I’ll explain what needs doing. We also need translations, although that can perhaps wait until GNOME Firmware moves to GNOME proper, rather than just being a repo in my personal GitLab. If anyone does want to translate it before then, please open merge requests, and be sure to file issues if any of the strings are difficult to translate or ambigious. Please also file issues (or even better merge requests!) if it doesn’t build or work for you.

If you just want to try out a new application, it takes 10 seconds to install it from Flathub.

Alexander Mikhaylenko: GNOME and gestures, Part 1: WebKitGTK

Pre, 13/09/2019 - 10:32pd

Re-publishing, the original post is too old to show up on Planet GNOME at this point. It was published on August, 8th initially.

I’m a big fan of responsive touchpad gestures. For the last half a year (mostly January, February and during the summer) I’ve been working on improving gestures in many areas throughout GNOME. In this series I will do a (belated) overview.

Back/Forward Swipe

Late in the 3.32.x cycle, I saw a commit by Jan-Michael Brummer adding a back/forward swipe to Epiphany. It was really nice to finally have gestures, but it didn’t have any visual feedback. Less importantly, the direction was reversed, as if when scrolling with Natural Scrolling being off. I wanted to give a shot at improving it.

A proper gesture would have to “stick to finger”, showing screenshot of the previous or next page during the gesture, more or less what Safari does on macOS. Specifically, Epiphany would have to take screenshot of every page that is added into back/forward history, show it while the gesture is performed, then continue showing it until the next page loads enough to replace it. Unfortunately, this isn’t really possible to achieve in Epiphany itself: while WebKit does provide API to take snapshots, there’s no way to know when the previous/next page has loaded “enough”.

So I started looking into WebKit instead, where I found out that Safari’s gesture is actually implemented right there! Most parts are present, but the code was not cross-platform. So I started slowly adapting it for GTK. For the most part, the reusable code was a large part of the back end: page snapshot store and snapshot removal logic. That code is now shared between the platforms. The other parts, like actual event processing and drawing, had to be written from scratch.

One interesting detail about the gesture is that it doesn’t actually use gesture events! Libinput defines swipe gestures as synchronous movement of three or more fingers in the same direction. Mac gesture API is even more strict: it’s three fingers only, with four-finger swipes being reserved for the OS. But the gesture uses two fingers, how is this possible? Turns out it actually uses scroll events instead. (That’s also why it works with Magic Mouse in macOS, even though the code does not special-case anything for it)

When using scroll events, one has to be very careful. Touchpads generate scroll events continuously, in GTK it means that these gestures have GDK_SCROLL_SMOOTH scroll direction. At the very end of the scrolling, there will be a special event with is_stop field set to TRUE, which is used as a signal to start kinetic scrolling or, in our case, to end a swipe.

But there are other input devices, for example, mice. Most mice have “clicky” wheels that generate scroll events with direction instead of deltas. These events are impossible to use for swipes, so there’s no point in even trying to handle them. But there are also mice with freely scrolling wheels which generate the same events as touchpad, except there’s no event with is_stop == TRUE at the end. This means that they can be used to start a swipe, but it will get stuck as soon as the wheel stops. So, these mice have to be skipped too. Then there are touch mice where scrolling probably works same as on touchpad. I suspect swiping can work very well with them, same as it does with Magic Mouse on macOS, but there’s no way to distinguish these kinds of mice, at least as far as I know, so I had to disable it for any mice.

Another problem is that in order to not interfere with actual scrolling, the gesture controller must check whether it is possible to scroll the page first. Maybe there’s still space to scroll, maybe the page intercepts the scroll events. Then there has to be a threshold so that it’s hard to accidentally trigger the gesture. Thankfully, this part is shared with the Mac gesture. :)

A side effect of using scroll events is that this gesture still works with X11 and/or older semi-mt touchpads for which Libinput normally does not support any gestures.

So, now the gesture sticks to fingers. But there’s still an important bit missing: a snap-back animation. In order for a gesture to feel natural, it should snap back smoothly as soon as you lift your fingers, respecting the momentum. This was a lot easier to do than I expected, and after a few iterations applying Tobias Bernard’s suggestions I had an animation that I’m very satisfied with. How it works:

  • The animation uses easeOutCubic interpolation
  • Duration is naturally calculated as remaining distance divided by velocity, or if the velocity is 0, by a constant value instead
  • After that, duration is multiplied by 3, matching easeOutCubic derivative at t=0. This ensures that initial velocity is same as it was before lifting the fingers
  • Finally, the duration is clamped into [100ms, 400ms] range. This ensures that it’s never too slow or too fast, while still allowing it to respect momentum when possible
  • If the page was swiped less than halfway through the window, there’s a small velocity threshold. If fingers are lifted when not moving, the gesture will be canceled and the page will smoothly slide back. On the other hand, if the page was swiped more than half way through, just lifting the fingers would finish the gesture, and one has to specifically flick back to cancel it
  • If one starts swiping again while the animation is going, it will actually be stopped. This allows to continuously grab and release the page

Interestingly, Mac has a helper function that takes care of all this, and WebKit makes use of it.

Finally, the gesture should look nice. On Mac it uses CoreAnimation for drawing; WebKitGTK has to use Cairo. Since I didn’t have any mockups to work with, I reused Apple’s visuals, consisting of a dimming layer and a long subtle gradient for drop shadow, intending to replace them with something else later. But everybody whom I showed it liked it, so I left it as is.

The end result is that since version 2.24.0, WebKitGTK optionally supports 2-finger swipe gestures on touchpad.

Unfortunately, I was a little too late to enable it in Epiphany 3.32.0, but it was merged into 3.32.1 nevertheless. In 3.34.x, Yelp and Devhelp will also support it. Additionally, it’s enabled in Eolie and Odysseus browsers.

A bit later I also added touchscreen support. That was easy, because WebKit literally generates scroll events for touch scrolling, so it was simply a matter of feeding those events into the gesture controller. Additionally, I had to implement canceling, as all touchscreen gestures have to support it. Since scrolling on touchscreen is 1-finger swipe, the gesture is performed the same way.

This will be available in upcoming WebKitGTK 2.26.x, corresponding to GNOME 3.34.

This can be very disruptive feature in many cases, such as authentication widgets, so applications wanting to use it have to opt in by changing the value of this property.

Pinch Zoom

A smaller change was getting pinch zoom gesture to work on touchpads. Since this gesture was already available on touchscreens, it involved simply feeding touchpad gesture events into the gesture tracker, but the performance is severely lacking on heavy pages. Speeding it up is unfortunately still above my skill level. :)

Pinch zoom is enabled unconditionally, so it already works everywhere where WebKitGTK is used, including but not limited to Geary and documentation view in GNOME Builder.

I want to say thanks to the following people:

  • Michael Catanzaro and Carlos Garcia Campos for code review and helping me with understanding WebKit codebase
  • Tobias Bernard for testing and numerous design suggestions
  • Jonas Dreßler for testing and feedback, especially on a touchscreen

Peter Hutterer: Unit-testing static functions in C

Enj, 12/09/2019 - 6:21pd

An annoying thing about C code is that there are plenty of functions that cannot be unit-tested by some external framework - specifically anything declared as static. Any larger code-base will end up with hundreds of those functions, many of which are short and reasonably self-contained but complex enough to not trust them by looks only. But since they're static I can't access them from the outside (and "outside" is defined as "not in the same file" here).

The approach I've chosen in the past is to move the more hairy ones into separate files or at least declare them normally. That works but is annoying for some cases, especially those that really only get called once. In case you're wondering whether you have at least one such function in your source tree: yes, the bit that parses your commandline arguments is almost certainly complicated and not tested.

Anyway, this week I've finally found the right combination of hacks to make testing static functions easy, and it's:

  • #include the source file in your test code.
  • Mock any helper functions you'd need to trick the called functions
  • Instruct the linker to ignore unresolved symbols
And boom, you can write test cases to only test a single file within your source tree. And without any modifications to the source code itself.

A more detailed writeup is available in this github repo.

For the impatient, the meson snippet for a fictional source file example.c would look like this:


test('test-example',
executable('test-example',
'example.c', 'test-example.c',
dependencies: [dep_ext_library],
link_args: ['-Wl,--unresolved-symbols=ignore-all',
'-Wl,-zmuldefs',
'-no-pie'],
install: false),
)

There is no restriction on which test suite you can use. I've started adding a few of test cases based on this approach to libinput and so far it's working well. If you have a better approach or improvements, I'm all ears.

Fabián Orccón: GUADEC 2019 wrap-up

Enj, 12/09/2019 - 2:00pd

This year is the third edition of the GUADEC. Things were slightly different now: I was not a GSoC student anymore and I had my first jet lag. Three flights, some trains (including a type of train which rails were suspended in the air) were enough to go to Thessaloniki lands. When I arrived to Greece, I was a bit scared of the language since the alphabet would be almost impossible to type in my smartphone. However, I could easily reach the accomodation point.

My purpose for this GUADEC was different than the past ones. In the past I went basically to talk about my Google Summer of Code projects, but this time I wanted to show to the attendees the project I was working on as part of my dissertation project. I wanted to re-write almost everything of what I did and in the best case my plan was to find a contributor to my project. I am very happy to say that I found one contributor to this project. The project I talk about consisted on adding face overlay effects to Cheese developing a GStreamer plugins which elements should be better than gstfaceoverlay and gstfacedetect. The code of the project I made for my dissertation project can be found on this link and the one that is being written from scratch can be found on this repository. The slides are available on Google Docs and the full details (actually the thesis document) is written (in Spanish) in this document.

My lightning talk about face effects in Cheese.

Although this GUADEC has not been very productive to me in the sense that I was pretending to make more progress on this project, I feel very happy to have shared my knowledge with a polish guy I met the first day I arrived to Thessaloniki. His name is Mieszko Mazurek and he has never been a GSoC student or contributed to any project. However he is very talented, and has a company that develops drivers and software to manage embeded systems using GNOME-based technologies. I taught him about GStreamer, how to create an application, and he learned on his own to make a GStreamer plug-in. That was amazing! He is from Poznan (Poland) and coincidentally I was going there to have some vacations there. So we agreed to meet each other for further discuss about my project.

The conference days have been very informative as always. I got very hooked up for the Engagement team announcement and found out very interesting some intern lighning talks specially the ones about Usability given by Clarissa, who would later become a nice friend, and the one about the thumbnail generator for Polari. A talk that I liked the most was one about controlling GNOME Shell windows with a VR remote controller. That was a actually a project from Lubosz who works for Collabora and who was doing a Google Summer of Code in parallel to my first GSoC. Was nice to see him again but it was also nice to test his project. He is a really skilled guy. Finally, among other talks other that I found very intersting was the one about GTK4 and its new ways to define layouts.

During the event I met some of the Endless people I’ve been working with. I met also some ex-endlessers who I worked with, Cosimo and Meg. I had some time with Manuel Quiñones, who works with me on Hack, which develops software in top of Endless OS which is intended to teach kids to learning to code. We were also discussing about our workflows, commenting about some possible improvements on the software we develop and, because I had with me the Hack Computer, we showed Cosimo our progress in such software. It was really a pleasure to talk with each of these people in person. Also, I cannot skip to say that it was a surprise to hear about the donation of Endless to the GNOME Foundation for the “Coding Education Challenge”.

Endless people With Manuel Quiñones

I also attended to the GStreamer-related BOFs/workshops. It was basically a workshop about developing a simple webcam application in Rust with GStreamer. The workshop was lead by Sebastian Droge. Unfortunately, I couldn’t be there during the whole session, but I stayed there most of its duration. I will set the personal goal of learning Rust by February of the next year. During the event, I also had the opportunity to talk with Sebastian Droge and Tim, GStreamer maintainers, and told them to review some very old patches I have. I am not in a hurry, but I hope that happens soon. Now I remember, that I also talked with another GStreamer developer about the possibility of writing a GStreamer element that inherits from GstOpenCVVideoFilter in Python. There is an issue regarding this, since cv::Mat types are not introspectable. So even when I tried to write my own filter similar to GstOpenCVVideoFilter in Python from scratch, I faced that I couldn’t access the raw data of the GstVideoInfo of each plane. I wrote a patch long time ago that I did not published but faced with the problem that I could not wrap that raw data info about the frames in a GLib.Array (from Python) without creating a copy.

The conference was an enjoyable event. Definitevely this event is more than its talks and about meeting people, discussing ideas for future projects and having some fun there. There were scheduled and non-scheduled social events. I also had some fun going to spontaneous dinners, having some drinks and a small boat trip and the visit of the various museums in Thessaloniki. Finally, I want to thank to the foundation for sponsoring me. To be honest, I was not expecting to go this year to the conference, but there I was.

Post-GUADEC

After GUADEC I had some vacations in Greece for about one week more and then I was going to Poznan, Poland. As I mentioned, the first day of the event I met Mieszko Mazurek who actually lives in that city. He was showing me the city and his office in which he works were he develops low-level and high-level software to control batteries. He uses GNOME-based technology for this high-level software. I also continued to show him and explaining him about the Cheese Face Effects project. Finally, that day I could get the code I wrote during the event with the help of him to work as expected. Now I am on Krakow, and he is going to do an inter-city trip to continue talking about the mentioned project.

Mieszko and I trying the gst-plugins-cv repository

Alexander Mikhaylenko: GNOME Games 3.34

Enj, 12/09/2019 - 1:01pd

A year ago, Adrien Plazas stepped down as a maintainer, so Games 3.32.0 was released without an accompanying blog post, since I didn’t have a blog at the time. Now it’s time to make up for it with a blog post about 3.34.0.

GSoC and savestates

As part of his GSoC project, Andrei Lişiţă implemented a savestate manager.

Savestates are a common feature in game emulators, that work similarly to snapshots in virtualization: emulator takes a full snapshot of RAM and storage, which can be loaded later to restore the game to the same exact state it was in when saved.

The app has supported savestates for a long time: when you exit a game, a savestate is created. Then when you run it again, Games offers to restore that savestate or reset the game. However, there was no way to manage savestates during the game, or to have more than one savestate at a time.

Now we have a shiny new sidebar for managing savestates and can save and load them on demand. This can be used for saving memorable moments in games, or for cheating your way through difficult games via saving and reloading every time you make a mistake.

There is still room for improvement, for example, there is no way to use savestates with gamepad right now.

Since directory layout is different now, existing data is automatically migrated on the first run. This is a one-way process, downgrading to 3.32.1 after running 3.34.0 is not possible!

Nintendo DS screen layouts

GNOME Games supports running Nintendo DS games since 3.30. However, the system’s two screens make it awkward to play without having a screen in portrait orientation. While ideally we want retro-gtk to support support rearranging screens, for now I implemented this using options of DeSmuME core:

So far we support a vertical mode, two horizontal modes and a single screen mode, corresponding to DeSmuME’s top/bottom, left/right, right/left and quick switch modes. While the first 3 modes work exactly the same as in RetroArch, single screen mode has some improvements: we remember and restore the currently viewed screen, provide a visible button for switching between the screens and have separate keyboard shortcuts for that (it’s using top only and bottom only internally, not quick switch).

Naturally, a downside is that it only works with DeSmuME and DeSmuME 2015 cores.

At first it was a global setting. This led to some problems, such as squished screenshots when loading the game if the current layout doesn’t match the one the game was played with the last time, which was made even more apparent after Andrei’s savestate work landed, showing the squished screenshots even more prominently.

Because of that, I reworked the feature to store the screen layout inside savestate metadata instead of a global setting. While this required a fix in retro-gtk, screen layout now can be different not just for every game, but even for every savestate, meaning previews always perfectly match the loaded game.

Platforms instead of Extensions

For a long time, Games had an “Extensions” page in preferences, showing all the installed plugins. This was one of the more confusing parts of the app:

There are many problems with this page: many people assumed that the listed platforms were the only platforms we support, even though there are 24 more platforms that are supported directly, not via a plugin. Additionally, “platform support”, via plugin or not, means that Games can list games from this platforms in collection view. It does not automatically mean that the app can run those games, that would also require the corresponding libretro core to be installed. And even then, some games need firmware to run. None of this is really obvious from the page.

All in all, this means that Games can run NES and Game Boy games, even though they aren’t listed on the page, and at the same time it cannot run DOS or Sega Saturn games, even though they are listed, which makes no sense unless you’re involved in development of the app.

Even more confusingly, the modules that the page lists are actually called “plugins” internally, not “extensions”.

To solve these problems, this page has been removed and replaced with a Platforms page that lists supported platforms, lists the currently used core for libretro platforms and grays out the platforms without cores, so that it’s visible which games can and cannot be run.

While our official build on Flathub ships only one core per platform, it’s possible to have multiple cores for a single platform. In that case, the page will allow to choose the core to run the games with. I hope the new page will be a lot more useful.

Backup and Restore

Another feature that landed during this cycle is backing up and restoring for savestates, courtesy of Adwait Rawat.

These changes include a page in preferences that allows to export all the savestates as one archive, or to restore an existing backup. This can be useful, for example, to move data to a different device.

Goodbye, developers view

Developers view was implemented along with Platforms view by Saurabh Sabharwal as part of his GSoC 2018 project. While Platforms view was very successful, Developers view used thegamesdb metadata, which wasn’t very reliable. Often, it mixed developers and publishers, or duplicated the same developer multiple times with slight variations. And then, thegamesdb changed its API, so for the last few months, this view has been completely empty. Because of that, I went ahead and removed this view:

Adaptive UI

I’ve been slowly making the UI adaptive for the last year. 3.34.0 finally marks the point where the app can run on a phone:

However, there is still a very important bit missing: touch controls to actually play games without a gamepad or keyboard.

Better Fullscreen

In 3.32.1, headerbar in fullscreen is hidden, but shows up on any cursor movement. This can be annoying when playing a mouse-heavy game (for example, on Nintendo DS), so in 3.34.0 the headerbar can only be revealed by pushing the top of the screen, similar to the behavior in Epiphany and other apps.

Miscellaneous Changes

Andrei added an error message that shows up after opening a non-game file:

Media switcher has a dropdown arrow now:

Cursor now autohides in windowed mode after 3 seconds of inactivity, just as in fullscreen.

Game covers aren’t blurry on HiDPI screens and aren’t darkened anymore.

Thanks to all the contributors who made this release possible!

Getting Games

As always, the latest version of the app is available on Flathub.

Federico Mena-Quintero: Gdk-pixbuf modules - call for help

Enj, 12/09/2019 - 12:54pd

I've been doing a little refactoring of gdk-pixbuf's crufty code, to see if the gripes from my braindump can be solved. For things where it is not obvious how to proceed, I've started taking more detailed notes in a gdk-pixbuf survey.

Today I was looking at which gdk-pixbuf modules are implemented by third parties, that is, which external projects provide their own image codecs pluggable into gdk-pixbuf.

And there are not that many!

The only four that I found are libheif, libopenraw, libwmf, librsvg (this last one, of course).

All of those use the gdk-pixbuf module API in a remarkably similar fashion. Did they cut&paste each other's code? Did they do the simplest thing that didn't crash in gdk-pixbuf's checks for buggy loaders, which happens to be exactly what they do? Who knows! Either way, this makes future API changes in the modules a lot easier, since they all do the same right now.

I'm trying to decide between these:

  • Keep modules as they are; find a way to sandbox them from gdk-pixbuf itself. This is hard because the API is "chatty"; modules and calling code go back and forth peeking at each other's structures.

  • Decide that third-party modules are only useful for thumbnailers; modify them to be thumbnailers instead of generic gdk-pixbuf modules. This would mean that those formats would stop working automatically in gdk-pixbuf based viewers like EOG.

  • Have "blessed" codecs inside gdk-pixbuf which are not modules so their no longer have API/ABI stability constraints. Keep third-party modules separate. Sandbox the internal ones with a non-chatty API.

  • If all third-party modules work indeed as I found, the module API can be simplified quite a lot since no third-party modules implement animations or saving. If so, simplify the module API and the gdk-pixbuf internals rather drastically.

Do you know any other image formats which provide gdk-pixbuf modules? Mail me, please!

Richard Hughes: Please welcome Acer to the LVFS

Mër, 11/09/2019 - 1:44md

Acer has now officialy joined the LVFS, promoting the Aspire A315 firmware to stable.

Acer has been testing the LVFS for some time and now all the legal and technical checks have been completed. Other models will follow soon!

Tobias Mueller: Talking at GUADEC about defending against USB-borne attacks

Mar, 10/09/2019 - 10:44md

Ouf, long time no blog. Sorry about that. Life is.. busy these days. One of my occupations has been to defend against USB-borne attacks, as mentioned before. Probably the most known is the BadUSB attack which masquerades as a mass storage device but also is a keyboard. Then, the device would inject keystrokes which hack your machine. This class of attacks is difficult to defend against, because the operating system can hardly determine whether the user did indeed want to plug a device with those capabilities in. There is another class of attacks, though, which is a bit easier to defend against, I think. That other class is based on buggy drivers which the malicious USB device pulls in. So once you attach your device, the host’s kernel will look for the appropriate driver and let it speak with the device. That’s very convenient because it makes devices just work. However, certain drivers might not be of the same quality than others and there have indeed been cases which allow a malicious USB device to interact with a bit-rotten driver which in turn led to fatal consequences.

I have been thinking a lot about how to defend against either class of attacks. You can easily come up with various solutions based on pop-ups interrupting the user and authoring the device before it will be ready for use. Or with a policy-based solution that requires you to generate a firewall-like description of what the machine is allowed to do. Or a mixture of those two attempts. And that’s what has been done, already. Arguably, none of these solutions has been successful, as I am not aware of any built-in protection scheme for any major operating system. The reason, I believe, is that users expect things to just work and once you make it not work, users get grumpy. So the challenge is to unfold protection capabilities without changing the users’ experience.

The short version of our approach is that we are trying to be smart about the user’s intent. That is, if the screen is locked, then we block the device. If a new keyboard is present and it tries to perform “dangerous” actions, we block them. Of course, you may very well expect that device to work when the screen is locked or the new keyboard to perform actions deems dangerous. This is why is make sure you have a way to opt out of the mechanism and continue to enjoy your GNOME experience. Almost all credits go to Ludovico for coming up with a set of patches as well as following up to make sure we can get it merged. Our slides are here and the video of our presentation is here:

But I wanted to write more about GUADEC… This year’s GUADEC was in Thessaloniki, Greece, and I had the pleasure to be talking about the above mentioned protection. It was the end of the summer so the city was nicely warm and comfy. The coffee, juices, pastries, and other food and drinks in small shops on the streets were amazingly fresh and yummie. Arriving in Thessaloniki was okay. I’ve had better airport transfers in my life, but since there were only two buses it was hard to get lost. I needed to pay attention to the GPS, though, to find my right stop. It’s been long since I’ve slept in a bunk bed, but because we’re all GNOME people we had a good time.

The conference had a few interesting talks which can be followed on the recordings page. I enjoyed watching Daiki presenting about plans and ideas for managing credentials in a sandboxed world, Benzo talking about user sessions with systemd, and the (not recorded) one by Giannis on the impact of the GDPR. Of course, the 45 minutes or so we had for discussing all the facets of the GDPR were too short and I think that I would have focussed on other aspects such as choosing an appropriate legitimate ground, transfer to non-member states, or dealing with requests from data subjects. But it’s been a good introduction and I am happy to see non-technical topics at the conference.

Meeting friends, old and new, is really good and I have had a fantastically efficient time talking to people. It’s so much better to meet in-person and talk directly rather than via email or bug tracker.

Allan Day: Towards a UX Strategy for GNOME (Part 2)

Mar, 10/09/2019 - 5:43md

This post is a part of a short series, in which I’m setting out what I think could be the beginnings of a UX strategy for GNOME. In this, the second post, I’m going to describe a potential GNOME UX strategy in high-level terms. These goals are a response to the research and analysis that was described in the previous post and, it is hoped, point the way forward for how GNOME can achieve new success in the desktop market.

Strategic goals

For me, the main goals of a GNOME UX strategy could be:

1. Deliver quality

If GNOME is going to succeed in today’s desktop market, UX quality has to be job #1.

UX quality includes what the software looks like and how it is designed, but it also refers to how the software functions. Performance and bugs (or the lack of them) are both aspects of UX!

More than anything else, people are looking for a desktop that Just Works: they want a solution that allows them to get their work done without getting in their way. This means having a desktop that is reliable, stable, which does what people want, and which is easy to use.

People value solutions that Just Work. They’re also prepared to abandon them when they don’t Just Work.

To its credit, the GNOME project has historically recognised the importance of Just Works, and it has delivered huge improvements in this area. However, there is still a lot of work to be done.

My sense is that driving up quality is one of the key strategic challenges that the GNOME project needs to face up to; I’ll be returning to this topic!

2. Factor in the cloud

In my previous post, I worte about how the cloud has reconfigured the landscape in which GNOME operates. Accordingly, it’s important for our UX strategy to account for the cloud. There are various ways we can do this:

  • Focus on those bits of the desktop that are used by all users, even if they mainly use a web browser. This includes all the parts of the core system, as well as the most essential desktop apps.
  • Enable and encourage native cloud applications (including Electron and Progressive Web Apps)
  • Add value with high-quality native apps.
  • Integrate with existing cloud services, when it is safe to do so.

The last point might seem counter-intuitive, but it makes sense: in a world where the web is dominant, a fantastic set of native apps can be a powerful differentiator.

At the same time, GNOME needs to be careful when it comes to directly competing with sophisticated web apps and services, and it needs to recognise that, nowadays, many apps aren’t worth doing if they don’t have a cloud/cross-device component.

3. Grow the app ecosystem

The primary purpose of a platform like GNOME is to run apps, so it stands to reason that the number and quality of the apps that are available for the platform is of critical importance.

Recently, Flatpak has allowed the GNOME project to make great progress around application distribution, and this is already positively impacting app availability for GNOME. However, there is a lot of work still to be done, particularly around GNOME’s application development platform. This includes work for both designers and developers.

4. Support modern hardware

One of the things that my research revealed is that, for most users, their choice of desktop OS is thoroughly entwined with hardware purchasing choices, with hardware and software typically being seen as part of the same package. Attracting users to GNOME therefore requires that GNOME be available for, work well with, and be associated with high-quality hardware.

A lot of hardware enablement work is done by distros, but a lot also happens in GNOME, including things like high-definition display support, touchscreen support, screen casting, and more. This is important work!

Do less, prioritise

Any UX strategy should address the question of prioritisation: it ought to be able to determine how resources can be directed in order to have maximum impact. This is particularly important for the GNOME project, because its resources are limited: the core community is fairly small, and there’s a lot of code to maintain.

The idea of prioritisation has therefore both influenced the goals I’ve set out above, as well as how I’ve been trying to put them into practice.

When thinking about prioritisation in the context of GNOME UX, there are various principles that we can follow, including:

  • User exposure, both in terms of the proportion of people that use a feature, and also the frequency with which they use it. Improvements to features that everyone uses all the time have a bigger impact than improvements to features that are only used occasionally by a subset of the user base.
  • User needs and desires: features that are viewed as being highly attractive by a lot of people are more impactful than those which are only interesting to a small subset.
  • Common underpinnings: we can prioritise by focusing on common subsystems and technical components. The key example here is something like GTK, where improvements can surface themselves in all the apps that use the toolkit.

When we decide which design and development initiatives we want to focus on (either by working on them ourselves, or advertising them to potential contributors), principles like these, along with the high-level goals that I’ve described above, can be very helpful.

I also believe that, in some cases, the GNOME project needs to have some hard conversations, and think about giving up some of its existing software. If quality is job #1, one obvious answer is to reduce the amount of software we care about, in order to increase the overall quality of everything else. This is particularly relevant for those parts of our software that don’t have great quality today.

Of course, these kinds of conversations need to be handled delicately. Resources aren’t fungible in an upstream project like GNOME, and contributors can and should be free to work on what they want.

What’s next

In my previous post, I described the research and analysis that serves as inputs to the strategy I’m setting out. In this post I’ve translated that background into a high-level plan: four strategic goals, and an overarching principle of prioritisation.

In the next post, I’m going to introduce a raft of design work which I think fits into the strategy that I’ve started to lay out. GNOME is lucky to have a great design team at the moment, which has been pumping out high-quality design work over the past year, so this is a great opportunity to showcase what we’ve been doing, but what I want to do is also show how it fits into context.

Alexander Larsson: Gthree – ready to play

Hën, 09/09/2019 - 11:07pd

Today I made a new release of Gthree, version 0.2.0.

Newly added in this release is support for Raycaster, which is important if you’re making interactive 3D applications. For example, it’s used if you want clicks on the window to pick a 3D object from the scene. See the interactive demo for an example of this.

Also new is support for shadow maps. This allows objects between a light source and a target to cast shadows on the target. Here is an example from the demos:

I’ve been looking over the list of feature that we support, and in this release I think all the major things you might want to do in a 3D app is supported to at least a basic level.

So, if you ever wanted to play around with 3D graphics, now would be a great time to do so. Maybe just build the code and study/tweak the code in the examples subdirectory. That will give you a decent introduction to what is possible.

If you just want to play I added a couple of new features to gnome-hexgl based on the new release. Check out how the tracks casts shadows on the buildings!