GSoC 2021: Selection Editing and Window Selection

This summer I’m implementing a new screenshot UI for GNOME Shell. In this post I’ll show my progress over the past two weeks.

The new screenshot UI in the area selection mode

I spent the most time adding the four corner handles that allow you to adjust the selection. GNOME Shell’s drag-and-drop classes were mostly sufficient, save for a few minor things. In particular, I ended up extending the _Draggable class with a drag-motion signal emitted every time the dragged actor’s position changes. I used this signal to update the selection rectangle coordinates so it responds to dragging in real-time without any lag, just as one would expect. Some careful handling was also required to allow dragging the handle past selection edges, so for example it’s possible to grab the top-left handle and move it to the right and to the bottom, making it a bottom-right handle.

Editing the selection by dragging the corner handles

I’ve also implemented a nicer animation when opening the screenshot UI. Now the screen instantly freezes when you press the Print Screen button and the screenshot UI fades in, without the awkward screenshot blend. Here’s a side-by-side comparison to the previous behavior:

Comparison of the old and new opening animation, slowed down 2×

Additionally, I fixed X11 support for the new screenshot capturing. Whereas on Wayland the contents of the screen are readily available because GNOME Shell is responsible for all screen compositing, on X11 that’s not always the case: full-screen windows get unredirected, which means they bypass the compositing and go straight through the X server to the monitor. To capture a screenshot, then, GNOME Shell first needs to disable unredirection for one frame and paint the stage.

This X11 capturing works just as well as on Wayland, including the ability to capture transient windows such as tooltips—a long-requested feature. However, certain right-click menus on X11 grab the input and prevent the screenshot UI hotkey (and other hotkeys such as Super to enter the Overview) from working. This has been a long-standing limitation of the X11 session; unfortunately, these menus cannot be captured on X11. On Wayland this is not a problem as GNOME Shell handles all input itself, so windows cannot block its hotkeys.

Finally, over the past few days I’ve been working on window selection. Similarly to full-screen screenshots, every window’s contents are captured immediately as you open the screenshot UI, allowing you to pick the right window at your own pace. To capture the window contents I use Robert Mader’s implementation, which I invoke for all windows from the current workspace when the screenshot UI is opening. I arrange these window snapshots in a grid similar to the Overview and let the user pick the right window.

Window selection in action

As usual, the design is nowhere near finished or designer-approved. Consider it an instance of my “programmer art”. 😁

My goal was to re-use as much of the Overview window layout code as possible. I ended up making my own copy of the WorkspaceLayout class (I was able to strip it down considerably because the original class has to deal with windows disappearing, re-appearing and changing size, whereas the screenshot UI window snapshots never change) and directly re-using the rest of the machinery. I also made my own widget compatible with WindowPreview, which exports the few functions used by the layout code, once again considerably simplified thanks to not having to deal with the ever changing real windows.

The next step is to put more work into the window selection to make sure it handles all the different setups and edge cases right: the current implementation is essentially the first working draft that only supports the primary monitor. Then I’ll need to add the ability to pick the monitor in the screen selection mode and make sure it works fine with different setups too. I also want to figure out capturing screenshots with a visible cursor, which is currently notably missing from the screenshot UI. After that I’ll tackle the screen recording half.

Also, unrelated to the screenshot UI, I’m happy to announce that my merge request for reducing input latency in Mutter has finally been merged and should be included in Mutter 41.alpha.

That’s it for this post, see you in the next update!

8 thoughts on “GSoC 2021: Selection Editing and Window Selection

  1. It looks very slick. I really like how simple and intuitive it is.

    I have 3 considerations for you, if you don’t mind.

    Consideration 1: Defaults and keybindings.
    Keybind the circle/”take picture”-button to Prnt Screen and make fullscreen default. This way if you double press the Prnt Screen button, you get a fullscreen screenshot without having to press the button and THEN having to click the icon.

    Consideration 2: Freezing the screen
    What if you’re in a fullscreen application (like a game or watching a fullscreen movie). How would your implementation affect the use of such program – If at all?

    Consideration 3: Consider a way to quickly access the picture once it’s taken.
    I have a feeling that when people take screenshots, they want to edit/send/upload the screenshot immediately. Maybe consider some way to do that? Perhaps a notification to open the file manager at screenshot’s location is the right way? Maybe not?

    Just food for thought. I really like the direction of where you’re going with this. This has great potential 🙂

    Liked by 1 person

    1. Thanks!

      > Consideration 1: Defaults and keybindings.

      Yeah, keybindings is something that hasn’t been discussed yet. I think you should forward your suggestion to the design issue: https://gitlab.gnome.org/Teams/Design/os-mockups/-/issues/108

      > What if you’re in a fullscreen application (like a game or watching a fullscreen movie). How would your implementation affect the use of such program – If at all?

      It’ll freeze and let you select an area, just like anywhere else.

      > Consideration 3: Consider a way to quickly access the picture once it’s taken.

      The screenshot will be copied into the clipboard and likely saved into the pictures folder with a notification. This part doesn’t have finalized design yet, but I’m sure either way it should be easy for e.g. an extension to add automatic uploading to some image hosting.

      Like

  2. This looks really awesome! Do you also plan to add some edit functionality? Most of the time when I make screenshots I want to send it to someone in order to illustrate something. This means that most of the time I want to highlight a special area, obfuscate a password, add some small text snippets etc.

    If I could do this directly while taking the screenshot and afterwards directly copy/save the result, this would be really great.

    At the moment I use the deepin screenshot tool which offers exactly this functionality. If you don’t know it yet, it is definitely worth having a look at it. I think it could contain some nice ideas for your work.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s