Debarshi's den

Fedora Toolbox is now just Toolbox

with 2 comments

toolbox-logo-landscape

Fedora Toolbox has been renamed to just Toolbox. Even though the project is obviously driven by the needs of Fedora Silverblue and uses technologies like Buildah and Podman that are driven by members of the wider Fedora project, it was felt that a toolbox container is a generic concept that appeals to a lot many more communities than just Fedora. You can also think of it as a nod to coreos/toolbox which served as the original inspiration for the project, and there are plans to use it in Fedora CoreOS too.

If you’re curious, here’s a subset of the discussion that drove the renaming.

There have already been two releases with the new name, so I assume that almost all users have been migrated.

Note that the name of the base OCI image for creating Fedora toolbox containers is still fedora-toolbox for obvious namespacing reasons, but the names of the client-side command line tool, and the overall project itself have changed. That way you could have a debian-toolbox, a centos-toolbox and so on.

It should be obvious, but the Toolbox logo was designed and created by Jakub Steiner.

Advertisements

Written by Debarshi Ray

3 April, 2019 at 19:39

About -Wextra and -Wcast-function-type

leave a comment »

About eight months ago, around the time when GCC 8.x started showing up on my computers, I started moving my code away from using -Wextra. This aligns nicely with the move to the Meson build system, which is nice; but went against the flow of Autotools’ AX_COMPILER_FLAGS, which isn’t ideal but is an acceptable trade-off.

But why?

GCC 8.x added a warning called -Wcast-function-type to the -Wextra umbrella. It warns when a function pointer is cast to an incompatible function. At a glance, this seems desirable, but it isn’t. It runs contrary to one of the widely used C idioms in GNOME. For example, it’s triggered by this text book use of g_list_copy_deep to copy a list of reference counted objects:

another_list = g_list_copy_deep (list, (GCopyFunc) g_object_ref, NULL);

Note that this is different from -Wincompatible-pointer-types, which would’ve triggered if the cast to GCopyFunc was missing:

another_list = g_list_copy_deep (list, g_object_ref, NULL);

It’s easy to imagine similar examples with uses of gtk_container_forall or gtk_container_foreach with gtk_widget_destroy, and so on.

The C standard (eg., see article 6.5.2.2 of the C11 standard) steps around the issue of passing more arguments to a function than it actually has parameters for by calling it undefined behaviour. However, the calling conventions of all the platforms supported by GLib are defined in a way to make this work.

So how do we disable -Wcast-function-type?

One option is to use a compiler directive with #pragma as suggested by AX_COMPILER_FLAGS. However, attempts to ignore it through a #pragma on older versions of GCC that didn’t have this specific warning will trigger -Wpragmas, and, ironically, using G_GNUC_CHECK_VERSION to conditionally disable it on newer compilers will trigger -Wexpansion-to-defined, again, because of -Wextra and the fact that the implementation of the macro has a #ifdef around __GNUC__. Regardless, for a warning that gets triggered by such a widely used programming construct, it would lead to a ton of boilerplate all over the codebase, instead of being a solitary exception tucked away in one corner of the project.

Therefore, my preference has been to append -Wno-cast-function-type and -Wno-error=cast-function-type to the list of compiler flags of modules using -Wextra. This avoids almost all of the above problems. One small wrinkle is that if a translation unit or file does trigger some other unrelated diagnostic, then an older compiler will also emit -Wunknown-warning for the presence of the unknown -Wno-cast-function-type flag. I find this acceptable because, in the first place, a file shouldn’t trigger any other diagnostic, especially on an older compiler, and if there does happen to be something, say a deprecation warning, then it’s likely something that needs to be fixed anyway.

Given that this can repeat with future versions of GCC, it seems wiser to avoid -Wextra and instead explicitly list out the desired compiler warnings. This isn’t hard to do because the GCC documentation clearly marks which warnings are turned on by -Wextra, -Wpedantic, etc..

Written by Debarshi Ray

1 April, 2019 at 13:05

Posted in GNOME, GNU, C

GNOME Photos: an overview of zooming

with 2 comments

I was recently asked about how zooming works in GNOME Photos, and given that I spent an inordinate amount of time getting the details right, I thought I should write it down. Feel free to read and comment, or you can also happily ignore it.

Smooth zooming

One thing that I really wanted from the beginning was smooth zooming. When the user clicks one of the zoom buttons or presses a keyboard shortcut, the displayed image should smoothly flow in and out instead of jumping to the final zoom level — similar to the way the image smoothly shrinks in to make way for the palette when editing, and expands outwords once done. See this animated mock-up from Jimmac to get an idea.

For the zooming to be smooth, we need to generate a number of intermediate zoom levels to fill out the frames in the animation. We have to dish out something in the ballpark of sixty different levels every second to be perceived as smooth because that’s the rate at which most displays refresh their screens. This would have been easier with the 5 to 20 megapixel images generated by smart-phones and consumer-grade digital SLRs; but just because we want things to be sleek, it doesn’t mean we want to limit ourselves to the ordinary! There is high-end equipment out there producing images in excess of a hundred megapixels and we want to robustly handle those too.

Downscaling by large factors is tricky. When we are aiming to generate sixty frames per second, there’s less than 16.67 milliseconds for each intermediate zoom level. All we need is a slightly big zoom factor that stresses the CPU and main memory just enough to exceed our budget and break the animation. It’s a lot more likely to happen than a pathological case that crashes the process or brings the system to a halt.

Mipmaps to the rescue!

A 112.5 megapixel or 12500×9000 image being smoothly zoomed in and out on an Intel Kaby Lake i7 with a HiDPI display. At the given window size, the best fit zoom level is approximately 10%. On a LoDPI display it would’ve been 5%. Note that simultaneously encoding the screencast consumes enough extra resources to make it stutter a bit. That’s not the case otherwise.

Photos uses GEGL to deal with images, and image pixels are held in GeglBuffers. Each GeglBuffer implicitly supports 8 mipmap levels. In other words, a GeglBuffer not only has the image pixels at the original resolution, or level zero, at which they were fed into the buffer, but it also caches progressively lower resolution representations of it. For example, at 50% or level one, at 25% or level two, and so on.

This means that we never downscale by more than a factor of two during an animation. If we want to zoom an image down to 30%, we take the first mipmap level, which is already cached at 50%, and from there on it’s just another 60% to reach the originally intended zoom target of 30%. Knowing that we won’t ever have to downscale by more than a factor of two in a sensitive code path is a relief.

But that’s still not enough.

It doesn’t take long to realize that the user barely catches a fleeting glimpse of the intermediate zoom levels. So, we cut corners by using the fast but low quality nearest neighbour sampler for those; and only use a higher quality box or bilinear sampler, depending on the specific zoom level, for the final image that the user will actually see.

With this set-up in place, on the Intel Kaby Lake i7 machine used in the above video, it consistently takes less than 10 milliseconds for the intermediate frames, and less than 26 milliseconds for the final high quality frame. On an Intel Sandybridge i7 with a LoDPI display it takes less than 5 and 15 milliseconds respectively, because there are less pixels to pump. On average it’s a lot more faster than these worst case figures. You can measure for yourselves using the GNOME_PHOTOS_DEBUG environment variable.

A lot of the above was enabled by Øyvind Kolås’ work on GEGL. Donate to his fund-raiser if you want to see more of this.

There’s some work to do for the HiDPI case, but it’s already fast enough to be perceived as smooth by a human. Look at the PhotosImageView widget if you are further interested.

An elastic zoom gesture

While GTK already comes with a gesture for recognizing pinch-to-zoom, it doesn’t exactly match the way we handle keyboard, mouse and touch pad events for zooming. Specifically, I wanted the image to snap back to its best fit size if the user tried to downscale beyond it using a touch screen. You can’t do that with any other input device, so it makes sense that it shouldn’t be possible with a touch screen either. The rationale being that Photos is optimized for photographic content, which are best viewed at their best fit or natural sizes.

For this elastic behaviour to work, the semantics of how GtkGestureZoom calculates the zoom delta had to be reworked. Every time the direction of the fingers changed, the reference separation between the touch points relative to which the delta is computed must be reset to the current distance between them. Otherwise, if the fingers change direction after having moved past the snapping point, the image will abruptly jump instead of sticking to the fingers.

The image refuses to become smaller than the best fit zoom level and snaps back. Note that simultaneously encoding the screencast consumes enough extra resources to make it stutter a bit. That’s not the case otherwise.

With some help from Carlos Garnacho, we have a custom gesture that hooks into GtkGestureZoom’s begin and update signals to implement the above. The custom gesture is slightly awkward because GtkGestureZoom is a final class and can’t be derived, but it’s not too bad for a prototype. It’s called PhotosGestureZoom, in case you want to look it up.

The screencasts feature a 112.5 megapixel or 12500×9000 photo of hot air balloons at ClovisFest taken by Soulmates Photography / Daniel Street available under the Creative Commons Attribution-Share Alike 3.0 Unported license.

The touch points were recorded in an X session with a tool written by Carlos Garnacho.

Written by Debarshi Ray

8 February, 2019 at 18:36

Posted in C, Fedora, GEGL, GNOME, GTK+, Photos

Fedora Toolbox — Under the hood

with 2 comments

rpm-ostree-flatpak-silverblue

A few months ago, we had a glimpse at Fedora Toolbox setting up a seamlessly integrated RPM based environment, complete with dnf, on Fedora Silverblue. But isn’t dnf considered a persona non grata on Silverblue? How is this any different from using the existing Fedora Workstation then? What’s going on here?

Today we shall look under the covers to answer some of these questions.

The problem

The immutable nature of Silverblue makes it difficult to install arbitrary RPMs on the operating system. It’s designed to install graphical applications as Flatpaks, and that’s it. This has many advantages. For example, robust upgrades.

However, there are legitimate cases when one does want to install some random RPMs. For example, when you need things like *-devel packages, documentation, GCC, gofmt, strace, valgrind or whatever else is necessary for your development workflow. While rpm-ostree does offer a way around this, it’s painful to have to reboot every time you change the set of packages on the system, and it negates the advantages of immutability in the first place.

Containers

By this time some of you are surely thinking that containers ought to solve this somehow; and you’d be right. Fedora Toolbox uses containers to set up an elaborate chroot-like environment that’s separate from the immutable OSTree based operating system.

And once you are down to containers, Docker isn’t far away — surely this can be hacked together with Docker; and you’d be right again. Almost. You can hack it up with Docker but it wouldn’t be ideal.

The problem with Docker is that it requires root privileges. Every time you invoke the docker command, it has to be prefixed with sudo or be run as root. That’s fine if you all you want is a place to install some RPMs. It would’ve required root anyway. However, it’s annoying if you want GNOME Terminal to default to running a shell inside your RPM based development environment. You’d have to enter the root password to even get to an unprivileged shell prompt.

So, instead of using Docker, Fedora Toolbox uses something called Podman. Podman is a fully-featured container engine that aims to be a drop-in replacement for Docker. Thanks to the Open Container Initiative (or OCI) standardizing the interfaces to Docker images and runtimes, every OCI container and image can be used with either Docker or Podman.

The good thing about Podman is that can be used rootless — that is, without root privileges. So, that’s great.

Containers are weird, though

Containers are pretty widely popular these days, but not everybody who is transitioning from the current RPM based Fedora Workstation to Silverblue can be expected to set things up from first principles using nothing but the podman command line. It will surely increase the cognitive load of undergoing the transition, hindering Silverblue adoption.

Even if someone familiar with the technology is able to set things up, pitfalls abound. For example, why is the display server not working, why is the SSH agent not working, why are OpenGL and Vulkan programs not working, or why is sshfs not working, or why LLVM and LibreOffice are failing to build, etc..

Let’s be honest. The number of people who understand both container technology and the workings of a modern graphical operating system well enough to sort these problems in a jiffy is vanishingly small. I know that at least I don’t belong to that group.

Container images are optimized for non-interactive use and size, whereas we are talking about the interactive shell running in your virtual terminal emulator. For example the fedora OCI image comes with the coreutils-single RPM, which doesn’t have the same user experience as the coreutils package that we are all familiar with.

So, it’s clear that we need a pre-configured, and at times, opinionated, solution on top of Podman.

The solution

Fedora Toolbox starts with the similarly named fedora-toolbox OCI image hosted on the Fedora Container Registry. There’s one for every Fedora branch. Currently those are Fedoras 28, 29 and 30. These images are based on the fedora image, with an altered package set to offer an interactive user experience that’s similar to the one on Silverblue.

When you invoke the fedora-toolbox create command, it pulls the image from the registry, and then tailors it to the local user. It creates a user with a UID matching $UID, a home directory matching $HOME and the right group memberships; and it ensures that various bits and pieces from the host, such as the home directory, the display server, the D-Bus instances, various pieces of hardware, etc. are available inside the container. These customizations are saved as another image named fedora-toolbox-user. Finally, an OCI container, also named fedora-toolbox-user, is created out of this image.

If you are curious, run podman images and podman ps –all to verify the above.

Once the toolbox container has been created, subsequent fedora-toolbox enter commands execute the users preferred shell inside it, giving the impression of being in an alternate RPM flavoured reality on a Silverblue system.

If you are still curious, then open /usr/bin/fedora-toolbox and have a peek. It’s just a shell script, after all.

Written by Debarshi Ray

21 January, 2019 at 21:20

Fedora Toolbox — Hacking on Fedora Silverblue

with 6 comments

rpm-ostree-flatpak-silverblue

Fedora Silverblue is a modern and graphical operating system targeted at laptops, tablets and desktop computers. It is the next-generation Fedora Workstation that promises painless upgrades, clear separation between the OS and applications, and secure and cross-platform applications. The basic operating system is an immutable OSTree image, and all the applications are Flatpaks.

It’s great!

However, if you are a hacker and decide to set up a development environment, you immediately run into the immutable OS image and the absence of dnf. You can’t install your favourite tools, editors and SDKs the way you’d normally do on Fedora Workstation. You can either unlock your immutable OS image to install RPMs through rpm-ostree and give up the benefit of painless upgrades; or create a Docker container to get an RPM-based toolbox but be prepared to mess around with root permissions and having to figure out why your SSH agent or display server isn’t working.

Enter Fedora Toolbox.

It makes it trivial to get a mutable development environment on Silverblue:

[rishi@bollard ~]$ fedora-toolbox create
[rishi@bollard ~]$ fedora-toolbox enter
🔹[rishi@toolbox ~]$

It uses OCI containers underneath, but takes away the cognitive overhead of thinking about containers by providing a seamless integration with the host environment. It uses rootless podman and buildah, so there’s no root in the picture either.

fedora-toolbox

If you are going to try it out, make sure that you have the runc-1.0.0-56.dev.git78ef28e package in your Silverblue image. There’s also an ongoing review to get fedora-toolbox added to Fedora. If you don’t feel comfortable mucking around with rpm-ostree on the command-line, then fear not. Very soon all the necessary pieces will be part of the OS image, making it that much easier to start hacking on your Silverblue.

Written by Debarshi Ray

22 October, 2018 at 20:07

GNOME Terminal: a little something for Fedora 29

with 5 comments

Can you spot what that is?

GNOME Terminal: Fedora 29 teaser

Written by Debarshi Ray

24 May, 2018 at 12:14

Posted in Fedora, GNOME, GTK+, Terminal, VTE

GNOME Terminal: separate menu items for opening tabs and windows

with one comment

Astute users might have noticed that the GNOME Terminal binary distributed by Fedora has separate menu items for opening new tabs and windows, while the vanilla version available from GNOME doesn’t.

gnome-terminal-menuitems-tabs-windows

With separate menu items

This has been the case since Fedora 25 and was achieved by a downstream patch that reverted two upstream commits.

gnome-terminal-menuitems-unified-tabs-windows

Without separate menu items

I am happy to say that since version 3.28 GNOME Terminal has regained the ability to have separate menu items as a compile time option. The gnome-terminal-server binary needs to be built with the DISUNIFY_NEW_TERMINAL_SECTION pre-processor macro defined. Here’s one way to do so.

Written by Debarshi Ray

11 May, 2018 at 14:42

Posted in Fedora, GNOME, GTK+, Terminal, VTE