Debarshi's den

Image wrangling with GEGL: an introduction

with 3 comments

One of the core dependencies of GNOME Photos, other than GTK+ and Tracker, is a library called GEGL. It is a GObject-based image processing library primarily developed for GIMP. GEGL is used by Photos to load pixels from files, create thumbnails, edit, share and export images.

Unfortunately, even though GEGL is a powerful and generic image processing framework, it can be hard to find documentation and code samples to refer to, and the pool of people who understand it well enough is relatively small. I am going to do a series of blog posts to address this by feeding the search engines. Hopefully this will be useful for new contributors to GIMP and GNOME Photos, and some of it can be folded back into the reference GEGL documentation; or maybe it will encourage adoption in new and interesting places.

Nodes and operations

Processing images with GEGL requires the creation of a graph, represented by a GeglNode. A GeglNode can either have a number of child nodes connected to each other forming a directed acyclic graph, or it can have a GeglOperation. An operation is where the actual image processing takes place. Multiple operations are chained together in a graph to obtain the desired outcome.

This is enough to get started with some basic effects and enhancements. Here is a snippet takes a path to an input image, enhances the blacks and saves it as a PNG.

  #include <gegl.h>
  …
  g_autoptr (GeglNode) graph = NULL;
  GeglNode *exposure;
  GeglNode *load;
  GeglNode *sink;

  graph = gegl_node_new ();
  load = gegl_node_new_child (graph,
                              "operation", "gegl:load",
                              "path", /* input path as C string */,
                              NULL);
  exposure = gegl_node_new_child (graph,
                                  "operation", "gegl:exposure",
                                  "black-level", 0.03,
                                  NULL);
  sink = gegl_node_new_child (graph,
                              "operation", "gegl:png-save",
                              "bitdepth", 8,
                              "path", /* output path as C string */,
                              NULL);

  gegl_node_link_many (load, exposure, sink, NULL);
  gegl_node_process (sink);

Notice the many similarities with GStreamer.

There is a whole list of such filters to choose from. Such as gegl:cartoon to simulate a cartoon drawn with a black felt pen, gegl:mosaic to transform an image into a mosaic, gegl:saturation to change the colourfulness of the image, or gegl:posterize, which is the guts of the similarly named tool in GIMP.

example-00

Buffers

Image pixels are held in a GeglBuffer. Most applications will want to directly interact with a GeglBuffer at one point or the other. For example, to decode an image file and carry the pixels around instead of repeatedly decoding them off the storage. In the above code sample, the buffers were implicitly created by GEGL unbeknownst to us, but we can use a similar graph to load pixels off a file into a GeglBuffer.

  #include <gegl.h>
  …
  g_autoptr (GeglBuffer) buffer = NULL;
  g_autoptr (GeglNode) graph = NULL;
  GeglNode *load;
  GeglNode *sink;

  graph = gegl_node_new ();
  load = gegl_node_new_child (graph,
                              "operation", "gegl:load",
                              "path", /* input path as C string */,
                              NULL);
  sink = gegl_node_new_child (graph,
                              "operation", "gegl:buffer-sink",
                              "buffer", &buffer,
                              NULL);

  gegl_node_link_many (load, sink, NULL);
  gegl_node_process (sink);

A loaded buffer can be then fed into a graph using a gegl:buffer-source.

As the custodian of pixels, GeglBuffer is similar to the role played by GdkPixbuf, but it has some extra features that are handy for image processing.

Most notably, a GeglBuffer is designed to handle massive images that are larger than the amount of physical RAM available on the system. Instead of holding all the pixels in a linear sequence of bytes, it splits them up into small tiles that can be paged out into a file when not in use. However, if necessary, it is possible to optionally dumb down a GeglBuffer by setting it up to use a single array of bytes, or forcing all tiles to be held in RAM.

A GeglBuffer is not restricted to a single pixel format such as RGB with 8 bits per channel. It can transparently handle a whole horde of formats — monochrome, Lab, HSL, etc. with different degrees of precision per channel. Finally, it is mipmap-capable.

All these features make GeglBuffer a very sophisticated data structure for storing image pixels. However, they aren’t that important for an introduction to GEGL, so we will save them for a future article.

Happy hacking

This is enough to start playing with GEGL. Here is the code used to create the above image, and is proof that knowing just this much is enough to do practically useful things.

Advertisements

Written by Debarshi Ray

20 November, 2017 at 13:30

Posted in GNOME, Photography, C, Photos, GEGL, GIMP

Fedora meets RHEL

with 5 comments

As we enter the final freeze before the Fedora 27 Workstation release, I’d like to highlight a new feature that will hopefully make Fedora more attractive for developers. Last month, I had written about our experiments to make it easier to consume gratis, self-supported Red Hat Enterprise Linux installations from Fedora Workstation. I am happy to report that this is now a reality.

gnome-boxes-new-source-selection-rhel

Starting from Fedora 27 Workstation, you’ll be able to install an infinite number of RHEL 7.x VMs right from inside GNOME Boxes. All you need is an account on developers.redhat.com, and it will automatically set up a RHEL virtual machine that’s entitled to the RHEL Developer Suite subscription.

gnome-boxes-new-source-selection-rhel-01

Thanks to Felipe Borges for a seemingly endless round of patch reviews, and Fabiano and Victor for occasionally lending us their brain.

Written by Debarshi Ray

20 October, 2017 at 16:36

Stable GNOME Photos Flatpaks moved to Flathub

leave a comment »

Starting from version 3.26, the stable GNOME Photos Flatpaks have been moved to Flathub. They are no longer available from GNOME’s Flatpak repository.

To migrate, first delete the old build:

$ flatpak uninstall org.gnome.Photos/x86_64/stable

Then install it from Flathub:

$ flatpak remote-add --from flathub https://flathub.org/repo/flathub.flatpakrepo
$ flatpak install flathub org.gnome.Photos

Note that this is only about the stable build. The nightly continues to be available from its existing location in GNOME’s repository. You can keep updating it with:

$ flatpak update --user org.gnome.Photos/x86_64/master

Written by Debarshi Ray

10 October, 2017 at 16:20

Posted in Flatpak, GNOME, Photos

Downloading RHEL 7 ISOs for free

with one comment

A year and a half ago, frighteningly close to 1st April, Red Hat announced the availability of a gratis, self-supported, developer-only subscription for Red Hat Enterprise Linux and a series of other products. Simply put, if you went to developers.redhat.com, created an account and clicked a few buttons, you could download a RHEL ISO without paying anything to anybody. For the past few months, I have been investigating whether we can leverage this to do something exciting in Fedora Workstation. Particularly for those who might be building applications on Fedora that would eventually be deployed on RHEL.

While trying to figure out how the developers.redhat.com website and its associated infrastructure works, I discovered that its source code is actually available on GitHub. Sadly, my ineptitude with server-side applications and things like JBoss, Ruby, etc. meant that it wasn’t enough for me to write code that could interface with it. Luckily, the developers were kind enough to answer my questions, and now I know enough to write a C program that can download a free RHEL ISO.

The code is here: download-rhel.c.

I’ll leave it here in the hope that some unknown Internet traveller might find it useful. As for Fedora Workstation, I’ll let you know if we manage to come up with something concrete. 😉

Written by Debarshi Ray

14 September, 2017 at 14:34

GtkBuilder, Vala and WebKit

leave a comment »

This article is about a set of bugs that used to exist, and are in various stages of getting fixed. As such this is merely a historical anecdote. I had planned to write about something entirely different, and hopefully more useful, but I ended up having too much fun with this to just ignore it.

To use a WebKitWebView inside a GTK+ template, one needs to workaround the fact that WebKitWebView breaks the heuristics in GtkBuilder to guess the GType from the human readable type name. That’s easy. Anybody who has used GObject is likely to have encountered some dialect of g_type_ensure, or, as the more learned will point out, GtkBuilder has a type-func attribute for cases like these.

The fun begins when you start debating which workaround to use.

It turns out that type-func doesn’t work with Vala.

A few buglets in GtkBuilder means that if you use class and type-func together, the latter will be ignored. It’s likely nobody used them together because even if class is specified as mandatory the parser doesn’t enforce that. On the other hand, the Vala compiler effectively treats class as mandatory because it doesn’t understand type-func. So, you must use both to avoid a build failure, but if you do, you get a run-time failure because your type-func is ignored.

So, typeof (WebKit.WebView) wins, which is Vala’s equivalent of g_type_ensure.

I don’t know how things are with other language bindings. Vala is what I happen to be using right now, so that’s where I chose to focus.

Thanks to Saiful, for pointing out the problem with WebKitWebView and GtkBuilder. It was fascinating.

Written by Debarshi Ray

29 August, 2017 at 09:24

Posted in Blogroll, C, GNOME, GTK+, Vala, WebKit

GdMainBox — the new content-view widget in libgd

with 4 comments

Now that I have written at length about the new fluid overview grids in GNOME Photos, it is time to talk a bit about the underlying widgets doing the heavy lifting. Hopefully some of my fellow GNOME developers will find this interesting.

Background

Ever since its incubation inside Documents, libgd has had a widget called GdMainView. It is the one which shows the grid or list of items in the new GNOME applications — Boxes, Photos, Videos, etc.. It is where drag-n-drop, rubber band selection and the selection mode pattern are implemented.

However, as an application developer, I think its greatest value is in making it trivial to switch the main content view from a grid to a list and back. No need to worry about the differences in how the data will be modelled or rendered. No need to worry about all the dozens of little details that arise when the main UI of an application is switched like that. For example, this is all that the JavaScript code in Documents does:

  let view = new Gd.MainView({ shadow_type: Gtk.ShadowType.NONE });
  …
  view.view_type = Gd.MainViewType.LIST; // use a list
  …
  view.view_type = Gd.MainViewType.ICON; // use a grid


Unfortunately, GdMainView is based on GtkIconView and GtkTreeView. By this time we all know that GtkIconView has various performance and visual problems. While GtkTreeView might not be slow, the fact that it uses an entirely separate class of visual elements that are not GtkWidgets limits what one can render using it. That’s where GdMainBox comes in.

GdMainBox

GdMainBox is a replacement for GdMainView that is meant to use GtkFlowBox and GtkListBox instead.

GListModel *model;
GtkWidget *view;

model = /* a GListModel containing GdMainBoxItems */
view = gd_main_box_new (GD_MAIN_BOX_ICON);
gd_main_box_set_model (GD_MAIN_BOX (view), model);
g_signal_connect (view,
                  "item-activated",
                  G_CALLBACK (item_activated_cb),
                  data);
g_signal_connect (view,
                  "selection-mode-request",
                  G_CALLBACK (selection_mode_request_cb),
                  data);
g_signal_connect (view,
                  "selection-changed", /* not view-selection-changed */
                  G_CALLBACK (selection_changed_cb),
                  data);


If you are familiar with with old GdMainView widget, you will notice the striking similarity with it. Except one thing. The data model.

GdMainView expected applications to offer a GtkTreeModel with a certain number of columns arranged in a certain order with certain type of values in them. Nothing surprising since both GtkIconView and GtkTreeView rely on the existence of a GtkTreeModel.

In the world of GtkListBoxes and GtkFlowBoxes, the data model is GListModel, a list-like collection of GObjects [*]. Therefore, instead of columns in a table, they need objects with certain properties, and methods to access them. These are codified in the GdMainBoxItem interface which every rendered object needs to implement. You can look at this commit for an example. A nice side-effect is that an interface is inherently more type-safe than a GtkTreeModel whose expected layout is expressed as enumerated types. The compiler can not assert that a certain column does have the expected data type, so it left us vulnerable to bugs caused by inadvertent changes to either libgd or an application.

But why a new widget?

You can definitely use a GtkFlowBox or GtkListBox directly in an application, if that’s what you prefer. However, the vanilla GTK+ widgets don’t offer all the necessary features. I think there is value in consolidating the implementation of those features in a single place that can be shared across modules. It serves as a staging area for prototyping those features in a reasonably generic way so that they can eventually be moved to GTK+ itself. If nothing else, I didn’t want to duplicate the same code across the two applications that I am responsible for — Documents and Photos.

One particularly hairy thing that I encountered was the difference between how selections are handled by the stock GtkFlowBox and the intended behaviour of the content-view. Other niceties on offer are expanding thumbnails, selection mode, and drag-n-drop.

If you do decide to directly use the GTK+ widgets, then I would suggest that you at least use the same CSS style classes as GdMainBox — “content-view” for the entire view and “tile” for each child.

The future

I mentioned changing lists to grids and vice versa. Currently, GdMainBox only offers a grid of icons because Photos is the only user and it doesn’t offer a list view. That’s going to change when I port Documents to it. When that happens, changing the view is going to be just as easy as it used to be.

gd_main_view_set_view_type (GD_MAIN_BOX (view), GD_MAIN_BOX_LIST);



[*] Yes, it’s possible to use them without a model, but having a GListModel affords important future performance optimizations, so we will ignore that possibility.

Written by Debarshi Ray

29 March, 2017 at 00:06

Posted in Blogroll, C, Documents, GNOME, GTK+, Photos

GNOME Photos 3.24.0

with 7 comments

After exploring new territory with sharing and non-destructive editing over the last two releases, it was time for some introspection. We looked at some of the long-standing problems within our existing feature set and tried to iron out a few of them.

Overview Grids

It was high time that we overhauled our old GtkIconView-based overview grids. Their inability to reflow the thumbnails leads to a an ugly vertical gutter of empty space unless the window is just the right size. The other problem was performance. GtkIconView gets extremely slow when the icons are updated, which usually happens when content is detected for the first time and start getting thumbnailed.

gnome-photos-flowbox-1

Fixing this has been a recurrent theme in Photos since the middle of the previous development cycle. The end goal was to use a GtkFlowBox-based grid, but it involved a lot more work than replacing one user interface component with another.
Too many things relied on the existence of a GtkTreeModel, and had to be ported to our custom GListModel implementation before we could achieve any user-visible improvement. Once all those yaks had been shaved, we finally started working on the widget at the Core Apps Hackfest last year.

Anyway, I am happy that all that effort has to come fruition now.

Thumbnails

Closely related to our overview grids are the thumbnails inside them. Photos has perpetually suffered from GIO’s inability to let an application specifically request a high resolution thumbnail. While that is definitely a fixable problem, the fact that we store our edits non-destructively as serialized GEGL graphs makes it very hard to use the desktop-wide infrastructure for thumbnails. One cannot expect a generic thumbnailer to interpret the edits and apply them to the original image because their representation will vary greatly from one application to another. That led to the other problem where the thumbnails wouldn’t reflect the edited state of an image.

Therefore, starting from version 3.24.0, Photos has its own out-of-process thumbnailer and a separate thumbnail cache. They ensure that the thumbnails are of a suitably high resolution, and the edited state of an image is never ignored.

Exposure and Blacks

Personally, I have been a heavy user of Darktable’s exposure and blacks adjustment tool, and I really missed something like that in GNOME Photos. Ultimately, at this year’s WilberWeek I fixed gegl:exposure to imitate its Darktable counterpart, and exposed it as a tool in Photos. I am happy with the outcome and I have so far enjoyed dogfooding this little addition.

Written by Debarshi Ray

21 March, 2017 at 13:13

Posted in GEGL, GNOME, GTK+, Photos