Debarshi's den

Archive for the ‘Fedora’ Category

GNOME Photos: an overview of thumbnailing

leave a comment »

From time to time, I find myself being asked about various details about how content is thumbnailed in GNOME Photos, and the reasons behind various implementation decisions. I can never remember all the details, and always have to dig through Git history and bug reports across multiple modules to come up with an answer. I am hoping that this brain dump will be more persistent than my memory, and more holistic than random comments here and there.

Feel free to read and comment, or you can also happily ignore it.

Background

Having accurate and quality thumbnails is absolutely crucial for Photos. The main user interface is a grid of thumbnails. By design, it tries hard not to expose the filesystem, which means that the user doesn’t have the path or directory hierarchy to complement the contents of the grid. In comparison, thumbnails can be optional in a file manager. Note how Files has settings to disable thumbnailing, and defaults to not thumbnailing remote content, but users can still go about interacting with their files.

Thumbnailing in GNOME is spread across GIO, GVfs, GnomeDesktopThumbnailFactory, and together they implement the Thumbnail Managing Standard. Usually, one uses GIO to lookup thumbnails from the cache and the state they are in, while GnomeDesktopThumbnailFactory is used to create and store the thumbnail files. These thumbnails are stored in the global thumbnail cache in $XDG_CACHE_HOME/thumbnails, and are often, but not necessarily, created by the thumbnailers listed under /usr/share/thumbnailers. This is how most components (eg., GTK+’s GtkFileChooserWidget), and applications (eg., Files and Videos) show thumbnails.

Then there are those “odd” ones that have their own custom setup.

Prior to version 3.24, Photos entirely relied on the global cache and the aforementioned GNOME APIs for its thumbnails. That changed in 3.24 when it switched to its own custom thumbnailer and application specific cache.

Requirements

Ever since editing was added in 3.20, we felt the need to ensure that the thumbnail represents the current state of each item. Being a non-destructive editor, Photos never modifies the original file but separately serializes the edits to disk. The image is rendered by loading the original file, deserializing the edits into objects in memory and running the pixels through them [1]. Therefore, to have the thumbnails accurately represent the current state of the item, it would have to do something similar. However, the edits are application-specific [2], so it is not reasonable to expect the generic OS-wide thumbnailers to be able to handle them.

I believe this is a requirement that all non-destructive image editors have [3]. Notable examples are Darktable and Shotwell.

Secondly, it is important to be able to create and lookup thumbnails of a specific size, as opposed to enumerated constants with pre-determined presets.

The standard specifies two sizes – normal, which is 128×128, and large, which is 256×256. I think this was alright in a world without HiPPI, and is also fine if the thumbnails are either too small or are not an existential necessity for the application. For a HiPPI display with a scaling factor of N, we want to make the thumbnail grid as visually appealing as possible by pumping in NxN times more pixels. Since Photos wants the thumbnails to be 256×256 logical pixels, they should be 256Nx256N raw device pixels on HiPPI. To make things complicated, the cache might get used across different scaling factors – either display or disk got switched, multi-monitor with different resolutions, etc..

Upscaling the low-resolution counterpart of a thumbnail by N is still passable, but it looks much worse if the thumbnail is significantly smaller. Although, I must note that this was the easiest hurdle to surmount. It originates from GIO’s desire to fallback to 128×128 thumbnails, even if the application asked for 256×256. This is pretty straightforward to fix, if necessary.

Last but not the least, I find it important to version the cache to tide over bugs in the thumbnailer. If the cache isn’t versioned, then it is difficult to discard thumbnails that might have been generated by a broken thumbnailer. Hopefully, such bugs would be rare enough that it won’t be necessary to invalidate the cache very often, but when they do happen, it is very reassuring to be able to bump the version, and be guaranteed that users won’t be looking at a broken user interface.

Solution

Starting from version 3.24, Photos uses its own out-of-process thumbnailer and cache [4]. The cache is at $XDG_CACHE_HOME/gnome-photos/thumbnails/$SIZE-$GENERATION, where SIZE is the thumbnail size in raw device pixels and GENERATION is the cache’s version. The main application talks to the thumbnailer over peer-to-peer D-Bus and a simple, cancellable private D-Bus API.

The thumbnailer isn’t separately sandboxed, though. It might be an interesting thing to look at for those who don’t use Flatpak, or to restrict it even more than the main application when running inside Flatpak’s sandbox.

Known bugs

Photos’ thumbnailing code can be traced back to its origins in GNOME Documents. They don’t persistently track thumbnailing failures, and will attempt to re-thumbnail an item that had previously failed when any metadata change is detected. In short, they don’t use G_FILE_ATTRIBUTE_THUMBNAILING_FAILED. The current behaviour might help to overcome a temporary glitch in the network, or it can be simply wasteful.

They predate the addition of G_FILE_ATTRIBUTE_THUMBNAIL_IS_VALID and don’t update the thumbnail once an item gets updated. This could have still been done using GnomeDesktopThumbnailFactory, but that’s water under the bridge, and should possibly be fixed. Although, images don’t tend to get updated so often, which is probably why nobody notices it.

Related to the above point, currently the modification time of the original doesn’t get stored in the thumbnail. It slipped through the cracks while I was reading the sources of the various modules involved in creating thumbnails in GNOME. However, a versioned cache makes it possible to fix it.

[1] If you are reading between the lines, then you might be thinking that it is serializing and deserializing GeglOperations, and you’d be right.

[2] GEGL might be a generic image processing library with its set of built-in operations, but for various reasons, an application can end up carrying its own custom operations.

[3] The idea of an application storing its edits separately from the original can strike as unusual, but this is how most modern image editors work.

[4] Both Darktable and Shotwell have similar thumbnailing infrastructure. You can read about them here and here respectively.

Advertisements

Written by Debarshi Ray

29 January, 2018 at 17:17

Posted in C, Fedora, Flatpak, GEGL, GNOME, GTK+, GVfs, Photos

Fedora meets RHEL

with 6 comments

As we enter the final freeze before the Fedora 27 Workstation release, I’d like to highlight a new feature that will hopefully make Fedora more attractive for developers. Last month, I had written about our experiments to make it easier to consume gratis, self-supported Red Hat Enterprise Linux installations from Fedora Workstation. I am happy to report that this is now a reality.

gnome-boxes-new-source-selection-rhel

Starting from Fedora 27 Workstation, you’ll be able to install an infinite number of RHEL 7.x VMs right from inside GNOME Boxes. All you need is an account on developers.redhat.com, and it will automatically set up a RHEL virtual machine that’s entitled to the RHEL Developer Suite subscription.

gnome-boxes-new-source-selection-rhel-01

Thanks to Felipe Borges for a seemingly endless round of patch reviews, and Fabiano and Victor for occasionally lending us their brain.

Written by Debarshi Ray

20 October, 2017 at 16:36

Downloading RHEL 7 ISOs for free

with one comment

A year and a half ago, frighteningly close to 1st April, Red Hat announced the availability of a gratis, self-supported, developer-only subscription for Red Hat Enterprise Linux and a series of other products. Simply put, if you went to developers.redhat.com, created an account and clicked a few buttons, you could download a RHEL ISO without paying anything to anybody. For the past few months, I have been investigating whether we can leverage this to do something exciting in Fedora Workstation. Particularly for those who might be building applications on Fedora that would eventually be deployed on RHEL.

While trying to figure out how the developers.redhat.com website and its associated infrastructure works, I discovered that its source code is actually available on GitHub. Sadly, my ineptitude with server-side applications and things like JBoss, Ruby, etc. meant that it wasn’t enough for me to write code that could interface with it. Luckily, the developers were kind enough to answer my questions, and now I know enough to write a C program that can download a free RHEL ISO.

The code is here: download-rhel.c.

I’ll leave it here in the hope that some unknown Internet traveller might find it useful. As for Fedora Workstation, I’ll let you know if we manage to come up with something concrete. 😉

Written by Debarshi Ray

14 September, 2017 at 14:34

Custom terminal titles are back in Fedora

with 3 comments

Almost four years ago, in GNOME 3.12, the ability to have custom terminal titles was removed from gnome-terminal. As is wont to happen, users who dealt with scores of similar looking terminal tabs and windows were quick to express their grief at this loss.

Thankfully, a year ago, Christian partly restored it by bringing back the --title command line option. It helped, but it still wasn’t enough.

Anyway, the good news is that custom terminal titles have been restored in their entirety since Fedora 25.

I must point out that this is a downstream patch carried by Fedora. If you want, you can ask your distributor to include it. Versions of the patch applicable to different GNOME branches can be found in this Git tree.

Written by Debarshi Ray

27 February, 2017 at 20:36

Posted in Fedora, GNOME, Terminal

New GNOME API Key for Google Services

with 3 comments

Tl;dr: update evolution-data-server to stop your client from misbehaving; update gnome-online-accounts to shield yourself from other buggy clients.

Recently, a few bugs in evolution-data-server were causing various GNOME components to hit Google’s daily limit for their CalDAV and Tasks APIs. At least evolution, gnome-calendar and gnome-todo were affected. The bugs have since been fixed, but until every single user out there installs the fix, everybody will be susceptible even if they have a fixed copy of evolution-data-server. This is because Google identifies the clients by the OAuth 2.0 API key used to access their services, and not the version of the code running on them.

evolution-data-server-quota-exceeded-error-message

We are therefore phasing out the old Google API key used by GNOME so that users updating their systems will have no connection to the one that was tainted by these bugs.

Fedora Users

If you are using Fedora 25, make sure you have evolution-data-server-3.22.3-1.fc25 and gnome-online-accounts-3.22.3-1.fc25. For Fedora 24, the versions are evolution-data-server-3.20.6-1.fc24 and gnome-online-accounts-3.20.5-1.fc24.

GNOME Distributors

If you are distributing GNOME, make sure that you are shipping evolution-data-server-3.22.3 and gnome-online-accounts-3.22.3, or evolution-data-server-3.20.6 and gnome-online-accounts-3.20.5.

Written by Debarshi Ray

15 December, 2016 at 18:11

#photos happenings

with 3 comments

Out of a mojito bar in South Beach with a lobotomised plastic picnic spoon and a crew of control freaks.

3.22 is here

We recently released GNOME 3.22. It will be in Fedora Workstation 25. Go look at the video — it’s awesome!

GNOME Photos has again taken significant strides forward – just like we did six months ago in 3.20. One of the big things that we added this time was sharing. This nicely rounds out our existing online acccounts integration, and complements the work we did on editing six months ago.

gnome-photos-sharing

Sharing is an important step towards a more tightly integrated online account experience in GNOME. We have been interested in a desktop-wide sharing service for some time. With Flatpak portals becoming a reality, I hope that the sharing feature in Photos can be spun off into a portal for GNOME.

Thanks to Umang Jain, our GSoC intern this summer for working on sharing.

We overhauled a lot of hairy architectural issues, which will let us have nicer overview grids in the near future. Alessandro created a Flatpak. This means that going forward, you can easily try out the nightly builds of Photos thanks to the Flatpak support in GNOME Software 3.22.

gnome-photos-flatpak2

Thanks to Kalev Lember for the wonderful screenshot.

The future

I think that we are reaching a point where we can recommend Photos to a wider group of users. With editing and sharing in place, we have filled some of the bigger gaps in the user experience that we want to offer. Yes, there are some missing features and rough edges that we are aware of, so we we are going to spend the next six months addressing the ones that are most important. You can look at our roadmap for the full picture, but I am going to highlight a few.

Better overview grids (GNOME #690623)

We have been using GtkIconView to display the grid of thumbnails that we call the overview. GtkIconView has been around for a long while, but it has some issues – both visual and performance. Therefore, we want to replace it with GtkFlowBox so (a) that the application remains responsive while we are populating the grid, and (b) we can have really pretty visuals.

Eventually, we want this:

photos-photos

Import from device (GNOME #751212)

This is one of the biggest missing features, in my opinion. We really need a way to import content from removable devices and cameras that doesn’t involve mucking around with files and directories.

Petr Stetka has already started working on this, but I am sure he will appreciate any help with this.

More sharing (GNOME #766031)

Last but not the least, I definitely like showing off on Facebook and so do you! So I want to add a Facebook share-point and possibly a few more.

Come, join us

If any of this interests you, then feel free to jump right in. We have a curated list of newcomer bugs and a guide for those who are relatively new. If you are an experienced campaigner, you can look at the roadmap for more significant tasks.

For any help, discussions or general chitchat, #photos on GIMPNet is the place to be.

Written by Debarshi Ray

22 September, 2016 at 17:18

MALLOC_PERTURB_

with 4 comments

MALLOC_PERTURB_ is a useful thing. If you are developing on glibc-based systems, I encourage you to put this snippet in your ~/.bash_profile:

MALLOC_PERTURB_=$(($RANDOM % 255 + 1))
export MALLOC_PERTURB_

I have been using it for the last six years on all my computers (3 laptops running every Fedora x86_64 build released since then), and while things haven’t exploded, it has helped uncover the odd bug every once in a while. One such occasion presented itself this week.

I was busy following Ondrej’s hint, debugging why Eye of GNOME was taking so long to open a file from ownCloud. Imagine my shock when it would just crash after showing the window. The same optimization was working just fine on the gnome-3-18 branch, while master was crashing even without any changes. How could that happen? Obviously, while it was failing for me, it was working fine for all those who run unstable GNOME versions via jhbuild, gnome-continous, Fedora rawhide, etc.. Otherwise we would have been debugging this crash, and not a performance issue.

I guess, most of them didn’t have MALLOC_PERTURB_.

Here is another such story.

In case you were wondering, there is already an update on its way to Fedora 24 address the crash.

Written by Debarshi Ray

9 April, 2016 at 02:28