GVfs | Debarshi's den

Archive for the ‘GVfs’ Category

GNOME Photos: an overview of thumbnailing

From time to time, I find myself being asked about various details about how content is thumbnailed in GNOME Photos, and the reasons behind various implementation decisions. I can never remember all the details, and always have to dig through Git history and bug reports across multiple modules to come up with an answer. I am hoping that this brain dump will be more persistent than my memory, and more holistic than random comments here and there.

Feel free to read and comment, or you can also happily ignore it.

Background

Having accurate and quality thumbnails is absolutely crucial for Photos. The main user interface is a grid of thumbnails. By design, it tries hard not to expose the filesystem, which means that the user doesn’t have the path or directory hierarchy to complement the contents of the grid. In comparison, thumbnails can be optional in a file manager. Note how Files has settings to disable thumbnailing, and defaults to not thumbnailing remote content, but users can still go about interacting with their files.

Thumbnailing in GNOME is spread across GIO, GVfs, GnomeDesktopThumbnailFactory, and together they implement the Thumbnail Managing Standard. Usually, one uses GIO to lookup thumbnails from the cache and the state they are in, while GnomeDesktopThumbnailFactory is used to create and store the thumbnail files. These thumbnails are stored in the global thumbnail cache in $XDG_CACHE_HOME/thumbnails, and are often, but not necessarily, created by the thumbnailers listed under /usr/share/thumbnailers. This is how most components (eg., GTK+’s GtkFileChooserWidget), and applications (eg., Files and Videos) show thumbnails.

Then there are those “odd” ones that have their own custom setup.

Prior to version 3.24, Photos entirely relied on the global cache and the aforementioned GNOME APIs for its thumbnails. That changed in 3.24 when it switched to its own custom thumbnailer and application specific cache.

Requirements

Ever since editing was added in 3.20, we felt the need to ensure that the thumbnail represents the current state of each item. Being a non-destructive editor, Photos never modifies the original file but separately serializes the edits to disk. The image is rendered by loading the original file, deserializing the edits into objects in memory and running the pixels through them [1]. Therefore, to have the thumbnails accurately represent the current state of the item, it would have to do something similar. However, the edits are application-specific [2], so it is not reasonable to expect the generic OS-wide thumbnailers to be able to handle them.

I believe this is a requirement that all non-destructive image editors have [3]. Notable examples are Darktable and Shotwell.

Secondly, it is important to be able to create and lookup thumbnails of a specific size, as opposed to enumerated constants with pre-determined presets.

The standard specifies two sizes – normal, which is 128×128, and large, which is 256×256. I think this was alright in a world without HiPPI, and is also fine if the thumbnails are either too small or are not an existential necessity for the application. For a HiPPI display with a scaling factor of N, we want to make the thumbnail grid as visually appealing as possible by pumping in NxN times more pixels. Since Photos wants the thumbnails to be 256×256 logical pixels, they should be 256Nx256N raw device pixels on HiPPI. To make things complicated, the cache might get used across different scaling factors – either display or disk got switched, multi-monitor with different resolutions, etc..

Upscaling the low-resolution counterpart of a thumbnail by N is still passable, but it looks much worse if the thumbnail is significantly smaller. Although, I must note that this was the easiest hurdle to surmount. It originates from GIO’s desire to fallback to 128×128 thumbnails, even if the application asked for 256×256. This is pretty straightforward to fix, if necessary.

Last but not the least, I find it important to version the cache to tide over bugs in the thumbnailer. If the cache isn’t versioned, then it is difficult to discard thumbnails that might have been generated by a broken thumbnailer. Hopefully, such bugs would be rare enough that it won’t be necessary to invalidate the cache very often, but when they do happen, it is very reassuring to be able to bump the version, and be guaranteed that users won’t be looking at a broken user interface.

Solution

Starting from version 3.24, Photos uses its own out-of-process thumbnailer and cache [4]. The cache is at $XDG_CACHE_HOME/gnome-photos/thumbnails/$SIZE-$GENERATION, where SIZE is the thumbnail size in raw device pixels and GENERATION is the cache’s version. The main application talks to the thumbnailer over peer-to-peer D-Bus and a simple, cancellable private D-Bus API.

The thumbnailer isn’t separately sandboxed, though. It might be an interesting thing to look at for those who don’t use Flatpak, or to restrict it even more than the main application when running inside Flatpak’s sandbox.

Known bugs

Photos’ thumbnailing code can be traced back to its origins in GNOME Documents. They don’t persistently track thumbnailing failures, and will attempt to re-thumbnail an item that had previously failed when any metadata change is detected. In short, they don’t use G_FILE_ATTRIBUTE_THUMBNAILING_FAILED. The current behaviour might help to overcome a temporary glitch in the network, or it can be simply wasteful.

They predate the addition of G_FILE_ATTRIBUTE_THUMBNAIL_IS_VALID and don’t update the thumbnail once an item gets updated. This could have still been done using GnomeDesktopThumbnailFactory, but that’s water under the bridge, and should possibly be fixed. Although, images don’t tend to get updated so often, which is probably why nobody notices it.

Related to the above point, currently the modification time of the original doesn’t get stored in the thumbnail. It slipped through the cracks while I was reading the sources of the various modules involved in creating thumbnails in GNOME. However, a versioned cache makes it possible to fix it.

[1] If you are reading between the lines, then you might be thinking that it is serializing and deserializing GeglOperations, and you’d be right.

[2] GEGL might be a generic image processing library with its set of built-in operations, but for various reasons, an application can end up carrying its own custom operations.

[3] The idea of an application storing its edits separately from the original can strike as unusual, but this is how most modern image editors work.

[4] Both Darktable and Shotwell have similar thumbnailing infrastructure. You can read about them here and here respectively.

Written by Debarshi Ray

29 January, 2018 at 17:17

Posted in C, Fedora, Flatpak, GEGL, GNOME, GTK+, GVfs, Photos

New GNOME API key for Google services

Google Drive and GNOME — what is a volatile path?

with 6 comments

TL;DR: If you are using any GFile API that can create a new file or directory, then please take care of “volatile” paths. Look at standard::is-volatile, g_file_output_stream_query_info and g_file_query_info. Read further if you don’t trust me, or if you develop file management software.

Like other cloud storages, Drive is database-based. Every file or folder is identified by an opaque, server generated, immutable, blob (say “0B9J_DkziBDRFTj1TRWLPam9kbnc”), and has a human-readable title that can be displayed in the user interface (say “Summer Vacation”). This is unlike POSIX filesystems where a file is identified by its path and, barring encoding issues, the basename of that path is its name. ie., the name of the file “/home/alice/Summer\ Vacation” is “Summer Vacation”. Sounds like an innocuous distinction at first, but this is at the heart of the issue.

Let’s look at a few operations to get a feel for this.

Rename

POSIX:
mv /home/alice/foo /home/alice/bar
We need the paths to carry out the operation and once successfully finished, the file’s path changes. Since files are identified by their paths, the operation would fail if there existed another file called “bar” in the same location.

Google Drive:
set_title (id_of_foo, bar)
We only need the identifier (or ID) and the new title for the operation and once successfully finished, the title changes but the ID remains the same. Since IDs are unique and immutable, you can have two files with the same title in the same location. That’s a bit strange, you might think, but not so much.

Creating a New File

POSIX:
touch /home/alice/foo
If you want a file named “foo” in a certain location, barring encoding issues, you ask for a path that has “foo” appended to the path of the location. It will work as long as there isn’t another “foo” in the same location.

Google Drive:
create_file (id_of_parent_folder, foo) → id_of_new_file
Notice that we cannot specify the ID of the new file. We provide a title and the server generates the ID for it. Again, there is no restriction on having two files with the same title in the same location. This is where it starts getting weird.

GIO and GVfs

The file handling APIs in GIO are based on the POSIX model, with some room to accommodate minor diversions. They are ill-equipped to deal with a case like this.

As a quick aside, both MTP and PTP are somewhat similar in this regard. Both use IDs to refer to objects (think of them as files) stored on the phone or camera, and we have backends implementing them in GVfs. These backends construct POSIX-like paths using the human-readable titles of the files, and have an internal cache mapping IDs to paths and vice-versa. Easy enough, but there are two important differences:

MTP allows no parallelism. You can only perform one operation at a time — a new one won’t start until the earlier has finished. So, the GVfs backend can assume complete control over the storage. Google Drive, on the other hand, is massively parallel. You might be modifying it via GVfs, via a web browser, from a different computer, or someone might be sharing something to you from yet another computer. It is better to rely on the server generated IDs to identify files as much as possible, and let the server worry about keeping the data consistent.
MTP and PTP are usually backed by a traditional filesystem on the device. eg., FAT12, FAT16, FAT32, exFAT or ext3. Therefore the issue of duplicate titles doesn’t arise.

So, how do we deal with this?

Assuming that we are only doing read-only operations, the URIs used by GVfs’ Google Drive backend look like:
google-drive://foo@gmail.com/0B9J_DkziBDRFTj1TPam9kbnc/0B9J_GlyfBDRBHj1TRWL9khrc
The backend has its own scheme, the account is mentioned, and the path is made up of identifiers. Quite easy. The server doesn’t care about having a POSIX-like path, but we create one to fit into GIO’s POSIX-like world view.

These IDs are decidedly ugly and unreadable, so we use the standard::display-name attribute to tag them with the titles. If you run g_file_query_info on one of these URIs, you will get the ID as the standard::name and the title as the standard::display-name. Thus, the user won’t encounter any ugliness unless she deliberately went looking for it.

It gets complicated when we go beyond read-only access.

Let’s look at creating new files again. Since the GIO and GVfs APIs are designed around the POSIX model, a GVfs backend receives the request as:
create ("/id1/id2/some-title")

Even though the application thinks that it has created a new file named “some-title” inside “/id1/id2” that can be accessed by the path “/id1/id2/some-title”, the new file’s identifier is not “some-title”. It is whatever the server generated for us, say “id3”. If the application has to carry on the illusion of accessing the file as “/id1/id2/some-title”, then the backend has to map this “fake” path back to “/id1/id2/id3”; and this has to happen somewhat recursively because if we are copying a directory, then multiple elements in the file path can be “fake”. eg., “/id1/id2/title-of-subdir/title-of-file”, and so on.

We call these volatile paths. These are identified by the standard::is-volatile attribute and you can use standard::symlink-target to find the actual ID that it maps to. It is recommended that applications map volatile paths to the real IDs as soon as possible because the mapping can break if a parallel operation changes the title of the file.

In the specific case where you have created a new file and writing to it through a GFileOutputStream, then it is better to use g_file_output_stream_query_info instead of g_file_query_info. The output stream knows exactly which file it is writing to, and doesn’t have to do the mapping based on the volatile path, which could fail if the contents of the Drive changed out of band.

For working examples, see Nautilus.

Written by Debarshi Ray

13 September, 2015 at 01:21

Posted in GNOME, Google Drive, GVfs, Nautilus, Online Accounts

Google Drive and GNOME: 6 years later

with 25 comments

GNOME 3.18 is going to be another milestone in our journey to bring various online services closer to the desktop, but this is one that took us 6 years to reach. You can now access your Google Drive through your favourite GNOME applications and the usual GIO APIs

Thibault Saunier started working on this way back in 2009 as a Google Summer of Code project. I picked it up a year ago, and after a few hiccups along the way and some head scratching — more on that later — it is finally here for you to enjoy. You can either wait for distributions like Fedora Workstation 23 to ship GNOME 3.18, or try the 3.17.92 release candidate tarballs that are coming out next week.

Thanks to Alexander Larsson, Philip Withnall and Ondrej Holy for the endless discussions, code reviews and testing for the last one year.

Written by Debarshi Ray

10 September, 2015 at 11:22

Posted in GNOME, Google Drive, GVfs, Nautilus, Online Accounts

Debarshi's den

Archive for the ‘GVfs’ Category

GNOME Photos: an overview of thumbnailing

New GNOME API key for Google services

Google Drive and GNOME — what is a volatile path?

Rename

Creating a New File

GIO and GVfs

Google Drive and GNOME: 6 years later

GVfs and the GOA volume monitor