Tag images with people's names (from contacts)

Posted on 2010-03-09 20:50 UTC by Oskar Welzl. Status: Under consideration, Categories: Media.

You may want to tag images not only with geo-information and keywords. You may also want to add information about people who are in the picture. This way, you'll later be able to perform a search like "Show all photos of Harry" or "Show all photos of Harry and Sally".

The most natural way to do this is to take names from the built-in contacts database, so there'll be a connection between your contacts and the images showing your contacts.


t.m.o. thread:

Solutions for this brainstorm


Solution #1: Use Microsoft Live Photo Gallery People Tags

Posted on 2010-03-09 20:51 UTC by Oskar Welzl.

Maemo 5 already stores image metadata in RDF-based XMP format. Microsoft uses the very same format for what they call "people tags". People tags store the names of the people in the image and also allow you to draw a rectangle around the person; this way, in an image that shows a group of people, you can tell who's who.

Just to be on the safe side, the names of the people (not the rectangle) could be duplicated in the personinimage field from the IPTC Extension.

Both tagging and filtering by tags would have to be done in a third party application, but access the system contacts database. Optionally one could evaluate if and how tracker can be convinced to read the additional information.


Using these tags introduced by Microsoft makes the information useful also for people who are on MS Windows. It will not be a Maemo-only thing. Names written on a Windows box will be read on the Maemo device and vice versa. Still, because it's built upon XMP, it's an open standard.


Names are strings only. Doing the tagging from the contacts database may help keeping the strings consistent, but if your niece sends you a picture tagged "Aunt Martha" when you tagged Martha as "Honey", you're out of luck. Strings are bad.


Solution #2: Use inverse functional properties from contacts database

Posted on 2010-03-09 20:52 UTC by Oskar Welzl.

This solution proposes the same as "#1 Use Microsoft Live Photo Gallery People Tags" as it will also write these Microsoft-tags a) for compatibility and b) to store a human readable name. Also, as solution #1, this solution proposes to have a new application for people tagging instead of building it into the photo viewer.

In addition to that, a set of properties from the contacts database will be identified that most probably will be unique to one person: an instant messaging account, for example, is unlikely to be shared; a mobile phone number; a facebook profile; less reliable, but usually also acceptable are mail addresses and homepages/blog URLs. The application will use these as "inverse functional properties".

A new XMP-extension will be defined to hold information from these properties. The XMP-extension can either be in the MeeGo or Nokia namespace be in any namespace. When tagging a person, an MD5 hash will be created for each of these unique properties and stored in the respective (new) XMP-fields. (MD5 hashes are used for privacy reasons so that a tagged photo can safely be shared.)

When searching for a person from the address book, the application will again create MD5 hashes from there for all the properties agreed upon as unique. It will then search for the corresponding MD5-sums embedded in the image files. This way, you can make sure that people are found even if the names are spelled/abbreviated differently (which is likely to happen if images were shared/tagged by somebody else).

MD5-hashing inverse functional properties is a technique already in use by the FOAF project. While for some obscure technical reasons we cannot directly use FOAF within XMP metadata (although both are built upon RDF), FOAF had to deal with the problem of uniquely identifying people from data available, without introducing new IDs. They came up with using things like a personal email address, a instant messaging account etc., and it worked. It will work for Maemo, too.


Using both human readable names and machine readable unique identifiers makes it much easier for any software to identify people that you did not tag yourself on your device. Again, building on XMP makes it easy for developers outside the Maemo community to pick up the idea and use the same mechanism on other platforms. MD5-hashes prevent privacy problems but still allow you to find people in images tagged by somebody else - as long as you both have the same information about the person in the image.


A lot of effort needs to be put into the design of the search algorithm. (What to do if an image isn't "MD5-tagged" at all, but the names from the Microsoft-tags match exactly? Which properties are considered probably unique enough? ...)
Also, a new XMP-vocabulary needs to be carefully designed. Do we have people who are Maemo developers, know RDF and have experience in XMP?

Latest activities to brainstorm Tag images with people's names (from contacts)