2 September 2012

How do you manage your digital pictures?

Part 1: naming and tagging

The photos I take are the most precious things I have. Almost everything else can be bought again. Photos, once lost, cannot. Even more, photos are an integral part of my memory, of what I have experienced. With more than 70,000 pictures in my personal database, it has always been necessary for me to ensure their proper renaming and tagging (I'm talking about metadata, IPTC and XMP in particular), but also their offline and online backup. In this post, I'll share with you my semi-automated technique for renaming and tagging, which then allow me to be able to quickly find pictures and have fun "slicing and dicing" (wink at database engineers and business analysts). In upcoming posts, I'll share my technique for rating my pictures, from zero to five stars, and then how I like editing them.

In recent years, both Google Picasa and Adobe Lightroom have significantly improved their semi-automated tagging features (e.g. geolocated positioning, face recognition), probably making some of my steps obsolete or maybe a tad more complicated. Pick what works for you. Share your workflow and suggestions as well.

Step 1: Directory structure

I have a very simple one-level hierarchy: by year. If I capture a significant number of pictures on a particular trip (e.g. more than 1,000), it gets its own folder. So here is what my folders would look like:

2007 (would contain all pictures taken in 2007, at home, at parties, etc.)

2007-10 Japan (all photos from the trip to Japan)

2008

2009

2010

2010-02 India

Step 2: Renaming photos

This is how I systematically rename all pictures:

YYYY-MM-DD Location (country code) [id] [Main subject] - [Details].jpg

where :

- YYYY is the year on 4 digits, e.g. 2007

- MM is the month on 2 digits, e.g. 10

- DD is the day on 2 digits, e.g. 22

- Location is the place where the picture was taken, e.g. Kyoto

- (country code) is the international country code of the location, e.g. (JP)

- [id] is a unique identifier on 2 or 3 digits, ensuring that 2 almost identical pictures taken on the same day have distinct filenames, e.g. 003

- [Main subject] is the main monument / topic / subject, e.g. Jidai Matsuri (annual Kyoto festival)

- [Details] provides more details on the picture like the names of the people on the picture, e.g. Archer

For example : 2007-10-22 Kyoto (JP) 003 Jidai Matsuri - Archer.jpg

Software allows to do this semi-automatically. In particular, the date can automatically be retrieved from EXIF data (the data that's added by the camera), while all photos taken at the same location and/or with the same subject can be automatically named and indexed with an incremental ID. Only the details part of the filename will then remain to be manually completed. On Windows, you can use SIREN (free); on Mac, A Better Finder Rename.

Step 3: Capturing GPS data

If your camera doesn't already encode the GPS data, you have several options:

- track your whereabouts with your smartphone (on Android, Google Tracks or just using the historical data of Google Latitude) and export the GPX file; the GPX file can then be read by the latest version of Lightroom. I'm however still using my previous script and Phil Harvey's excellent exiftool (www.sno.phy.queensu.ca/~phil/exiftool/) to do this automatically.

- manually pin-point photos on a map in the latest versions of Picasa or Lightroom

- keep a database of the main locations where you took pictures and their corresponding latitude and longitude coordinates (easy to get by reverse geocoding using the Google Maps API); that's useful in any case if some of GPX data is missing. I wrote a simple script many years ago (in PHP/MySQL) that then detects whether a picture does not have GPS data, in which case, it looks for the location in the filename, matches the location found in the database and adds it to the photo's metadata (country code, location, etc.).

Step 4: Face recognition

This is something I haven't experimented with yet. I still manually add the names of people in photo filenames ("Details" in step 2 above). However, I could probably use Picasa to detect all the pictures with the same person on them and then automatically add that person's name to the photo's filename. It would save me a lot of time and prevent naming errors.

Step 5: Keyword structure

I have a 9-level keyword structure that I keep improving all the time (e.g. Architecture > Religious buildings > Islam > Mosques). This keyword structure can be encoded in a photo's XMP metadata. It lives in Lightroom, and is updated to a database from which I can retrieve the appropriate hierarchy for any matching keyword (see steps below). If there's no matching keyword, the keyword gets added to a top-level group called [NEW KEYWORDS] which then forces me to move those new keywords to their relevant categories (e.g. if it's the first time I photograph an elephant, I'll move the "elephant" keyword from [NEW KEYWORDS] to Fauna & Flora > Fauna > Animals > Mammals).

Step 6: Adding metadata

My script then automatically adds all relevant IPTC and XMP tags based on the filename (extracting the relevant elements such as location, main subject and details, but not the date, the ID, nor the country code, see step 2 above), and add all other useful metadata (photographer name, copyright, etc.). Again, this is done using some simple coding hacks and Phil Harvey's exiftool.

Step 7: Backup

Once this is done, I copy all the photos on two external hard drives (I haven't set up anything, it's still manual) and on an online server. I spot check that photos have been correctly copied over, before eventually formatting the camera's SD/CF cards.

Next steps for me on this metadata scripting:

- probably work on the face recognition automation script

- think of translating everything (filename and/or only metadata) automatically into English... since all the data is encoded in French and limits its discovery by non-French speakers

Next parts in this series:

- my system in rating pictures

- editing photos