Sunday, April 3, 2011

Who Lost My EXIF Data?

NOTE: This post has a lengthy introduction that might be properly called "another of Tom's rants about Adobe" before getting to the loss of EXIF data. I just couldn't control myself. Sorry. Feel free to skip ahead to the next occurrence of EXIF if you are just looking for an answer to your data loss problem.

I finally made an exit from Adobe Photoshop Album (APA). This dandy photo organizer has been my one true love since 2003. APA was released on February 18, 2003. This was convenient timing. I had retired 18 days earlier and we embarked on a lengthy driving trip to the sunny south on the 11th of February.

I carried along my first "real" digital camera, a Minolta Dimage X. This camera still rates in my mind as the best pocket digital camera ever produced. It got phenomenal reviews at the time of its introduction.

We had a great trip. We were on the road until March 27. In those six weeks we had warm weather and I took close to 700 pictures. It was our first time traveling with a computer and a digital camera and we chose our hotels this way:
Photo with EXIF data intact

I copied the photos over to my laptop PC each night and we viewed them and even sent a few along to friends and family in e-mails.

By the time we arrived home with 700 photos organized by day in a file structure, it was apparent that we needed some kind of photo organizer. I checked Popular Photography as well as other sources and settled on APA in spite of my love/hate relationship with Adobe.

Sure enough, Adobe managed to take a great product and screw it up beyond repair. They quit supporting the separate product, renamed it Organizer, bundled it into Photoshop Elements and then finally introduced the dreaded version 6! But just start typing into Google, "adobe photoshop organizer..." and watch the auto-completion box present some suggestions:

adobe photoshop organizer...
  • not responding
  • crashes
  • freezes
  • slow
Well, you get the idea. Pretty standard Adobe product stuff. Most of this started to happen around version 6 (I think I have that number right. It might be a off a bit) when Adobe abandoned Microsoft Access as their database manager and moved to SQLlite.

I did upgrade to version 3 which I think was the first version integrated with Photoshop Elements.

But each time I would consider upgrading to a more current version, I'd check the ratings and reviews and see a picture like this:

Look at the number of 1 and 2 star ratings. You just don't see that percentage of low ratings on very many products. Reading through the reviews for versions 6, 7, 8 and 9 over the years always prevented me from upgrading. I love this comment from the most helpful bad review on version 8:

If you can't quite make that out, it begins, "Adobe's customer support for PSE and is like the products - very slow and buggy." That parallels my experience exactly right down to their infamous "reinstall windows."

So I simply stuck with Version 3 over the years. It has worked fine and I ended up with over 48,000 photos in it and very good performance. A couple of times I have had to use the "secret weapon" of holding down the Ctrl key when starting the product to rebuild the database but otherwise we have gotten along just fine.

But, Adobe finally did themselves in. Version 3 won't run under Windows 7. It started out looking OK when I installed it and moved the pictures via a Save and Restore. But Adobe's usual non-standard coding practices soon led to a lockup that wouldn't resolve itself. OK, time to bite the bullet and either upgrade or move to a new organizer product. I considered an upgrade to Photoshop Elements 8 or 9 but only long enough to read the reviews on Amazon shown above and another in this review for Adobe Premier Elements 8:
On the hate side, it crashes...a lot. I've never seen an application crash as often as Premiere Elements does. It crashes when editing a video, rendering a video, editing text, even exiting the application. I haven't seen an application crash this often since the days of Windows 95. It's been like this for years - check out this post of mine from 2008. Premiere Elements crashes so much that in version 8, Adobe introduced an error reporting wizard that asks you what you were doing when the application crashed, asks you for your email address, and presumably sends them crash information. I say "presumably" because in all of the crash reports I've submitted - and I'd estimate I've submitted at least 50 of them in the past year - I've never heard back from Adobe tech support. Worse, Adobe as a rule doesn't release patches or updates for their Elements software - if it has a glaring bug, it will not get fixed. Adobe simply expects the user to buy the next version of the software that has the fix. Appalling, isn't it?
Let's try a different vendor!

I loaded up the Microsoft Windows Live Photo Gallery. I had considered Picasa but having played with it earlier, I agreed with the assessment in this review. See this Wikipedia article for some comparisons between image organizers.

My, oh my, is Windows Live Photo Gallery a dandy product. It is fast! It supports tags beautifully. It has wonderful editing tools built into it for straightening and handling the "Levels" function that is so critical to fixing photos. Don't take my word for it. Try it on a few of your photos and you will be a happy camper.

I purged my photos from the new machine and started over, introducing them from their locations on the old machine. (I had gone back and made sure that I wrote the tags into the files for all of the pictures.) I quickly had 48,000 photos in the gallery and was searching and straightening. I soon introduced my wife to the product. She has been downstairs working on old photos for most of the last 24 hours since. I really expected her to be a hard sell because the interface is quite different. She loves it.

Where's my EXIF data?

Meanwhile, I loaded the 750 pictures from this year's driving trip through Texas and Louisiana onto the new machine. I did this from the kitchen laptop while she was working on pictures downstairs. Try doing that in Adobe Organizer. I casually mentioned the fact to her that those pictures would also be found in the new Photo Gallery. A bit later, she said to me, "there are a whole bunch of photos from years ago that all filed under the current date." "No way," I thought.

But alas, there were over a hundred old photos that ended up in the current day. These photos really belonged back in 2003! What happened? Did the new Photo Gallery mess them up? Did the old Adobe Organizer mess them up? What's the pattern?

A little detective work in the Adobe Organizer revealed that all the "Metadata" was blank in these photos. What is Metadata? Basically it is the name that the Adobe Organizer version 3 gives to the EXIF data in a JPEG file. Besides the picture itself, a JPEG file can contain lots of interesting data about the photo. Things like what kind of camera took the picture, what exposure was used and when the picture was taken! By the way, there is another bunch of data in the JPEG called IPTC data that is more textual, like captions and tags. That data appeared to be present! (More on EXIF, IPTC and their new replacement, XMP data can be found here, here and here.)

Here is a sample photo with the problem of no EXIF data.
Rotated photo with lost EXIF data

Notice anything unusual about it? Well, besides the goofy looking guy and the smiling peanut. (One of them can be found near Plains, Georgia.)

It has been rotated 90 degrees. All of the photos that were missing the EXIF data had been rotated within the Windows XP image viewer found in the Windows Explorer back in 2003. It turns out that both Nikon and Canon had warned their users to avoid doing this. Of course, it didn't happen to everyone. But be aware that it did happen!

Once I acquired the Adobe Organizer, I did my rotations from within that product. That doesn't actually rotate the image. Organizer just makes an entry in its database that tells it to display it as rotated. There is no change to the JPEG.

This also explains why those pictures were slightly out of order. When the date/time taken was destroyed, Organizer went to the next best available date/time, when the file was last modified. When I had gone through all my photos and performed a "Write Tag Info to Files..." I inadvertently destroyed this date/time bringing it up to the current date.

So in my Organizer, those photos were a bit out of place, slipping from when they were taken, such as "in the morning," to when they were rotated, such as "in the evening." When I re-wrote the tags, they moved to "today."

Perhaps this will solve someone else's search for some EXIF data.

By the way, there is an interesting philosophical argument about whether software should write things like tags and display orientation into JPEG files. A good description is found here. I could argue either side.

Saturday, April 2, 2011

Windows 7 Libraries. What's Happened to Our Schools?

I've finally decided to upgrade from Windows XP into the modern world. I have a new Dell Inspiron on my desk and am gradually moving files over onto it.

One of the first new things I discovered on the Windows 7 machine was "Libraries" for such things as photos and music. They appear to be a very nifty concept, bringing together similar files from multiple file directories in a "virtual directory." You get to specify the real folders involved and Windows seems to keep them updated at a pretty rapid clip.

But, I'm one of those people who likes to read the manual and know what's really going on. Maybe I should buy a good Windows 7 book. Meanwhile, I checked the "Help" function in Windows. I get this definition explaining "What is a library?"
Libraries are where you go to...
In my grade school, I'd probably have gotten my knuckles rapped with a ruler by the time I got that much of a definition out of my mouth. "You don't define a noun by saying where or when, Mr. Sawyer," would have been the immediate response at St. Charles grade school in Portland!

The Windows 7 Help explanation goes on by explaining what libraries are not and how they are different from other things:
Libraries don't actually store your items. They monitor folders that contain your items, and let you access and arrange the items in different ways. For instance, if you have music files in folders on your hard disk and on an external drive, you can access all of your music files at once using the Music library.
Defining something by what it is not would have gotten a knuckle-rapping for sure!

But alas, no explanation of what they are. So if you arrived here trying to figure out what they are, let's take a crack at it!

Libraries are a conceptual view of storage on your computer. They are represented by the File Manager as a "virtual folder" which displays selected content from one or more real folders.

There, we defined a noun with a noun! End of rant.

Of course we still don't really know what libraries are. What is actually behind that virtual folder? Perhaps an index file of some sort that is maintained by a task or process that is of undefined name? It's a shame that this kind of basic knowledge that is key to figuring out "How did that happen" isn't shared out in the light of day.