RAW workflow on Linux?

One of my resolutions for 2015 is to try harder at this "RAW" thing.  So far I've been shooting (film digitized to) JPEG most of the time.  Primarily for lack of (time to test) good workflow solutions under Linux.

For JPEG, I've been quite happy with Shotwell for my general DAM workflow and some basic global edits (crop, rotate, saturation, contrast, exposure) and other editors (GIMP) as needed for dust removal or stronger retouching requirements.  I'm overwriting the original JPEG for scanned film (I could still re-scan it to get back to a pristine scan if I wanted to) or I can choose to create a new version and Shotwell can manage it as a derivative of the original without too many difficulties.  So I use Shotwell to catalog, title, describe, tag, rate (stars), organize (events) and publish/export.  Import is slow and unreliable to I switched the task of emptying my cards back to gThumb (nicely done) and it writes directly into my photo directories (by year/month/day) where Shotwell will safely auto-import them.  Strangely enough Shotwell doesn't crash on auto-import, only on copy/import.  Go figure... Also export used to work fine (but never was able to transfer the descriptions to Flickr, only the title appeared in both title and description) but as of Shotwell 20.2 there was a bug (on Ubuntu) when publishing.  So I switched back for a while to manual export/upload (using Flickr web interface) then to export/mail upload (using a homebrewed script to set tags, titles and descriptions as they should be set).  So far so good for JPEG only work...

For RAW, in 2014 I've done several attempts at using Shotwell to do the same.  However, I quickly found out that Shotwell works fine with RAW+JPEG and even RAW only (I used developer=camera) but that doesn't give me any advantage over shooting JPEG since I end up with the same JPEG image as provided by the processor in the camera, plus a bloated and so far useless RAW file on my disk.  If I really want to develop the RAW file I could use developer=shotwell (but then I don't get any manual control over the conversion process AFAIK) or create my own JPEG by opening each RAW (from within Shotwell or not) and maybe storing the result in the Shotwell-managed folders.  This will quickly make it pretty hard to manage (each keeper will appear at least twice in Shotwell and 3 files will be kept: RAW, Shotwell-JPEG, my JPEGs)...  Not so much a problem for disk space (cheap enough by now) but more a problem when it comes to managing the whole thing (what do I still need to process/rate/tag/tweak/publish?).  Another issue is that Shotwell (or its libraries) refuse to write any metadata back to the JPEG file as soon as a RAW or RAW+JPEG pair is part of the equation.  I can understand that some people don't want any software to ever touch the original RAW but I can't understand why this would also apply to the JPEG (especially when the format does provide support for metadata updates-e.g. DNG...).  That raises the issue about potential lock-in, or data loss in case of database corruption, or lack of interoperability with other software.

So I've decided that the current shotwell feature set is not satisfactory for RAW processing, at least as a standalone solution.  I might still make another attempt at using a RAW developer from within Shotwell and see if somehow I can overwrite the Shotwell-generated JPEG so as to keep the pairing between the RAW and derived JPEG.  Maybe that could help especially if the RAW developer can write metadata in such a way that Shotwell could read them and update its database accordingly.  That way I would not need the RAW software to also do my DAM, I could still continue to use Shotwell for this, and would not even need to develop the RAW as long as the automatically produced JPEG was good enough for my purposes.  Well I would still need something to write metadata to that JPEG then I guess...

So I made a number of tests with other software packages, that will be a subject for the next posts.

Kid host - disk crash and recovery/cleanup

Yup.  The old boot disk (with Windows Vista & Linux but no /home) crashed.  I did took this as an opportunity to rethink/rebuild my desktop PCs, re-test all old IDE drives lying around, and also redo my backup strategy and include the many laptops that now are lurking around the house without any good backup being made.

So here's what we have now:

Kid2
My main workstation.  Moved the 1TB disk from office that only contained backup partitions, as well as the 160Gb boot disk from office, to have currently 3 SATA disks in it:
sda Two NTFS partitions (1 hidden), swap and Linux boot/root partition
sdb old "home" from Office, with the intention to merge documents into a new /home on kid2 but currently this is full so needs enlarging... after backups are again working
sdc 1TB currently only using the sdc3 as a new /home

Office2
Rebuilt, reinstalled Ubuntu 14.10 afresh.  Empty /home to begin with.  No other users than myself at this point and no documents in there.  To be used as backup server only for the time being.  I might mount /home from kid2 using nfs at some point to avoid spreading personal data on two separate PCs.

Upgrade then repair

Upgraded to Ubuntu 14.10 last month (both Office & Kid2).  Then the boot disk on kid2 died.  Yesterday I put the 3 remaining drives from both PCs in the kid2 chassis.  The boot drive from Office is still good so this is now the new kid2 host.  I'll rebuild something soon to have another backup server somewhere in the house...  For now all is good and no data loss.  And more space available thanks to the 3rd drive in the remaining PC (2TB).

Aftershot Pro 2 review conclusions

When it comes to digital photography, I have to confess that I am primarily a JPEG shooter.  I don't like post-processing so I don't like the idea of using RAW format, as I have the impression it is going to slow down my post-processing tasks and add bulk to my digital archives.

Well let's be honest, I wanted to try it out despite the cons and see if there is anything in for me.  My photo workflow has been based in the past on (mainly) JPEG tools : Gthumb, then F-Spot and since a number of years I have settled down to using Shotwell, a great and fast catalog/edit/publish all-in-one tool.  Pretty limited for retouching but I don't like to do that much anyway so never a problem for me at least.  I can use the Gimp from within Shotwell when I need to remove some dust spots but for the rest it is fine to adjust exposure, contrast, saturation (and I don't do much of anything else 95% of the time).

That being said, last month I tried Corel Aftershot Pro 2 (Linux install; amd64 package downloaded from Corel's site) and started the 30 days trial.  Soon after the install I got an update from Corel.  Don't know exactly which version I have tested but it must be whatever is current as of July 2014.

I quite liked using the program, here are my main conclusions:

Pros:

• it is MUCH more powerful than Shotwell for retouching photos (including while doing the RAW conversion of course).
• The GUI is pretty sleek after you get used to it (took a couple of days maybe). 
• The program is pretty responsive even when working on RAW files.  There is clearly a lot of caching and performance improvements behind the interface (and I have old hardware)
• Can use hardware acceleration when available (I installed OpenGL to try that, not sure it helped a lot on my old hardware though)
• Good coverage of the entire workflow (but not entirely all-in-one)
• I could work with IPTC metadata and tagging in a way that was compatible with Shotwell (hierarchical tags are supported with semicolon separators).
• It was pretty easy and fast to convert RAW images to JPEG and the different adjustment modules are well organised. 
• 
  

Cons:

• workflow completeness is not 100% : I didn't find how to import photos from the camera or card.  Not a problem during the trial because I worked mostly on photos that were already in my Shotwell library anyway.
  Again, workflow completeness is not 100% : when finished you can export photos to different places and formats but not directly publish (to social media)
• Camera support is not bad but far from complete.  DNG is OK (tried with both my K-m and K-r raws).  Fuji EXR doesn't work (tried with some .RAF files downloaded from review sites as samples of RAW images from compact cameras that I was investigating to possibly add to my arsenal, like the F600EXR).  Even worse, the Olympus ORF format from my old Pen Mini 1 is supported but not more recent cameras (e.g. the OMD E-M10 that is still on my list and almost in my shopping cart, and the XZ-10 that I eventually bought last month).
• Some irritating behaviour from the GUI : when flagging an image for reject and having a filter to not show the rejects, the image is removed from the view but it is still selected.  Pretty annoying.  I would like it so show whatever the next image is but couldn't configure it to do so.
• Not free, not open source (but at least Linux is a supported platform).

Conclusion
Even though I quite liked using it, I decided against buying a license.  At least until my (current and future) Olympus RAW formats are supported.  During my testing, I also started using Rawtherapee and found that it had better support for more RAW formats so that might be my future way of working with RAW under Linux.

Current Tux state

Both PCs at home are now happily running Ubuntu 14.04 since it went out.
- Xubuntu used mainly on both
- Just installed Lubuntu on one of them to check that variant

Currently I'm evaluating more software related to the photography workflow, so that's probably a theme for the next posts.