File Format Follies
I am doing what many have and am trying to work out the best workflow post-Capture NX2. One of the main frustrations has been to find a file format that is a efficient as CNX is in using NEFs. I don’t think there is anything, so it is a question of finding the compromises that fit into a workflow that I like.
CNX has the beauty of storing everything (edits, versions, JPG preview, ColorEfexPro edits) in the NEF, as well as reading the camera settings as Nikon intended. In addition to a single file to manage, the files were relatively efficient. A D7100 file might record at 24MB and expand to 28 after editing in CNX. A D800 file might go from 42 to 50MB. In addition, because ColorEfex Pro 3 for CNX is available, there is no need to create a new file, or significantly expand the size of the existing file, to use that plug-in. Plus, everything done in CNX2 recorded in the NEF is entirely “non-destructive” in the sense that you can go back late, open the file in CNX, and see every edit that you have done, and adjust any of them.
Going outside to use a different plug-in from Nik or OnOne is simply a matter of creating a TIF or JPG and editing it in that app. That obviously creates another file to manage, and “bakes-in” the changes. Once done, they can’t be reversed.
So in looking at the options, I wanted to find what was the closest to non-destructive and resulted in minimum files to manage. It was also important to me that the NIK and OnOne plug-ins be usable.
While ACDSee and Capture One might be good options, they integrate the plug-ins least (you need to use them exclusively in stand-alone mode). So I did not spend a lot of time investigating them. I also did not look much at PhotoNinja since it definitely doesn’t use plug-ins and you can’t print from it.
Some of what I report below may be erroneous, as I may not have found all of the tricks that are in the software that would allow a user to avoid the restrictions mentioned. If so, please correct me.
Lightroom will use the NEF’s directly, and store edits in sidecar file. Of course it doesn’t read almost all of the Camera settings. If you don’t like the NEF/sidecar combo, you can use LR to import and created DNG files, which do not use sidecars. If you are a stickler for preserving the NEF and don’t want to manage 2 files, you can imbed the original NEF in the DNG, about doubling file size. Overall, the DNG file size is not appreciably different that the NEF. Edits in LR are non destructive, along as you retain the image in your catalog.
The problem with LR, however, is that once you try to use a plug in, you are no longer editing in LR and your edits are no longer non-destructive. All of the Nik and OnOne plug-ins actually act as standalone applications, and require you to export a TIF, JPG or PSD from LR to be edited in the app. All of the edits that you make are then baked into the resulting file.
So, in using LR, you can maintain some non-destructiveness, as long as you don’t leave LR to edit with a plug-in, or do anything with the file outside of LR. Not exactly the level of convenience that you get with CNX2, especially if you like to use ColorEfex Pro, like I do.
Photoshop seemed like a possible alternative. It uses the same raw converter engine as LR (Adobe Camera Raw or ACR) which ignores most camera settings but has the ability to almost duplicate them. It does offer true non-destructiveness since plug-ins actually work within Photoshop, not as standalones on exported file, as with Lightroom. But there are some serious costs.
First, PS will read DNG files created by ACR or the Adobe DNG converters, but will not save changes to a DNG file nor export to a DNG. This seems really odd, as Adobe is the creator of DNG. Maybe I missed something in PS.
NEF edits do end up in a sidecar. But those are limited to ACR changes. Any real edits in PS require you to save the file as a PSD to assure “non-destructiveness”. That’s fine but PSD files quickly become HUGE, if you maintain the layers and SmartLayers necessary to retain the non-destructive aspect. Files increase in size by a factor of 10. For example, a 50MB D800e NEF edited in PS with 2 SmartLayers then saved as a PSD was 540MB! Frankly that’s is absurd. It means that on a 1 TB drive, I would get 2000 images, instead of 20,000. Or for every 5 weddings that I shoot, I would need to add another TB of storage.
I know “storage is cheap” but it is not unlimited or cost free. Managing and backing up that much space is expensive and time consuming. I doubt if you could do that in the Cloud.
And yes, you can “flatten: the layers in PS, but then you lose the non-destructiveness.
So why is non-destructiveness important to me? Three reasons:
- I may want to go back later, maybe much later, and look at an image that I like to find out what I did and use that technique on a new image.
- I may learn something now that would benefit an image that I edited yesterday, last month, last year or several years ago and want to adjust it.
- I may be working on an image, be interrupted, or want to let it sit for a day or two before finalizing it.
Without a “non-destructive image file, I can’t do any of that.
Another option is to use CNX-D to read all of the camera settings then make the base adjustments. It actually works pretty well, much like ACR. Then, to use plug-ins, simply export the file, much like LR, and do the adjustments then bake into the TIF or JPG. You don’t get all of the LR editing capabilities, but most of those are in the Nik and OnOne apps. You do get the Nikon camera adjustments.
That might not be a bad option, except that CNX-D is not quite ready. I tried it on a PC and found the speed wasn’t too bad, but I’ve converted to a Mac, and the speed is awful. So this choice will have to wait for the release version to see if it makes sense.
Of course I can continue to use CNX with integrated ColorEfexPro, as I have in the past. I works better on the Mac than on my PC. There are still some instances when I would need to export a TIF or JPG for editing in another app, but that is only about 5 % of the time. ( With the other options, I would be exporting to ColorEfex Pro about 40-50% of the time.
Also playing into this is that I want to continue to use PhotoMechanic as my image manager. It is far more capable and faster that LR, and does not rely on cataloging. NEF’s edited with ACR/LR/PS will not who the changes in PM, but DNGs will. By exporting JPGs or TIFFs from LR, I can manage the files in PM quite easily.
So I still have not decided on a final way to do things. I am unwilling to accept that the LR model is best and just put all my faith in LR. But I thought sharing my thoughts might help some others, as well as to give those who are more expert than I a chance to correct any errors in my findings about what can be done with different file formats using various photo editing applications.
Web Site: http://www.mickklassphoto.com
Online gallery: http://mickklass.gallery
My nikonians gallery
Puget Sound Chapter Coordinator
Visit Our Gallery
#1. "RE: File Format Folloes" | In response to Reply # 0robsb Nikonian since 23rd Aug 2006Wed 23-Apr-14 05:36 PM
I still plan to use CNX2 as my RAW converter. I am now studying Dan Margulis's Picture Post card process which is NOT non destructive, but it uses CS5 or CS6 along with a free panel that automates a number of processes that allows you to process a file in about 4 minutes. It is capable of doing things that ACR nor LR is capable of and because the process is so quick and has many options to the process, you can quickly make a different version just by starting over with the original. I am in the process of reading the book which I have almost finished and will then try some processing with the tools. I have used Dan's methods before and am a big fan. There is a thread in the Nikonians Adobe forum on this. You would wind up with two files. Your original NEF and a converted TIFF or JPEG. That TIFF would be processed. You would then save the flattened file when you were done as a TIFF, but because it is flattened, it is not huge. I think this gives you the best of both worlds. The process works with ACR as the source as well. All of the actions in the process use old PS commands and tools, so you can even do the work with versions back to CS1, but only CS5 and CS6 support the panel and the automation it contains. That panel is a free download and includes documentation as part of the panel, but no support is provided. I consider the book essential to understanding the process and the book includes access to videos that teach the process. The book is $75 on Amazon.
My Nikonians Gallery
"Nikonians membership - My most important photographic investment, after the
Retirement is a gift of time - Don't waste it!
Old age is a special gift that very few receive. Be thankful if you get it.
#2. "RE: File Format Follies" | In response to Reply # 0photphil Nikonian since 24th Aug 2004Sat 26-Apr-14 10:44 AM
Mick-I'm a long time NX2 user and looking at my options for another RAW converter. I liked how NX2 allowed my edits to be stored and now finding I have a lot to learn about how other programs work. I noticed that you did not mention Aperture in your post. I'd be curious to read your opinion about it and how it works. Thanks.
Visit my Nikonians gallery.
#3. "RE: File Format Follies" | In response to Reply # 2mklass Nikonian since 08th Dec 2006Sat 26-Apr-14 11:59 AM
I haven't tried Aperture, but may give it a shot, since it has compatibility with the Nik ans onOne plug-ins. If I do, I'll let you know how it goes.
Web Site: http://www.mickklassphoto.com
Online gallery: http://mickklass.gallery
My nikonians gallery
Puget Sound Chapter Coordinator
Visit Our Gallery
#4. "RE: File Format Follies - Thanks" | In response to Reply # 0
Great summary, and, AFAIK, accurate. The loss of CNX2 is a major PITA, for the reasons you set out.
For me, Photo Ninja gives superb results. But the lack of printing ability and local adjustments have kept me from buying it. I hope very much that these will be added, and I had some indication in e-mails from PictureCode that they were working on the possibilities.
But the fact that there hasn't been a new non-pre-release version since 2/10 is quite worrisome. I can only hope that they're working on something spectacular.
Of course, PN also requires the use of the Nik sw etc. as external programs that will 'bake' in the edits. Until and unless some program does what CNX did with the Nik Color Effects, and opens the editing API to allow 3rd-party sw to work 'inside' the editing process in a non-destructive way, we'll be stuck with that, regardless of what processing sw we use. (With the exception of PS, as you note, but then the need for ever-increasing terabytes of storage.)
No good alternatives.
#5. "RE: File Format Follies" | In response to Reply # 0
I guess owning a "mega" mp camera does have its side affects,
A D800E may be my next camera, so I guess I will join the brontobyte club,
IMHO, it is just a fact of life, all that IQ and detail comes at a price
I sort of started preparing for the onslaught of all those pixels,
I have already upgraded the computer with SSD and 32GB ram in an iMac 27" screen, I'm only using a 12mp camera at the moment but any new DSLR is going to be anywhere between 16 and 36mp in any case, may be even 54mp!
Along with the above, I was able to get Aperture as I was also looking for a non Nikon software alternative.
I tried LR but found Aperture more intuitive and I like the way it handles the files and versions of the files, when needed, I can edit with PS for those little jobs that PS does best, and I just love ACR, corrects distortion and vignetting on my 24-120 in 2 clicks
Few things that happen with Aperture,
All edits in Nik are tiffs
All edits in PS are PSD's
You do not "see" any in camera picture controls, found this a little daunting at first but now have presets that get the look I want
Not sure what happens with in camera sharpening so doing a few tests today
I have Adobe CC which enables me to use LR and PS, I have gone back and tried LR again but do not like it, may be a personal thing though, anyway, I access ACR in PS so I don't have to go there
Aperture has not had a big upgrade for a while so who knows whether Apple will improve this software or not whereas you can be guaranteed LR and PS will
FWIW, If I didn't have the power of PS at my disposal, I would have gone with LR as it has a lot more "under the hood" than Aperture
I used to have a photographic memory but never got it developed
#6. "RE: File Format Follies" | In response to Reply # 0
>Also playing into this is that I want to continue to use PhotoMechanic as my image manager.
This has turned out to be an important issue for me as well. I find that I can get comparable final results with quite a few NEF editors, including LR and ACR. Using Photo Mechanic as my back end application, meaning my tool for after edit or in-between-edit work, means there must be a TIFF or a JPEG of the converted/edited/adjusted version for PM to render.
I guess there are essentially two non-NX2 options. A DNG route and a companion image file (tiff/jpeg) route. Strangely(?) DNG is not the raw format lingua franca I once thought it was.
In short, the DNG route means I must use an Adobe editor. Other editors may or may not read DNG. Outside the Adobe family, few if any, will update the embedded JPEG previews that I need for PM. ACR does that automaticcaly for DNGs like NX2 does for NEFs. LR will update DNG previews on command.
The companion image file route means to save the converted/adjusted/edited image with non-destructive edits in whatever format the editor uses for that, plus to do a save as to tiff or jpeg. Jpeg saves space and makes sense as PM support.
Some NEF editors create suitable companion files by default. I have just been revisiting Lightzone where non destructive edits are saved as instructions in a editor specific jpeg file which PM can render. A final jpeg or tiff can be exported from the editor. This leaves me with multiple files of the same image, the original NEF, the Lightzone jpeg and the exported jpeg/tiff where edits are baked in. It creates more of a file mess than the DNG route, but it works with any NEF editor, also those in the Adobe family, it even works with Nikon NX-D.
Companion files may get out of sync with my edits if I forget to do the save as jpeg required by some editors. Likewise, DNG previews may get out of sync with LR edits if I forget to do the preview update. Right now, my vote is on the DNG route with ACR, but I will keep on exploring my options for some more time.
As far as non destructive edits go, I found them generally not transferable between editors.