anabasis Edwardsville, US Nikonian since 26th Sep 2003
Tue 04-Mar-14 08:30 PM
"Is digital archival"
Would you consider digital files archival? The trouble with a digital image is that it needs a machine to convert the file to an image that we can see. To do this we need a machine to recognize both the medium it is stored on, and the images needs to be in a format that can be read.
I look at slides my grandfather shot in the 1970's and think of the various incarnations of digital file formats and storage media since then (say about 40 years).
This whole Nikon Capture NX-D software change has me thinking of this. With the loss of Capture NX2, all of the corrections I made to my files are essentially lost. It's no large leap in my mind to think that the ability to read NEF, TIFF, or even JPGS could be either lost or corrupted in the future.
Media changes is less of a problem since you can usually transfer the files from one media to another during the transition phase. I have files that were once stored on a 5.25" floppy disk that have been transitioned over several generations of media updates. Still it takes time and effort to convert files, and if some files are missed or neglected, it takes great effort to change them over from long unused formats and media. What about in 100 years? Will that zip disk with artist X's long-lost NEF's shot with a D1 be unreadable?
Will all of my digital photographs be lost when I pass away and there is no one caring enough to put in the work to update them?
#1. "RE: Is digital archival" In response to Reply # 0
Digital files are certainly archival in the sense that the image will not fade or degrade. That's one huge benefit of digital over the days of slides, film, and prints.
Your other point is valid: care must be taken “every so often” to move the archive from dying formats to whatever is new. Get those Zip disks transferred before it's too late!. But I'm not worried about technology and file formats moving past my files; I and they go along for the ride.
After we're gone it may be another story. Better print and frame the best of your work for your grandkids to appreciate. Oh, wait — they will fade over time…
Jon Kandel A New York City Nikonian and Team Member Please visit my website and critique the images!
#3. "RE: Is digital archival" In response to Reply # 0
>This whole Nikon Capture NX-D software change has me thinking >of this. With the loss of Capture NX2, all of the corrections >I made to my files are essentially lost.
Not yet! CNX2 still works (and will for some unknown amount of time). So you can convert your edited file to some other format, digital or event print. TIF and JPG are probably as "archival" as you can get right now. DNG might be a candidate. If they are digital, you can always convert them later to some new format that comes out. You can even revert to the original, unedited NEF, if you wish. Keep multiple backups, and you'll be as safe as can be.
All bets are off when the Four Horsemen of the Apocalypse show up.
#4. "RE: Is digital archival" In response to Reply # 0
“Will all of my digital photographs be lost when I pass away and there is no one caring enough to put in the work to update them?”
This is now and always has been a problem with any media. Nothing is archival if there is no one taking the steps necessary for preservation. Countless oil paintings, drawings, photographs and negatives have been lost forever due to simple neglect.
Dave Summers Lowden, Iowa Nikonians Photo Contest Director
Nikonians membership - "My most important photographic investment, after the camera"
#5. "RE: Is digital archival" In response to Reply # 4
In the same vein, I recall that NASA has trouble reading the data (that hadn't been converted) stored on magnetic tapes during the moon landings because the large tape drives used in the early computers are no longer operable.
#6. "RE: Is digital archival" In response to Reply # 4
I'm with Dave.
I would ask, how do we define archival? Do we mean indefinite storage without maintenance, indefinite storage with maintenance, storage for 5, 10, 25, 50, 100, 1000 years?
I would posit that nothing is really archival without maintenance. I would also posit that indefinite storage is generally a myth. I think both are definitely true of negatives, prints, and image files.
With special regard to image files, any storage medium could be discarded by future technology, any storage receptacle can fail, and any storage medium could be unusable due to no devices available for it at any point in the future. Moreover, no file format is forever.
Digital photographs in any form require maintenance to preserve for the future, and that may mean massive image transfers if you have a huge library.
#8. "RE: Is digital archival" In response to Reply # 0 Wed 05-Mar-14 01:26 AM by rectangularimage
I just watched an episode of The Story of Film last night in which Mark Cousins, who the the creator of the series and quite a student of movies and the industry, when asked if film (as a technology) will die out, said that soon all movies will be made digitally, but film will still be used to archive them.
If this is true (and I have no reason to doubt him) then the movie industry doesn't trust the new fangled digital stuff as an archival medium. Or at least they're hedging their bets.
I highly recommend The Story of Film, BTW. I DVR'd the series, 15 episodes, when it was on Turner Classic Movies a few months ago.
#9. "RE: Is digital archival" In response to Reply # 0 Wed 05-Mar-14 01:36 AM by Omaha
This is a subject I've devoted a fair amount of thought to since I was asked to assemble a slide show for a funeral last year.
I was handed a shoebox of old photos...some over 80 years old, some newer...scanned them in, put the show together. Some of the photos were pretty rough. Some were lousy shots to begin with. But all of them were very meaningful to the family, and that's what mattered. A little fading over the years was not important.
Anyway, for what its worth, my view is this: While it is theoretically possible to archive digital photos (or digital anything for that matter) in perpetuity, for the average person (ie, someone who is not interested enough in photography to read a forum like this) the likelihood of non-printed photos surviving into the distant future is very slim.
A good friend is an avid digital shooter and an avid (and professional) computer expert. I talked to him about this, and he went at great lengths to explain to me his procedure for continuously refreshing his archive...something about multiple RAID arrays and whatever. He went through in detail his monthly and annual procedures, and talked about how some of it had been automated, and how he even budgeted $1000 per year for hardware upgrades.
The whole time, he thought he was explaining to me how robust digital archiving can be, but in reality, the message I received was precisely the opposite. If it takes that level of care and feeding (or even if it takes only 10% of that level, given that this particular guy tends to over-do everything), that is still too much.
Consider what happens the day my friend dies. Who is going to take over all that maintenance? Probably no one. Its probably the case that his setup will sit in his den for a week or a month or a few years until for whatever reason someone decides they need the space for something else. And then best case scenario it all gets tossed into a box and stored somewhere. Worst case it all gets pitched.
So assume it gets put in a storage unit, and 50 years later his great grandson hears rumors about all these cool old photos and decides to get them back. Maybe he can track down the storage unit, maybe not. Maybe all the cords and cables are there, maybe not. Maybe it all powers up, maybe not. Maybe the storage media is intact, maybe not.
You get the idea.
Is the answer "the cloud". Ok. Good luck with that. Say my hobbyist friend decides to upload all his stuff to Google Drive. Is Google going to be around in 75 years? What about Facebook? If they are, how is the great grandson going to get access to the accounts?
That it more than theoretical to me. I lost a bunch of my early digital photos that I had uploaded to Ofoto years ago when they shut down. My daughter had a bunch of photos on MySpace, and can't recover the password due to no longer having access to the email account, so they are probably lost as well.
My conclusion in all that is that if you are an institution with sufficient institutional persistence (ie, a major museum, for example) then there is no doubt that with proper care and budget a digital archive can be maintained perpetually. But for individuals, I think its risky.
The film guys (of which I am one) like to point this out as an advantage to analog photography, but I think that's wrong. The question isn't "analog vs digital", it is "prints vs no-prints". That's everything.
I endeavor to print as many of my photos as I can. For family snapshots, I send them off to Adoramapix and get 5x7s made. I probably average 20 or 30 prints a month. For my "good" work, I print on my HP Designjet Z2100. HP says the ink is archival for 250+ years if displayed under regular glass, or 300+ years if displayed under UV resistant glass or in enclosed boxes or albums. For B&W, I make analog prints in my darkroom.
So, then the question is what expectation do I have that any of those will survive. And the answer is I really don't know, other than 100+ years of family experience that suggests that photos tend to be valued heirlooms that are passed down and rarely thrown away. Keep them reasonably well organized (ie, sturdy boxes or well organized albums) so that it is convenient for people to store and preserve them, and they are likely to do so.
For my big prints off the HP, they get professionally framed under UV glass. I also write a little note that I have the framer include behind the print that describes who is in the shot, the story behind it, when it was taken, etc. Maybe someday someone will find one of those to be interesting.
Its all about the human factor, and all about making it easy for people in the future to value your photos. Even slides, being as they are less convenient to view, are (in my experience, at least) more likely to be discarded. Prints are everything. Prints are something that anyone can hold and look at and appreciate.
Anyway, that's my view on things. I think paper is still the best archival media for the real world.
Edit: One thing I would add on digital is that keeping RAW files is fine, but if you are interested in archival properties, make sure everything is saved in a common format like jpg. My assumption is that it may be quite difficult to round up the required software to read a D7100 NEF file in 2089. It is more likely the folks in 2089 will be able to view a jpg (although even that is not certain).
#12. "RE: Is digital archival" In response to Reply # 0 Wed 05-Mar-14 02:18 AM by agitater
>Will all of my digital photographs be lost when I pass away >and there is no one caring enough to put in the work to update >them?
The first question to ask is, why should anyone care?
Don't misunderstand - I'm not trying to be needlessly harsh. The question is based primarily on what we already do as a westernized society to preserve historically significant records, creatively significant records, culturally significant records and politically significant records.
The museums are full of artwork of all sizes, types and descriptions made of every kind of media you can think of. As the generations pass, the most culturally significant pieces (globally numbering in the tens of millions now) are preserved, periodically restored, presented in exhibitions, photographed and reproduced in books and on web sites and so on. To preserve anything of value - with value defined as social, political or cultural significance - stewards of collections must raise money in order to house and preserve the works and care for them over time. Digital photography is no different.
What we have trouble with, as individual photographers, is the idea that nobody will consider much of what we photograph to be of significant enough value to take the time and effort to preserve. A significant and directly related problem is that there are already far too many digital photos present in the world right now. The notion that even the smallest portion of the currently astounding volume of photos should be preserved for the ages is impossibly unrealistic. The associated fact that a staggeringly large portion of all those photos will in fact be preserved through repeated backup and storage in a succession of social networks which are all, in actuality, graphical user interfaces hooked into massively networked databases is only slightly less surprising.
The fact remains though that families often do preserve photos and portrait paintings for many, many, many generations. We have artworks of all kinds on display in great galleries and great museums around the world. If we look at the great portraiture created by myriad painters starting in the 13th century alone, we can trace some of the familial evolutions of those families through to later centuries. Great wealth, powerful churches and powerful political leaders provided the means and the money to preserve and periodically restore such works. Many have been lost - great wealth has been laid low through greed, war, political strife, dramatic social change, and through simple anger and disagreement. Many have been preserved. Many that have been lost have been written about and preserved as drawings or etchings or lithographs in books. Over the past century or so, many have been preserved in books. Now, many that might have otherwise been lost are being preserved online. So we come full circle.
The notion that all great paintings have been preserved - or rather, all great paintings by the great documentarists and social painters who've worked to create countless great works - should be discarded because it's simply not true. Many exist that will never been seen by the greater public because the paintings reside in private collections. Many, as mentioned above, have been lost through time, tides and tragedy.
Digital photos are easier to preserve than anything that was ever painted on canvas. But once again, the question to ask is, why should any particular photo or collection of photos merit preservation? It is merely because of the fact that we made some photos? That is hardly a reason!
My grandparents came to Canada, through New York and then Montreal, arriving by train eventually in Winnipeg in November. It had been a journey of many months from their temporary home in Odessa, Ukraine (a timely tale, n'est ce pas?). A Russian jew and a Ukrainian jew, married, making a life in a new land in 1909. So many photos, so many portraits, so many black & white prints, so many 16mm home movies in the '50s and '60s when their children (i.e., my parents, aunts and uncles) began having children of their own and snapping photos and making short little films. My sister and I collected a huge pile of the stuff, cleaned it up, made use of dubbing services and had it all burned dubbed to VHS. After the next media battle, we collected it all again and had in dubbed to CD. Last year, I gathered it all up again and had it dubbed to DVD. In a couple of years, I'm going to have it all fully digitized and stored online in so-called archival storage. Of course, archival storage online is only good as long as someone, somewhere is making the annual account payment. And who is to say whether or not my kids will keep it going or whether or not my grandchildren (when they show up!) will keep it going?
In any photo collection - no matter how important we think a record of our family or travels and travails might be - there has to be something of value to others in order for us to expect our successors to do a bit of work to preserve and pass on the collection. So it comes down to that. Is there anything in a collection worth attempting to preserve for generations? I think that has to be determined well before any decision is made to invest in archival storage and the legitimate effort that goes into organizing it.
#13. "RE: Is digital archival" In response to Reply # 9
Even the great films in "film" storage need to be maintained. They may find it easier and less expensive to maintain film instead of digital files, but it still will take effort and expense.
The difference is, the films will be institutional and industry archives, while for the most part, our archived work, pros and amateurs alike, will be accomplished by the photographers themselves.
I would hasten to add that over the years, thousands of films, many mediocre, more than a few, some of the best, have been partially or totally lost forever, as the industry and institutions have drastically underestimated over and over again what it takes to maintain an archive.
#14. "RE: Is digital archival" In response to Reply # 0
FWIW I'd use PNG as an archive format over JPEG as PNG is lossless and JPEG is not. PNG has no known patent issues and is a very popular internet image format. It doesn't compress as well as jpeg but that's the price of lossless-ness.
#15. "RE: Is digital archival" In response to Reply # 12 Wed 05-Mar-14 04:01 AM by rectangularimage
> The first question to ask is, why should anyone care?
It's always exciting to find that box of Grandpa's photos or negatives. But it seems a less sentimental scenario to find Grandpa's old rotating plate mag storage unit at the bottom of the trunk. Are the kids going to try and access its contents, order the antique adapters and get special software (to connect it to their watches), or are they just going to laugh and throw it in the trash?
#16. "RE: Is digital archival" In response to Reply # 12
>The first question to ask is, why should anyone care?
I remember getting to my grandparents house after they were both deceased. I was the last grandchild on the scene, as I came from out of state. The cousins and sister took all the good furniture and other personal belongings.
On the middle of the living room floor were all the photos and albums that I had so often perused as a child. All very rich in family history. As the family genealogist, I felt that I had the really important items. Their memory is far more valuable than a corner hutch!
Scott Chapin Powder Springs, GA, USA Nikonians Team Member
#17. "RE: Is digital archival" In response to Reply # 0
This whole conversation has compelled me to make a lot of prints on thick ("archival") cotton paper in black and white ink (which I have a inclination to do anyway) and put them in boxes.
My digital creations are destined to be ephemera, realistically thinking. The physical stuff possibly less ephemeral. Which brings up the question: what pictures would I less not liked to be remembered by?
#18. "RE: Is digital archival" In response to Reply # 5
Engineering faces much the same problem with a shorter time scale but much larger databases representing an object, say a computer board. The only way the engineering community has found to preserve an old design over many years is to preserve a workstation _with software_ that can manipulate and open the database.
For photographers, that means be cautious with upgrades to both hardware and software. Keep the old "known-good" software loaded on your computer. If software version n is the newest release, keep version n-1 on your computer and keep the distribution files for version n. When you move to a new computer, keep the old one up and running until you're satisfied that the new one is stable.
For image files, back them up on at least three separate storage devices. I don't trust hard drives but the size and number of images that I produce means there isn't a reasonable alternative to hard drives, so I have two fault-tolerant servers that mirror each other at opposite ends of the house. I'm very glad I did because last year I had a spate of dying hard drives. Because of the RAID configuration and multiple servers, I was never in danger of losing anything.
Aside from the servers, I keep a separate backup on a portable USB hard drive so I can grab it and run. There's a lot of talk about Cloud storage but the amount of storage isn't large enough and the access is very slow, making backups difficult. If it's hard, you won't do it.
I'm afraid that "archiving" digital images even 50 years out, on the scale needed to open and manipulate my photo archive is just fantasy. I have a fairly large number of images (about 55,000) but I'm sure there are people here, especially professionals, who have many more. Fifty years out, it may be possible to open and manipulate any given image if it is valuable enough to make the effort but to do so on a mass basis would require preserving that workstation and software I mentioned above.
jbloom Wethersfield, US Nikonian since 15th Jul 2004
Wed 05-Mar-14 08:51 AM
#19. "RE: Is digital archival" In response to Reply # 0
Some interesting comments here. I'll add my two cents worth.
First regarding the value of our photo archives: I like looking through old photo collections because they immerse me in the time, place, culture and style of the period. Naturally, I especially like my own family's photos because the people in them are more close to me, but even unrelated people are of interest to me. It doesn't require that the photos be artistically significant, either.
With that in mind, when I think about my own photo archive I realize that the landscape images I've so painstakingly crafted are likely to be among the least interesting of my photos to future generations. There are plenty of other landscape images being made of essentially the same scenes, and I'm sure a lot of them are better than mine. I enjoy doing them and am not going to stop, but I recognize that their value is not likely to be long-lasting. Hopefully, though, my images of people and human places will have more to say to my descendants.
Meanwhile, my brother has been undertaking a project to sort through my father's thousands of slides to scan and preserve the ones of most interest. While we tend to scoff when someone lines up the family in front of a historic or scenic view to take a family snap, I have to admit that when I look at the images my dad took with his Instamatic in the 1960's, the ones where we are lined up in front of Stonehenge are more interesting than the ones of Stonehenge with no people in the shot.
Now on to technical matters. I'm not especially concerned about the longevity of common file formats such as TIFF and JPEG. (While the Capture NX debacle is what sparked this discussion, I'd never consider an intermediate format such as that to be a candidate for long-term archival use.) If the files themselves can be preserved, I'm sure there will be software capable of reading them 100 or more years from now. Why am I confident of that? Because we are not alone in this concern. There are millions of images stored in these formats, and there will be a demand many years from now to be able to recover the images from such files. And even for the more obscure formats (cough, NEF, cough), as a friend of mine is wont to say when asked if we could write a program to do some task, "it's only software."
So the nub of the preservation problem, I think, is preserving the physical manifestation of the image, whether that be in magnetically aligned molecules on a storage medium or as a final image on paper or film. For individuals, I think a final image medium -- paper or film -- is much the better bet for really long term survival. To preserve your images, print them and store them securely.
It is also possible to preserve the digital bits on paper. Somewhere in my boxes of junk I have a computer program that I wrote in about 1977. It exists because it was recorded on punched paper tape via an ASR 28 teletype machine. If I wanted to, I'm sure I could turn that paper tape back into electronic bits without too much difficulty. A more modern approach to bits-on-paper would be something akin to a 2-D or 3-D bar code. Whether anything like that is worth doing as an archival approach is a question, but that it's doable I have no doubt.
#20. "RE: Is digital archival" In response to Reply # 12
Great and thoughtful post Howard. Thanks.
Actually, for many of the professional photographers here, and I'll bet, some amateurs, we do have many of our images kind of archived, one can say, by the government of the country in which we live.
My images, at least my published images which number in the thousands, are both physically and digitally archived by the US Copyright Office. I'm old enough that they have many copies of prints, as well as digital image files. The "archival" repository they maintain is not exactly convenient to see a work they have in storage, or retrieve a copy, at this point, but it is there, and they are accessible.
jec6613 Norwalk, US Registered since 12th Feb 2013
Wed 05-Mar-14 12:46 PM
#21. "RE: Is digital archival" In response to Reply # 19
>Meanwhile, my brother has been undertaking a project to sort >through my father's thousands of slides to scan and preserve >the ones of most interest. While we tend to scoff when someone >lines up the family in front of a historic or scenic view to >take a family snap, I have to admit that when I look at the >images my dad took with his Instamatic in the 1960's, the ones >where we are lined up in front of Stonehenge are more >interesting than the ones of Stonehenge with no people in the >shot.
When my grandfather died, he had boxes and boxes of slides - only ones with people in them were kept. It's a lesson I've learned, and so now anything with people in it, I have printed, neatly label it on the back with who's in it, the date, location and photographer, and then store in the proverbial shoebox.
The digital archive is more impressive, with it indexed by every family member and every picture they're in (want a picture with my sister and my aunt? a few clicks and I can deliver them all), but the prints are longer term.
#22. "RE: Is digital archival" In response to Reply # 16
>On the middle of the living room floor were all the photos and >albums that I had so often perused as a child. All very rich >in family history. As the family genealogist, I felt that I >had the really important items. Their memory is far more >valuable than a corner hutch!
Great point, Scott. In an increasingly, densely populated world containing so much enabling mobility, genealogy is gowing in popularity for very good reasons. So it follows, I think, that one anwer to the OP's question is that the storage format used in the present is irrelevant, as long as the data is accessible in some way for the next steward (genealogist?) who comes along in some sort of family succession. The chain of custodianship or stewardship will only be broken when a successor fails to materialize at some point down the road. That would suggest that it is impossible, in many families, for the maintenance of photo archives to be guaranteed.
A bequest in a Will for a certain amount of money to be paid toward a family photo archive seems worthy of consideration. The establishment of an archive for permanent storage and expansion of family archives might be an investment worth making too - control the shape and evolution of the archive while we're still alive to have some influence. So that begs the question, what companies - what big and stable companies with a reasonably predictable future that is - are offering archival services at this time?
#23. "RE: Is digital archival" In response to Reply # 15
>It's always exciting to find that box of Grandpa's photos or >negatives. But it seems a less sentimental scenario to find >Grandpa's old rotating plate mag storage unit at the bottom of >the trunk.
Perhaps, and I share the feeling, but I feel that way primarily because my formative years were filled with prints and negatives. For a generation now (and for future generations) raised on digital photography, an SSD will often be just as fascinating to them in the future as a box of old prints and negs is to us now.
>Are the kids going to try and access its contents, >order the antique adapters and get special software (to >connect it to their watches), or are they just going to laugh >and throw it in the trash?
Great question. I'd bet that the exact same percentage of people who dump boxes of old photos and negs in the trash now will dump storage devices loaded with photo and video files in the future. People don't change. There will always be a significant percentage of people who retain/hoard/store everything with the intention of sorting it all out and keeping only the good stuff but never actually do the work, a significant percentage of people who sort things out with brutal efficiency and retain only what they consider worthwhile, a significant percentage who work at it gradually and then lose interest after a while, and a significant percentage who simply dump it all because it's either too much work or brings up too much emotion or is simply utterly uninteresting to them.
It think the tiniest percentage of all is populated by those who are genuinely interested in photography history, family history, travel and social history as represented by their photo collections and who also can boast of successors who share the same interests.
#24. "RE: Is digital archival" In response to Reply # 0
If we're going to compare digital image file formats, it's probably worth considering and comparing the companies and organizations that incept and maintain such formats. Nikon, as a company, has been around for a while and has a parent of sorts. Founded in 25 July 1917 as Nippon Kōgaku Kōgyō Kabushikigaisha (日本光学工業株式会社 "Japan Optical Industries Co., Ltd."), the company was renamed Nikon Corporation, after its cameras, in 1988. Nikon is one of the companies of the Mitsubishi Group.
The point is that such a massive, collective entity that has been around for so long in a readily identifiable way is a much better bet for long term NEF file format support than JPEG, PNG or even Adobe DNG, none of which have been around for more than a very small fraction of the time. NEF, for Nikon photographers, might be a better bet than anything else out there. So a question to ask might be, what is Nikon doing or considering with respect to very long term archivability of its NEF image file format in order to ensure that great digital photos made as NEF images that exist only on digital storage media as originals are readable by some utility and converter utility in the future?
Has anybody in this thread started a conversation with Nikon about this? I have not, but as I write this post it seems like a good idea to contact Nikon and ask the question.
#25. "RE: Is digital archival" In response to Reply # 0
>My images, at least my published images which number in the >thousands, are both physically and digitally archived by the >US Copyright Office. I'm old enough that they have many copies >of prints, as well as digital image files.
A great point, Ned! For U.S. photographers who are diligent about copyright registration (as you've rightly pointed out over the years they should be), it is an awkward but fully legitimate solution. Arguments for and against the survivability of image files registered for copyright and stored by a government often acknowledge that various government departments literally can't find anything when personnel are pressed even by legal fiat to dig something up out of the archives. Still, I think that it's a legitimate bet if a photographer has no other archival storage plan.
For Canadian photographers who register their image files for copyright it's a different matter. Mass uploading under a single fee payment is not possible so far. One photo, one fee - that's the (expensive) Canadian way up to now, and I don't hear any rumblings about change to a U.S.-style registration system. The point is that there are consequently far fewer registered photos per photographer in Canada.
One of the biggest problems with any considerations about archivability (of anything) might be the idea that just because a person suddenly thought about digital image archiving, that it is also suddenly an imperative.
It occurs to me now that a number of formerly small photography museums and photography centres around the world have expanded in recent years. The Photographers' Gallery in London recently underwent a years-long renovation and expansion after moving from a small, dingy location to bigger digs in a more accessible location. The Maison Européenne de La Photographie in Paris has also benefited from expansion, new digs and a lot more financial support. Both of these organizations are nibbling at archival storage of their own exhibitions and might be an ideal place to start (at least in the UK and in France) for photographers in those countries to begin lobbying for genuinely supportable archival storage of digital images of all kinds. Certainly, the rise of digital photography and its impact on the accessibility to and spread of photography in general has resulted in an increase in the number of dedicated photography galleries all over the world. So is that the place to start a conversation about digital image archiving resources?
Does Facebook have a plan for long term archival storage of the literally billions of digital images currently accessible across its registered membership footprint? Will Facebook even be around in ten years (and will anyone even care)? The answer to the latter question is likely, no. So what are individual Facebook account holders doing to preserve the photos they've posted - you know, the ones titled "Doesn't this chicken marinara look great!?" Well, maybe we can lose those ones deliberately.
The question of archivibility doesn't necessarily come down to a choice of image file format, I think, so much as it comes down to a willingness on the part of an individual and his or her successors to create a plan and stick to it. For any photographer genuinely interested in creating a long term archive (to which future generations can add - there's that too), a balance of cost, storage reliability and stewardship has to be established along with a budget to support it all. It's for all those reasons that the great art and photography from current and previous generations most often ends up in the hands of or in the care of publicly and privately supported galleries and museums because very few people in the world can muster the resources to do so on their own.
There are a billion paintings and sculptures (or a million or a hundred million or a hundred billion - the exact number is irrelevant) that have been lost to time, tides, carelessness and disinterest over the past couple of thousand years. Though a few people in archeology and the art world wonder about it sometimes, we generally don't mourn the uncontrollable loss of all that stuff over the ages. Besides that, where would be put it all if all of it had survivied or been preserved to this day!? Far more than enough has survived and been preserved for now and for the edification of future generations. The same will be true of digital photos.
#26. "RE: Is digital archival" In response to Reply # 24
What a great idea about speaking with Nikon about the NEF format, Howard. I think I'll try, however, it may be extremely hard to find out anything about it, considering how hard it is to get tech information from Nikon, and any statement of long terms plans from them. They are truly "inscrutable," more often than not.
By the way, I agree with you about NEF. JPEG isn't a quality archival format anyway due to it's lossy characteristics, and range of options. While PNG is lossless, I'm not sold on it due to its myriad of options, something that makes TIFF even worse. And let's not even consider DNG since it depends on the largess of Adobe, and has inherent weaknesses too. At least Nikon has been around for a long time and they have a vested interest in the NEF format. The way tech companies come and go, however, even Nikon isn't an "archival" type bet, in my opinion.
I think for now we need a multi-faceted approach and archive via several prongs of image filing.
There are no easy answers concerning archiving of digital images. Much more thought is needed in this area, by photographers and the graphics industry.
jbloom Wethersfield, US Nikonian since 15th Jul 2004
Wed 05-Mar-14 02:30 PM
#27. "RE: Is digital archival" In response to Reply # 24
>If we're going to compare digital image file formats, it's >probably worth considering and comparing the companies and >organizations that incept and maintain such formats.
OK, JPEG and PNG are maintained by international organizations, the ISO, which dates back to 1946, and the ITU, which dates back to 1865. The ISO also puslishes the TIFF and DNG standards (DNG being a variant of TIFF). Both of those organizations are supported directly or indirectly by numerous governments (including the US and Japan) as well as many companies.
But I don't think the longevity of the controlling organization is at all the way to judge, especially given that Nikon just backed off on supporting aspects of NEF that they previously supported! No, the better test is how widely used a format is. Any format that is 1) not published and 2) used by only one company is inherently risky.
The value of published standards is that no reverse engineering is needed to support them. Because of that, those standards (especially JPEG and TIFF) are supported interoperably by multiple companies and open-source products. One can say that about NEF only insofar as extracting the raw data, white balance and preview information is concerned, and much of that was reverse-engineered.
Also, NEF is not a final-image format (although it includes a JPEG preview). Just reading the file is insufficient; data demosaicing and tone/color mapping is required to recover the original image. That's not what I hope for in an archival format.
In terms of the archivability of a RAW format, I'd vote for DNG over NEF. Not only is it a published standard, but it will have more future applicability since other manufacturers' RAW formats can also be converted to DNG.
#28. "RE: Is digital archival" In response to Reply # 25
>A great point, Ned! For U.S. photographers who are diligent >about copyright registration (as you've rightly pointed out >over the years they should be), it is an awkward but fully >legitimate solution. Arguments for and against the >survivability of image files registered for copyright and >stored by a government often acknowledge that various >government departments literally can't find anything when >personnel are pressed even by legal fiat to dig something up >out of the archives. Still, I think that it's a legitimate bet >if a photographer has no other archival storage plan.
Yes, that's true. The government loses stuff all the time, or more accurately, misplaces it, or can't find it. That said, at least in the US, between the National Archives and the Copyright Office, the job they've been doing to date has been pretty good considering the task.
>For Canadian photographers who register their image files for >copyright it's a different matter. Mass uploading under a >single fee payment is not possible so far. One photo, one fee >- that's the (expensive) Canadian way up to now, and I don't >hear any rumblings about change to a U.S.-style registration >system. The point is that there are consequently far fewer >registered photos per photographer in Canada.
I didn't know the Canadian system was that bad, or better said, still that expensive.
>One of the biggest problems with any considerations about >archivability (of anything) might be the idea that just >because a person suddenly thought about digital image >archiving, that it is also suddenly an imperative.
>It occurs to me now that a number of formerly small >photography museums and photography centres around the world >have expanded in recent years. The Photographers' Gallery in >London recently underwent a years-long renovation and >expansion after moving from a small, dingy location to bigger >digs in a more accessible location. The Maison Européenne de >La Photographie in Paris has also benefited from expansion, >new digs and a lot more financial support. Both of these >organizations are nibbling at archival storage of their own >exhibitions and might be an ideal place to start (at least in >the UK and in France) for photographers in those countries to >begin lobbying for genuinely supportable archival storage of >digital images of all kinds. Certainly, the rise of digital >photography and its impact on the accessibility to and spread >of photography in general has resulted in an increase in the >number of dedicated photography galleries all over the world. >So is that the place to start a conversation about digital >image archiving resources?
I agree, I'm hoping that the museums and major pros in my community start the archival dialog they have been speaking of, and complaining about for years, finally begins in earnest. I think it can be worth the effort.
>Does Facebook have a plan for long term archival storage of >the literally billions of digital images currently accessible >across its registered membership footprint? Will Facebook even >be around in ten years (and will anyone even care)? The answer >to the latter question is likely, no. So what are individual >Facebook account holders doing to preserve the photos they've >posted - you know, the ones titled "Doesn't this chicken >marinara look great!?" Well, maybe we can lose those ones >deliberately.
Not only don't I think Facebook and other social networking sites have an archival plan, I don't think they have given it much thought, nor do I think they are likely to worry about it. I don't see how it fits into their publicly stated business plans.
>The question of archivibility doesn't necessarily come down to >a choice of image file format, I think, so much as it comes >down to a willingness on the part of an individual and his or >her successors to create a plan and stick to it. For any >photographer genuinely interested in creating a long term >archive (to which future generations can add - there's that >too), a balance of cost, storage reliability and stewardship >has to be established along with a budget to support it all. >It's for all those reasons that the great art and photography >from current and previous generations most often ends up in >the hands of or in the care of publicly and privately >supported galleries and museums because very few people in the >world can muster the resources to do so on their own.
Agreed, and the plan has a multitude of factors which will need to be part of it, and in the long run, few will actually have a plan and most images will disappear forever.
>There are a billion paintings and sculptures (or a million or >a hundred million or a hundred billion - the exact number is >irrelevant) that have been lost to time, tides, carelessness >and disinterest over the past couple of thousand years. Though >a few people in archeology and the art world wonder about it >sometimes, we generally don't mourn the uncontrollable loss of >all that stuff over the ages. Besides that, where would be put >it all if all of it had survivied or been preserved to this >day!? Far more than enough has survived and been preserved for >now and for the edification of future generations. The same >will be true of digital photos.
#29. "RE: Is digital archival" In response to Reply # 27
Great points, Jon. An ISO-supported format may be an even better bet.
So when are we going to lobby the ISO board of directors and Mitsubishi Group's board about preserving and maintaining image file readers? Like Ned, I think I'm going to contact Nikon about it just for kicks. We'll see what, if anything, they have to say about it. Same goes for ISO and possibly ITU. Who knows - maybe they've got something in the works already.
jbloom Wethersfield, US Nikonian since 15th Jul 2004
Wed 05-Mar-14 02:43 PM
#30. "RE: Is digital archival" In response to Reply # 26
>By the way, I agree with you about NEF. JPEG isn't a quality >archival format anyway due to it's lossy characteristics, and >range of options. While PNG is lossless, I'm not sold on it >due to its myriad of options, something that makes TIFF even >worse.
Are you aware that a NEF file is also a TIFF file? And while TIFF formats can be complex, NEF being an example, the standard 3- or 4-channel image-data formats (for RGBa and/or CMYK) are quite stable. You can export a TIFF image from almost any supporting program, open it in any other program and get exactly the same result. That's not something one can say about a NEF file. (Plus there are open-source implementations of the JPEG, PNG and TIFF standards, which argues for their long-term viability.)
>And let's not even consider DNG since it depends on the >largess of Adobe, and has inherent weaknesses too. At least >Nikon has been around for a long time and they have a vested >interest in the NEF format.
I'd say their vested interest isn't what many thought it was up until a few days ago. Nikon is in the camera business. Adobe is in the software business. Who has the greater incentive to make their software products widely interoperable and stable?
>There are no easy answers concerning archiving of digital >images. Much more thought is needed in this area, by >photographers and the graphics industry.
Indeed. It's a moving target, and I can't think of any present solution that I'd consider a slam-dunk answer.
#31. "RE: Is digital archival" In response to Reply # 30
>>There are no easy answers concerning archiving of digital >>images. Much more thought is needed in this area, by >>photographers and the graphics industry. > >Indeed. It's a moving target, and I can't think of any present >solution that I'd consider a slam-dunk answer.
Agreed. We are all still grasping at answers which don't yet exist. That said, each of us must do something at this point to do the best we can to preserve our work, as best we can, if for no other reason than need.
#32. "RE: Is digital archival" In response to Reply # 30
>Are you aware that a NEF file is also a TIFF file?
Another good point. I'm familiar with most of the current formats and their antecedents because of a background in digital photo editing software development. It's that background, and the related familiarity with format licensing issues, the myriad TIFF versions in the '90s and a number of other problems which form the basis of concern in my thinking especially when considering the absence, for private individuals, of a clear archiving solution on the (hardware) back-end of all this that includes future file readability.
>I'd say their vested interest isn't what many thought it was >up until a few days ago. Nikon is in the camera business. >Adobe is in the software business. Who has the greater >incentive to make their software products widely interoperable >and stable?
I'd add that it's also worth considering that Nikon is the originator of its format and will most likely always have the best, direct converter (even if it's less feature-filled) of its own NEF format into other image file formats. So as long as I continue my use of Nikon cameras, Nikon's image file tools seem to be the best bet for archival reading and conversion insofar as a decision about where to put my archival efforts are concerned. Then again, your earlier point about ISO and ITU standards control and maintenance is just as relevant if not moreso for practical future considerations.
I do not trust Adobe. Of all the companies out there that can at least be expected to some small degree to address archival stored image file readability, I think that Adobe should be front and centre on the issue. Adobe has offered, indirectly, at least one bridging format - PDF - but left it to asset management software companies to create a unified archival methodology and software. Adobe Bridge, Lightroom and their associated asset management functions are all important areas of development at Adobe which are dealt with by separately budgeted, co-operative but distinct teams at Adobe. Still though, the company has not corporately directly addressed archival storage in any significant way so far. If Adobe turns out to be co-operating behind closed doors with other companies intent on developing and creating truly archival storage systems, then I'll probably start to like Adobe a bit.
>Indeed. It's a moving target, and I can't think of any present >solution that I'd consider a slam-dunk answer.
The Federal Bureau of Investigation (FBI) formed the Scientific Working Group on Imaging Technologies (SWGIT) in the mid 1990s. The same is true, during more or less the same time period, for other national policing organizations in other countries. I have no idea what progress has been made on any of those fronts. The efforts in this area are directly related to the preservation of digitally originated forensic evidence - photos and scans.
I do know that a number of local (to me) government organizations, public and private galleries and some private companies are either considering or actively using PDF and/or JPG file formats for document and image archiving, using either proprietary front ends bolted onto remote data storage facilities or third-party fronts ends for the same storage facilities. Often, important images are contained within documents and archived as PDF files. The vast majority of my investigative research (a few petabytes at this point - 40+ years of data and projects at this point) are archived and backed up in the form of PDF files because the documents and related image files frequently are shared and/or licensed to third and fourth parties. Accessibility and readability is King. Related to the OP's questions, if the data (documents, image files, video files) isn't accessible and readable it might as well not exist.
The scalability of archival storage and related (relatively) fast access used to be the biggest problem for my business partner and me. But Amazon's back end development (and the back ends being developed by competitors) solved that a few years ago when Amazon (for one) started selling access to third-parties who've created licenseable GUIs. The instantaneous scalability of storage is the next best thing to actual magic.
In the end, I have no choice thus far - I have to put my secure trust in massive third-party entities (Amazon on the back end, Adobe PDF as a primary format, CMS for file management) for archival stability and longevity. Expensive and complex.
#33. "RE: Is digital archival" In response to Reply # 25
>So what are individual >Facebook account holders doing to preserve the photos they've >posted - you know, the ones titled "Doesn't this chicken >marinara look great!?" Well, maybe we can lose those ones >deliberately.
Ok, now that's funny, right there.
FWIW, I approach this from the perspective of someone who is really only interested in family photos. I assume that the museums and fine art guys will take care of themselves.
Your earlier point ("will anyone even care?") is an interesting one.
100 years ago, a photograph of a family member was an expensive, rare and precious thing. Something to be cherished.
Today? Please. With Facebook/Twitter/Instagram/Vine/Youtube and a million others, photos are utterly commonplace. Snapchat set the bar here. People have an active expectation that photos are temporary.
It occurs to me that it may very well come to pass that, despite having taken more photos than at any time in history, the great grand children of today's teenagers may find that NONE of them survive. Some irony there.
> The question of archivibility doesn't necessarily come down to a > choice of image file format, I think, so much as it comes down to > a willingness on the part of an individual and his or her successors > to create a plan and stick to it.
I think that's exactly right. Far more than being a technical question, the real question is whether or not some future family member will place enough value on the collection to bother to preserve it. Its all about the human factor.
Which gets to the problem mentioned before. When there are millions of photos of someone out there, none of them (individually) have any value. If they all end up being lost, that is why.
And so I go back to my opinion that, for the typical individual (ie, not a photography enthusiast), quality prints on archival paper are more likely to survive precisely because they are different from the endless stream of digital photos of chicken marinara.
jgould2 Fort Pierce, US Nikonian since 13th Oct 2007
Wed 05-Mar-14 04:03 PM
#34. "RE: Is digital archival" In response to Reply # 27
Thanks for saving me the trouble of mentioning that a proprietary image format like NEF (which requires a new RAW conversion program for each new camera) is the last format I would choose for archivability.
#35. "RE: Is digital archival" In response to Reply # 24
>The point is that such a massive, collective entity that has >been around for so long in a readily identifiable way is a >much better bet for long term NEF file format support than >JPEG, PNG or even Adobe DNG, none of which have been around >for more than a very small fraction of the time.
I very much disagree with this, I think the more popular file formats (JPEG, TIFF, PNG) are a safer bet. There's orders of magnitude more of these files in existence, more processor implementations, they're ISO standardized, and they're standard internet media (MIME) types.
#36. "RE: Is digital archival" In response to Reply # 35
I think digital is a tiny bit more archival than prints-only.
Its quite possible to keep up-to-date with formats -- I can for example to a mass convert of my NEF+Edits to DNG today -- something else tomorrow. From Lightroom to Xyz next week.
The issue is the desire to do so. That applies to prints/negatives/slides/NEFs/TIFFs/whatever.
For personal data, will my kids care to keep all the photos? Did "I" keep all my Dad's photos? (not yet, trying, about 5000 slides to copy). I have 3 kids, who gets my Lightroom catalog? my B&W prints? if I choose the wrong one of the 3, they may just disappear.
For a country say, is there a central archive that keeps good historical data? Will there be a civil war where nobody cares enough and the all get blown up??
From a purely technical standpoint, digital CAN be perfect in an archival sense since one can keep converting to newer formats -- the real issue will be the people -- and from that standpoint, good prints may in fact be better because you have to actively discard them, while with digital if you ignore them they will likely cease to be viable.
I will pass all this on to my kids see who grabs hold
#37. "RE: Is digital archival" In response to Reply # 29
At least two principal agencies within the US Government have an interest in archiving documents - the National Archives and Records Administration (NARA), and the Library of Congress. NARA's FAQ site on permanent documents requires them to meet PDF/A-1 standards for permanent transfer into the National Archives. NARA imposes some additional requirements - see here for additional information. NOTE: In addition to PDF/A-1 are -2 and -3 files. All 3 are traceable to ISO standards. Based on this, I would venture to say the the present gold standard for archival-quality documents would be PDF or one of its derivative files. Of course very very few people will send their "personal" records to the National Archives , but PDF's appear to be the best format, at this time, for long-term document preservation, rather than NEF, PNG, JPEG, TIFF, or whatever other format one might want to consider.
So much for the file itself. The media on which files are stored present another problem, of course. I don't know if the highest-grade CD's/DVD's yet have a life span of more than 100 years before they might begin to deteriorate. Magnetic media probably still fall far short of that time-frame. Then there are the machines and software needed to read the files/media. These all need to be kept in environmentally ideal conditions. This is just to recap; most, if not all, of this has been discussed. A starting place for information on "Personal Archiving" is here, on a Libray of Congress web page.
There are some additional elements that need to be emphasized (forgive me if they have been mentioned and this is repetitious), and that is indexing and cataloging the files. There also must be a record of the location of the media containing the files, and the machines needed to access them electronically. Archiving a file/collection of files without identifying it, AND being able to find it, means that for all practical purposes it is lost. The index, catalog, and location information also should be preserved to the same standards of accessibility, and have the same expected life span, as the documents themselves.
sabbey51 Saddle river, US Nikonian since 11th Jan 2010
Thu 06-Mar-14 08:58 PM
#38. "RE: Is digital archival" In response to Reply # 33
As was mentioned before, "archival" really needs to be understood as a verb rather than a noun. My professional career was in IT, and had it's roots in database administration, where you learned early that the key to the integrity of your data was in the quality of the processes you built and operated rather than the technology, media or formats you used. So the key issue raised in this thread is correctly not CAN any of these formats/media/technology be preserved indefinitely, but WILL anyone actually do so.
For me, I've started actively segmenting my images into sets which will have different levels of interest to my decedents.
I think it unlikely that any of them will be terribly interested in the truly excellent images I took in Yosemite.
I think it's very likely that many of them will be interested in the carefully metadata'd group family pictures which I've been collecting that document our growth over the last number of years. This is where you can see what Grandpa looked like as a small boy, and what his parents looked like. You can see everyone on the ski vacation when one generation was small, and another in their prime years. I think the metadata is critically important, as it adds context that will make future generations interested - rather than trying to interpret the handwritten, faded, pencil note on the back of a B&W print, you can see exactly when the picture was taken, where, and who is in it (assuming someone preserves the metadata along with the image). These are published in many forms: jpeg copies on hard drives, Flickr, email, prints, etc. I believe that the law of large numbers (large number of copies, large number of formats/media, large number of holders) will make these family artifacts survivable for quite some time. This diversity is the essence of any viable disaster recovery plan, and also supports this type of archival process.
If they die from lack of interest, then by definition no one will care.
And in a pinch, one could always contact the NSA for their copy.
Life is what happens while you're busy making other plans