Network Attached Storage Workflow
DaddySS
Nikonian since 26th Dec 2006
Mon 29-Aug-16 08:39 PM
Richard
Visit my Nikonians gallery.
|
-
#1. "RE: Network Attached Storage Workflow" | In response to Reply # 0
adf Registered since 19th Mar 2006Tue 30-Aug-16 11:58 AMRichard,
I have a similar NAS setup to yourself, RAID5 although different hardware. My philosophy is to try and keep it as simple as possible with as few file transfers and copies to synchronise.
For the RAW file storage I import the files from the card/external portable drive to an import folder on the NAS and then use Lightroom to import them, adding simple metadata tags, renaming them, moving them to the right folder and adding them to the catalogue. I have snapshots enabled to capture a daily snapshot of the NAS and back it up to an external USB drive about once a month (or after a major upload) which I keep separately. I have looked at online cloud storage also but most/all exclude NAS.
My Lightroom catalogue is on an SSD drive in my main computer and is automatically backed up weekly to the NAS.
The difficult bit is the sharing of the Lightroom catalogue as it's not really designed for multiple users/devices and I've decided that it adds more risk than it's worth so restrict myself to using the main computer which is what I do in reality 99% of the time anyway. I use standalone Lightroom but have looked at Lightroom CC which allows for editing based on smart previews and across multiple devices.
From research and experience workflow is quite a personal thing based on style, risk appetite to name but a couple and I'd be interested to see others' approaches to see if I can tweak my approach.
Hope this helps.Visit my 500px gallery.
Visit my YouTube channel.
Visit my Nikonians gallery. -
#2. "RE: Network Attached Storage Workflow" | In response to Reply # 0
esantos Nikonian since 10th Nov 2002Tue 30-Aug-16 12:05 PMRichard,
Have you considered that on initial ingest of files that you import directly to your NAS instead of to the hard drive on your PC? This is what I do and then begin the archival/backup/working duplicate process from that point. And when I am traveling I import the files from my camera memory cards to my laptop and then while I am sleeping I export to my NAS via the cloud overnight. That way when I get home from my travels everything is already on my NAS. This is so much better than in the past when I imported to my laptop and at one time burned the images to DVDs and then when they became readily available portable external hard drives. Things are so much easier now with my NAS.Ernesto Santos
esartprints.com Ernesto Santos Photography
Get my new e-Book "Churches of Texas"See my portfolio.
-
#3. "RE: Network Attached Storage Workflow" | In response to Reply # 2
DaddySS Nikonian since 26th Dec 2006Tue 30-Aug-16 04:56 PMThanks Adrian and Ernesto for the replies. I see you are both on the same track. The ingest directly to the NAS would simplify the initial sharing of the files which I could then save to a different directory on the NAS - keeping the originals untouched. Now the question is how do I synch the working copies with the other PCs. QNAP has a synch function but it doesn't appear to maintain the file structure - just mashes all photos in one photo file. The other question is sharing the lightroom working copies. If I could synch the catalog on the NAS to catalogs on the PCs I think I'd be there.Richard
Visit my Nikonians gallery.-
#4. "RE: Network Attached Storage Workflow" | In response to Reply # 3
jpope42 Nikonian since 25th Nov 2004Tue 06-Sep-16 09:33 PMI'm VERY simple minded so here's what works for me.Initially I use Adobe Bridge to simultaneously download to my hard drive and to my NAS. The two identical directories store my negatives by date. I use a cloud based backup for both therefore have 3 copies two places. After that:
Working files go to a WIP directory on the local (cloud & NAS backup)
Master edit files go to a Master directory on local (cloud & NAS backup)
Master print files (size, paper & printer specific) go to a Print Directory on Local (cloud & NAS backup)
.. and so on.Jim
www.borrowedlightphotography.com
Visit my Nikonians gallery.
-
-
#5. "RE: Network Attached Storage Workflow" | In response to Reply # 0
Ferguson
Nikonian since 19th Aug 2004
Thu 08-Sep-16 10:56 AM
This is timely as I just built a new desktop, and have spent the last few days experimenting with recycling the old system in various NAS solutions.
I have gone through a lot of scenarios, from using the NAS as working storage, to using it as the focus for cloud and other detached backups.
Here is what I settled in on, with the primary goal being "simple": It's just a big external hard drive. Currently I back up irregularly to two EHD's, and nightly to two cloud services. One EHD's are getting a bit small. I had purchased a large EHD (Samsung D3 5TB) and had it fail in less than a week, so a more reliable NAS solution seemed timely.
Everything is on raid during the day to minimize loss from hardware issues in the short term, and I don't erase my camera cards until the next shoot just in case.
What I plan to do is set the NAS up for wake-on-LAN, back up nightly (incrementally) to it after waking it up, same for my wife's computer, then shut it down. I will likely keep at least one additional EHD for safety, and the two cloud backups.
I am using two different backup programs, just in case of a software issue, one for each cloud, and different from the EHD and NAS (or maybe both on NAS)
I am trying to avoid a lot of file movement during workflow, as that increases the chance of corruption. The (singular) home of the images is the location I import to, and they stay there forever.
On the new NAS I'm using zfs to reduce the chance of bit rot, and on the desktop I regularly scan all the images for MD5 checksums and compare to the prior time.
I also have been spooked a bit by all the ransomware stories, and I want to be sure I am keeping all "external" storage off line except when actually in use. I know some people keep the EHD's in and spun up all the time (or NAS is up and mapped), and several recent variants of the malware have been hunting out and encrypting attached storage as well. I have never been infected, and prevention is important, but I think planning for when you are is still prudent.
All backups are versioned, so any file changed/deleted is still retained for two years.
In other words, belt, suspenders, and a bit of super-glue to keep the pants up.
Linwood
Comments welcomed on pictures: Http://www.captivephotons.com
I have gone through a lot of scenarios, from using the NAS as working storage, to using it as the focus for cloud and other detached backups.
Here is what I settled in on, with the primary goal being "simple": It's just a big external hard drive. Currently I back up irregularly to two EHD's, and nightly to two cloud services. One EHD's are getting a bit small. I had purchased a large EHD (Samsung D3 5TB) and had it fail in less than a week, so a more reliable NAS solution seemed timely.
Everything is on raid during the day to minimize loss from hardware issues in the short term, and I don't erase my camera cards until the next shoot just in case.
What I plan to do is set the NAS up for wake-on-LAN, back up nightly (incrementally) to it after waking it up, same for my wife's computer, then shut it down. I will likely keep at least one additional EHD for safety, and the two cloud backups.
I am using two different backup programs, just in case of a software issue, one for each cloud, and different from the EHD and NAS (or maybe both on NAS)
I am trying to avoid a lot of file movement during workflow, as that increases the chance of corruption. The (singular) home of the images is the location I import to, and they stay there forever.
On the new NAS I'm using zfs to reduce the chance of bit rot, and on the desktop I regularly scan all the images for MD5 checksums and compare to the prior time.
I also have been spooked a bit by all the ransomware stories, and I want to be sure I am keeping all "external" storage off line except when actually in use. I know some people keep the EHD's in and spun up all the time (or NAS is up and mapped), and several recent variants of the malware have been hunting out and encrypting attached storage as well. I have never been infected, and prevention is important, but I think planning for when you are is still prudent.
All backups are versioned, so any file changed/deleted is still retained for two years.
In other words, belt, suspenders, and a bit of super-glue to keep the pants up.
Linwood
Comments welcomed on pictures: Http://www.captivephotons.com
G
Upload images
6
Uploading and including images in forums available to members on Silver membership level or higher. Please upgrade to take full advantage.
The post about what happens when you die got me to thinking about my backup strategy. I just got a QNAP TS-451 to help with reliable backup and file synch of my photos for my PCs and mobile devices, and share with family and freinds. I set up the QNAP with three WD 4TB reds (for now) in a thick volume using RAID 5. I am not sure how to go about this workflow:
Download photos from Camera to main PC then copy the Raw files to the TS-451 - these become the untouched originals
Then copy the originals on the PC to another folder or directory and those become the working copies
Synch the working copies with the QNAP and other PCs and devices
I will then import the files to lightroom on my main PC, use photoshop and other editors, all in the working copy folder
Synch the lightroom catalog, and other working copies with the QNAP and other devices. In this way, edits made on one PC don't need to copied or exported to be able to work on them from the other PCs?
Back up the QNAP originals and working copies to an external drive
Back it up to a portable drive for offsite storage.
I know it's a lot but hope you can help with which apps to use for which steps so that it is as automated as possible.
I'm not sure if I'm over complicating but I am getting tired of copying files from one Device to the other, and having to move or export edits from one device to another. There are three PCs in the equation that I would like to keep in synch.