Jump to content

Back-up plan needed for music stored on a NAS...


Recommended Posts

Posted (edited)

@JensA and @TheFlash I'm actually not sure that this kind of backup is for me. 

I've now completed the first backup from my Synology NAS to a WD external hard drive. That took almost 48 hours. I was actually expecting to find files on the external hard drive that I could access directly. This wasn't the case. I now need to use some other Synology app to access and restore them as needed. This is fine unless I decide to change from Synology to some other NAS further down the road. It also makes it impossible for me to loan my hard drive to a friend for him to listen to my music.

Now, I'm doing a 'backup integrity' check. This looks to be taking as long as the two day initial copying of files from NAS to hard drive. Why it takes two days to perform an integrity check on files that it's just copied I don't know! But it's infuriating. In fact, why does everything takes so long to do anything? Two days to transfer 4.3 TB from one disc to another. Really? Is this normal?

Basically all I want to do with all this is simply replicate what is on my NAS onto a separate hard drive without all this backup bullshit palaver.

A simple drag and drop would suit me fine but that doesn't seem to be an option.

Am I being unrealistically harsh?

Any other solutions you can think of?

Apologies, Mike.

Edited by SnapperMike
Link to comment
Share on other sites

1 hour ago, SnapperMike said:

Am I being unrealistically harsh?

Any other solutions you can think of?

Apologies, Mike

Sorry, I was under the impression that your goal was a pure backup.

If you want to copy your music library to an attached USB drive Hyper Backup is not the right tool.

I have used the Synology file manager fr this purpose but there is also another Synology app called USBCopy which might be even better for this task.

USB Copy Description

Quote

USB Copy

USB Copy helps you copy files between your Synology NAS and external USB/SD storage devices. This package comes with multiple copy and file organization strategies to ensure efficient storage usage and easy location of files.

and you will probably choose „Mirroring“

Quote

Create USB/SD Copy Tasks

You can customize each copy task for an external USB/SD storage device, and import/export files between your Synology NAS and the USB/SD device. When the USB/SD device is inserted again for copy, your Synology NAS will recognize the matched device and apply the same copy settings.

  1. Insert an external USB/SD storage device to your Synology NAS.
  2. Click Create (the plus icon) on the upper left corner.
  3. Select the desired task type: Photo/Video Import, Data Import, or Data Export.
  4. Specify basic task information and select a copy mode:
    • Multi-versioned: Each time the task runs, a folder will be created and named with the run time. All source files will be fully copied to the destination.
    • Mirroring: Each time the task runs, all changes made in the source folder will be copied to the destination, making the destination folder a complete mirror copy of the source.
    • Incremental: Each time the task runs, newly added and modified source files will be copied to the destination.
    •  

Jens

Edited by JensA
  • Like 1
Link to comment
Share on other sites

1 hour ago, SnapperMike said:

@JensA and @TheFlash I'm actually not sure that this kind of backup is for me. 

I've now completed the first backup from my Synology NAS to a WD external hard drive. That took almost 48 hours. I was actually expecting to find files on the external hard drive that I could access directly. This wasn't the case. I now need to use some other Synology app to access and restore them as needed. This is fine unless I decide to change from Synology to some other NAS further down the road. It also makes it impossible for me to loan my hard drive to a friend for him to listen to my music.

Now, I'm doing a 'backup integrity' check. This looks to be taking as long as the two day initial copying of files from NAS to hard drive. Why it takes two days to perform an integrity check on files that it's just copied I don't know! But it's infuriating. In fact, why does everything takes so long to do anything? Two days to transfer 4.3 TB from one disc to another. Really? Is this normal?

Basically all I want to do with all this is simply replicate what is on my NAS onto a separate hard drive without all this backup bullshit palaver.

A simple drag and drop would suit me fine but that doesn't seem to be an option.

Am I being unrealistically harsh?

Any other solutions you can think of?

Apologies, Mike.

rsync

Your NAS will be running linux (I use QNAP, but I am sure Synology is the same) and it is the simplest and best tool and does exactly what you want.

Make sure ssh service is enabled in the NAS.

ssh into the NAS using whatever tool you like (command line in windows powershell or mac shell or whatever or use a freeware ssh gui), find the mount mount of your USB drive in the NAS (e.g. it is "/share/external/DEV3301_1" for an external hard drive via USB on my QNAP) then run rsync, the command will be:

rsync -av /path/to/files_on_NAS /path/to/USB

For me it is:

rsync -av /share/Multimedia/FLAC /share/external/DEV3301_1/

I would ALWAYS do a dry run first, replace -av with -avn and it will show you what it is doing before it does it.

rsync -avn /path/to/files_on_NAS /path/to/USB

Note that not using a trailing "/" after FLAC will create a folder called FLAC on the USB, using a trailing "/", i.e. "...FLAC/" will not create a FLAC folder and will sync the contents of FLAC straight into the root of the USB.

Couldn't be easier (assuming you can manage with typing a command), will do exactly what you want, is efficient (much quicker than your backup above - 100 GB via usb3 in maybe five minutes, usb2 will be slower) and in future you can run it again and it will only copy changed files.

If you want it to delete stuff from the USB drive that you have removed from the NAS it would be:

rsync -av --delete /path/to/files_on_NAS /path/to/USB

PS to ssh in to the NAS use:

ssh -l NAS_USERNAME NAS_IP_ADDRESS

e.g. for me: ssh -l solanum 192.168.0.101

PPS to find the external usb path, just type "df" at the command line when you are logged into the NAS before and after plugging the usb in, it should be obvious what has changed.

PPPS don't be scared of the command line if you aren't used to it, as you can see from the above, it is a tiny amount of typing you need.

Edited by Solanum
  • Like 1
  • Upvote 2
Link to comment
Share on other sites

In summary, for me to do subsequent backups:

Run a shell (I use Linux, but windows powershell is fine), then I type:
 

ssh -l solanum 192.168.0.101

(enter password)

rsync -avn --delete /share/Multimedia/FLAC/ /share/external/DEV3301_1/FLAC/

(check nothing I want is being deleted or going to the wrong place)

rsync -av --delete /share/Multimedia/FLAC/ /share/external/DEV3301_1/FLAC/

(wait)

exit

All done!

If you have permissions problems you will need to use "sudo" at the start of the command:

sudo rsync -avn --delete /share/Multimedia/FLAC/ /share/external/DEV3301_1/FLAC/

Also, you don't even have to type all that, you can type the first couple of letters and then hit 'tab' key for autocomplete.

  • Upvote 1
Link to comment
Share on other sites

@SnapperMikeThanks for trialling and sharing your learning, saves me a huge amount of time and the same frustration. I do want a “proper” backup à la Hyper but separately I want my music on a WD drive to take with me to my Kendal man-cave.

@Solanum likewise thanks for this, generous of you to take the time. However this is exactly what I hope to avoid! To me the Synology is (or should be) a consumer device which has apps/tools, either bundled or after-market, which meets such an obvious need. I don’t want to need to understand what the operating system is, to run shells, understand syntax, mistype something and cock up... like Mike I simply want to be able to create an accessible/readable and portable copy of my bought music library. 

When I read up in USBCopy as mentioned by @JensA, it implied it was for copying from a USB drive to NAS and not t’other way. I’ll take a closer look and if it doesn’t do the job I’ll try File Manager.

 Thanks all, great stuff.

Link to comment
Share on other sites

33 minutes ago, TheFlash said:

When I read up in USBCopy as mentioned by @JensA, it implied it was for copying from a USB drive to NAS and not t’other way. I’ll take a closer look and if it doesn’t do the job I’ll try File Manager

@TheFlash In order to create a task for copying FROM USB TO NAS  just click at the plus symbol (+) on the bottom left of the USB copy window to create a new task, then select data export

  • Like 1
Link to comment
Share on other sites

10 hours ago, SnapperMike said:

@JensA and @TheFlash I'm actually not sure that this kind of backup is for me. 

I've now completed the first backup from my Synology NAS to a WD external hard drive. That took almost 48 hours. I was actually expecting to find files on the external hard drive that I could access directly. This wasn't the case. I now need to use some other Synology app to access and restore them as needed. This is fine unless I decide to change from Synology to some other NAS further down the road. It also makes it impossible for me to loan my hard drive to a friend for him to listen to my music.

Basically all I want to do with all this is simply replicate what is on my NAS onto a separate hard drive without all this backup bullshit palaver.

Hi, I think you chose the default backup type of "Local Folder & USB".  This creates a backup stored as a database which can keep multiple versions of files, and wants an integrity check once in a while.

You could try "Local Folder & USB (Single-version)" - this will just copy the files onto the USB drive:

image_2021-04-05_095622.png.7050cddc19ded006544d671538f258cd.png

It will also include a backup of your system configuration (no bad thing) and a separate folder (called @app) which records the original dates/owners/permissions of the backed up files.  You can ignore these, but if you ever restore the backup to a Synology server using HyperBackup, they will be used to re-create the files' dates/owners/permissions as they were.

This, combined with using ExFAT, should make your backup easier to manage and make it accessible from a Windows computer.  If you don't want to pay for the ExFAT package, you can try formatting the USB drive as NTFS in Windows.  Synology can access it but last time I tried that it was very slow - things might have improved since then!

This is the way I back up my music, except I allow the Synology to use it's native EXT4 file system for the fastest backup - there are options for reading that file system in Windows if necessary and those are not dependant on Synology.

  • Like 1
Link to comment
Share on other sites

1 hour ago, JensA said:

@TheFlash In order to create a task for copying FROM USB TO NAS  just click at the plus symbol (+) on the bottom left of the USB copy window to create a new task, then select data export

And from NAS to USB similarly, I presume (which is the way I want to copy, and the way I presume @SnapperMike does too. I can't test this right now, other stuff going on, apols.

Link to comment
Share on other sites

Is there a difference in speed depending on the file type of the USB? What is the hierarchy coming off of either a Synology or QNAP? I've just been dragging and dropping from my QNAP to my Synology across the network. It took several days. I might change this process by just formatting the drive in the Synology and plugging it in to the QNAP as an external USB if it's faster?

Link to comment
Share on other sites

17 hours ago, JensA said:

Sorry, I was under the impression that your goal was a pure backup.

If you want to copy your music library to an attached USB drive Hyper Backup is not the right tool.

I have used the Synology file manager fr this purpose but there is also another Synology app called USBCopy which might be even better for this task.

USB Copy Description

and you will probably choose „Mirroring“

Jens

Great, thank you Jens. I've looked at USBCopy and done a wee test and that is exactly what I want!

I think what I will do is now I have a 4.3 TB Hyper Backup on a WD external hard drive I will keep that but also buy another WD hard drive to do a complete a USBCopy too. That way I have a backup in two different formats on two different devices. Hopefully I shall never need them.

Thanks Jens for your kind help.

Mike.

  • Like 1
Link to comment
Share on other sites

Posted (edited)
16 hours ago, Solanum said:

In summary, for me to do subsequent backups:

Run a shell (I use Linux, but windows powershell is fine), then I type:
 



ssh -l solanum 192.168.0.101

(enter password)

rsync -avn --delete /share/Multimedia/FLAC/ /share/external/DEV3301_1/FLAC/

(check nothing I want is being deleted or going to the wrong place)

rsync -av --delete /share/Multimedia/FLAC/ /share/external/DEV3301_1/FLAC/

(wait)

exit

All done!

If you have permissions problems you will need to use "sudo" at the start of the command:



sudo rsync -avn --delete /share/Multimedia/FLAC/ /share/external/DEV3301_1/FLAC/

Also, you don't even have to type all that, you can type the first couple of letters and then hit 'tab' key for autocomplete.

Thanks @Solanum I really appreciate your kind help. Unfortunately I am not computer literate and to be honest wouldn't know where to begin with inputting code as you describe. For me shells are for either the beach or for fuelling my car. Computers and I have never really seen eye to eye and I don't have the patience nor inclination to learn.

I have however got USBCopy working as suggested by Jens and I'm pleased about that. It seems to be doing exactly as I had wished for.

Many thanks though. Mike.

Edited by SnapperMike
  • Like 3
Link to comment
Share on other sites

9 hours ago, TheFlash said:

When I read up in USBCopy as mentioned by @JensA, it implied it was for copying from a USB drive to NAS and not t’other way. I’ll take a closer look and if it doesn’t do the job I’ll try File Manager.

@TheFlash, it's certainly not as easy as I had initially thought. However, I now have a Hyper Backup of my music done to a WD external hard drive. This data of digital music needs Synology software to read and understand it. That's fine if I have Synology equipment when and if I need to backup sometime in the future. What I'm going to do in addition to this is buy another WD external hard drive to copy my music from the NAS to the WD using the app called USBCopy that @JensA suggests above. I've tested it and it seems to work perfectly in the way I had initially thought backups were performed. I've looked and the Flac files are immediately accessible. In USBCopy I chose 'Data Export' and then 'Mirroring'.

I hope this helps!

Mike.

  • Like 2
Link to comment
Share on other sites

8 hours ago, jarvis said:

Hi, I think you chose the default backup type of "Local Folder & USB".  This creates a backup stored as a database which can keep multiple versions of files, and wants an integrity check once in a while.

You could try "Local Folder & USB (Single-version)" - this will just copy the files onto the USB drive:

image_2021-04-05_095622.png.7050cddc19ded006544d671538f258cd.png

It will also include a backup of your system configuration (no bad thing) and a separate folder (called @app) which records the original dates/owners/permissions of the backed up files.  You can ignore these, but if you ever restore the backup to a Synology server using HyperBackup, they will be used to re-create the files' dates/owners/permissions as they were.

This, combined with using ExFAT, should make your backup easier to manage and make it accessible from a Windows computer.  If you don't want to pay for the ExFAT package, you can try formatting the USB drive as NTFS in Windows.  Synology can access it but last time I tried that it was very slow - things might have improved since then!

This is the way I back up my music, except I allow the Synology to use it's native EXT4 file system for the fastest backup - there are options for reading that file system in Windows if necessary and those are not dependant on Synology.

Many thanks @jarvis much appreciated. I tried to do as you suggested within Hyper Backup but when I selected the 'Local folder & USB (single-version)'  it then said that 'Version Incompaibility found. Please use multi-version tasks'. However, I have tested USBCopy and that seems to work perfectly for my requirements.

Link to comment
Share on other sites

In other news... I managed to snap up today on the 'Bay a WD Mybook Duo... not because I want one but because it contains 2 x 6TB WD Red drive and I've confirmed they are CMR not the poorer performing (and controversial) SMR variants. It is, at the very least, an economical way of buying 2 6TB drives, allowing me to double my NAS storage capacity.

I'm goung to put the current 2 x 3TB WD Reds somewhere useful... this will either be another Synology NAS for the Hyper Backup thing or I'll swap them into the Mybook and do a weekly USBcopy (I won't use RAID 1 so will be able to access the full 6TB). Will keep you posted.

Separately, I'll use USBCopy too to copy my music onto my 3TB WD MyPassport as and when, allowing me to take it to Kendal with me. I've yet to work out how to play it when I get there! I do have a Node 2i which might do the job; the reindexing could take a while but I can live with that.

  • Like 2
Link to comment
Share on other sites

1 hour ago, TheFlash said:

In other news... I managed to snap up today on the 'Bay a WD Mybook Duo... not because I want one but because it contains 2 x 6TB WD Red drive and I've confirmed they are CMR not the poorer performing (and controversial) SMR variants. It is, at the very least, an economical way of buying 2 6TB drives, allowing me to double my NAS storage capacity.

I'm goung to put the current 2 x 3TB WD Reds somewhere useful... this will either be another Synology NAS for the Hyper Backup thing or I'll swap them into the Mybook and do a weekly USBcopy (I won't use RAID 1 so will be able to access the full 6TB). Will keep you posted.

You're very clever and resourceful. Much more than me anyway!

  • Haha 1
Link to comment
Share on other sites

 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...