What solution to choose for online backup of 10-20 TB of data?

Amitz

My dearest LESbians,

first of all: Happy New Year to you all! May it be filled with much health, a truckload of wonderful moments and finally deep understanding for the human race!

I have a server that needs to be backed up once a week. Preferably via rsync. The problem is: we are talking about 10 TB of data at the moment, which could swell to 18 TB over the coming year. Access is only once a week, and a higher volume of data traffic only occurs during the first backup and in the event of a backup restore, which hopefully won't happen. But we all know that the devil is a squirrel.

I am no longer up to date because I have hardly devoted myself to the "IT hobby" for a good 2 years.

What's a smart bunny doing these days when he has to solve my "problem" above? I know that this place is teeming with smart bunnies who not only smell good but are also very helpful. So I'm really looking forward to your hints and tips.

Thank you very much in advance and all the best,
Amitz

Ympker

Happy New Year, caramba! It's good to see you

I'm sure, there will be many suggestions for some sort of server setup or Dropbox/GDrive Unlimited soon, so I'll just throw in two alternatives.

Koofr has served me well the past year. Based in Europe, supports rclone, uses Hetzner servers (iirc) and costs about 60€/mo for 10TB, also supports 20TB plan.

If you prefer lifetime/one time payment, pCloud supposedly also supports rclone and costs 1190€ one time for 10TB. Given that Koofr comes to about 720€/y at 10TB, pCloud already covers that after the second year. Lifetime storage stacks, unlike Koofr.

https://www.pcloud.com/cloud-storage-pricing-plans.html?period=lifetime

Nekki

If you don’t really give a crap about speed but just want the stuff backed up, how about Backblaze B2 + Rclone? $7/month for unlimited data?

Edit: ignore that, B2 is $1000/year for 18TB. It’s the computer backup that’s $7/month.

Amitz

Thank you, @Ympker & @Nekki! I will take a deeper look!

I was previously looking at a Hetzner Storage Box:
https://www.hetzner.com/storage/storage-box/bx31

I could start with 10 TB first and do a later upgrade to 20 TB if necessary. Seems less expensive to me - Do you maybe also have some experience with those "Storage Boxes"?

Ympker

@Amitz said:
Thank you, @Ympker! I will take a deeper look!

I was previously looking at a Hetzner Storage Box:
https://www.hetzner.com/storage/storage-box/bx31

I could start with 10 TB first and do a later upgrade to 20 TB if necessary. Seems less expensive to me - Do you maybe also have some experience with those "Storage Boxes"?

No worries Hetzner storage boxes are definitely cheaper and probably also a good choice. Haven't used them but no doubt it's good stuff. I would prefer pCloud/Koofr because I already use them and they might have more backup replications. pCloud and Koofr have 30 day file versioning , pCloud can also be upgraded to have 365 day file versioning.

There's also pCloud Business Pro with unlimited storage and several users that you can assign storage to starting at 16€/mo as far as I can tell: https://www.pcloud.com/business

It says 3+ users, though. So if Business Pro means you have to pay for each user and 3 is Minimum then perhaps it comes to 3x16€/mo (48€/mo). I'd reach out to them in that case. Still, given that you might be scaling "unlimited" rclone storage doesn't sound bad. 180 days file versioning in that case.

Falzo

@Amitz said: I have a server that needs to be backed up once a week. Preferably via rsync.

@Amitz said: Do you maybe also have some experience with those "Storage Boxes"?

storageboxes are awesome value for the money and quite speedy, especially when used over the internal network.

BUT. while you can access the storageboxes via ssh/rsync you only have permissions of your single unprivileged user. hence you sadly can't backup a full server via rsync and also keep uids and permissions.
you could create an image on the storagebox and mount that to rsync into it, however this will complicate it a bit.

that said you probably might run into permission issues with regular cloud solutions as well, at least if keeping/archiving that is of any concern here.

erk

@Nekki said:
If you don’t really give a crap about speed but just want the stuff backed up, how about Backblaze B2 + Rclone? $7/month for unlimited data?

Edit: ignore that, B2 is $1000/year for 18TB. It’s the computer backup that’s $7/month.

Backblaze (Computer) would be economical if you run a Windows VM and use 'virtio-fs' to map the host drive into the VM - although the shipped hard drive recovery was at 8TB last I looked.

deank

Speaking of which (kinda off topic still), has anyone seen M.2 NAS?

yoursunny

I hear Deep Atlantic Storage is prem.
1000PB uploaded and counting.

deank

@yoursunny said:
1000PB uploaded and counting.

I somehow read that as 1000PB flooded and counting.

Silly me.

ehab

Amitz ....... good to know your backing you

best wishes to you too.

all i can say there was a provider that sends you an ssd so you copy into your data and then send them back, after few days you can see all your data in buckets.... Maybe you can ask if the selected provider can support you with such a solution.

good luck and lots of sex.

Amitz

Beloved LESbians,

since the server to be backed up is at Hetzner and the matching StorageBox was not too expensive, I rented an instance yesterday evening. 6 TB have already found their way there. All night at full 1 Gbps transfer rate, currently at 300-500 Mbps throughput. That is fast enough for me. Everything should be ready this evening. It's been so easy and smooth now that I'll probably just stick with it. It's not data that I need IMMEDIATELY in an emergency. Whether restoring it takes a day or a week doesn't matter at all and the pricing is okay.

@Falzo said:
BUT. while you can access the storageboxes via ssh/rsync you only have permissions of your single unprivileged user. hence you sadly can't backup a full server via rsync and also keep uids and permissions.
you could create an image on the storagebox and mount that to rsync into it, however this will complicate it a bit.

Fortunately, this is not an issue here. I only backup raw data, uids and permissions are not important and can get easily set during restore. The StorageBox seems to fit my needs quite well!

Thank you all for your support, have a great day and feel hugged,
Amitz

root

Problems with UID? Use a different partition type for storage section (FAT, NTFS, etc).

Flying_Chinaman

Google drive for education?

seenu

@Flying_Chinaman said:
Google drive for education?

now Google limited space/quota per org

SnusBoks

I use google workspace, a few euros for unlimited storage.

Calin

Hello , it s possible give for you VPS with next specifications

20 TB Storage
2 Dedicated CPU
8 GB ram memory
64 TB BW/month
Romania location
Full root access

For 60 euros/month

user123

Netdynamics24's New Year promo on OGF has/had 10TB of SFTP storage with shared 1Gbps and unmetered traffic for $300/3yr in several locations that all support rsync.

Nekki

@Calin said:
Hello , it s possible give for you VPS with next specifications

20 TB Storage
2 Dedicated CPU
8 GB ram memory
64 TB BW/month
Romania location
Full root access

For 60 euros/month

Do it for 10.

user123

@Nekki said:

@Calin said:
Hello , it s possible give for you VPS with next specifications

20 TB Storage
2 Dedicated CPU
8 GB ram memory
64 TB BW/month
Romania location
Full root access

For 60 euros/month

Do it for 10.

If he can do those specs with 20TB for 60EUR/month, I'm sure he can do it with 10TB for the same price.

rajprakash

@Nekki said:

@Calin said:
Hello , it s possible give for you VPS with next specifications

20 TB Storage
2 Dedicated CPU
8 GB ram memory
64 TB BW/month
Romania location
Full root access

For 60 euros/month

Do it for 10.

FTFY. Do it for $10

jarland

@Amitz said:
Thank you, @Ympker & @Nekki! I will take a deeper look!

I was previously looking at a Hetzner Storage Box:
https://www.hetzner.com/storage/storage-box/bx31

I could start with 10 TB first and do a later upgrade to 20 TB if necessary. Seems less expensive to me - Do you maybe also have some experience with those "Storage Boxes"?

I love the Hetzner storage boxes. It was only recently that I learned you can put SSH keys on them: https://docs.hetzner.com/robot/storage-box/backup-space-ssh-keys/

Great performance, excellent storage to price ratio, no having to fuck around with the details any more than you want to.