How long does it take to backup your VPS?

Not_Oles

Hello!

I just backed up a tiny LXC VPS which runs an email Message Transfer Agent (MTA). I imagined at least a couple of you guys might be interested in how long the backup job took.

The VPS was stopped and offline during the backup, which isn't so good, but at least being stopped prevents files on the VPS from changing into a possibly inconsistent state during the time the VPS is being copied.

When I checked the log, I thought, "Whoa! Only six seconds to shut down the VPS, make the backup, and restart the VPS!! ? It's not a big VPS, but, still, that's fast!"

The six seconds includes just shutdown, backup on the node, and reboot. The time to copy the backup to a remote location wasn't counted.

Here's the log:

INFO: starting new backup job: vzdump XXX --mode stop --node Proxmox-VE --storage local --remove 0 --compress zstd
INFO: Starting Backup of VM XXX (lxc)
INFO: Backup started at 2021-04-02 02:21:35
INFO: status = running
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: CT Name: xxx.example.com
INFO: including mount point rootfs ('/') in backup
INFO: stopping vm
INFO: creating vzdump archive '/var/lib/vz/dump/vzdump-lxc-XXX-XX_XX_XX.tar.zst'
INFO: Total bytes written: 871854080 (832MiB, 208MiB/s)
INFO: archive file size: 299MB
INFO: restarting vm
INFO: guest is online again after 6 seconds
INFO: Finished Backup of VM XXX (00:00:06)
INFO: Backup finished at 2021-04-02 02:21:41
INFO: Backup job finished successfully
TASK OK

Maybe now might be a great time to backup your VPS? ? Please let me know how long your backup takes. . . .

Best wishes and friendly greetings from Mexico! ??????️

yoursunny

Backups are for the paranoid.
I use my disaster recovery plan instead.

Not_Oles

@yoursunny Wow! Great story! Thanks for posting it! ??

Not_Oles

I checked on my server for the backup time of a larger VPS.

For a VPS running a remote desktop, several web browsers, and several IDEs, the log says:

INFO: guest is online again after 75 seconds

Bearing in mind that the 75 seconds includes shutdown, making the backup, and rebooting, 75 seconds seems pretty fast.

How long does it take you to back up your VPS?

deank

666 seconds.

mikho

Backups?

I do like Linus, upload my files to a FTP and let the Internet mirror it.

foxone

Here's my log:

zxrlha

I only backup some logs, and in fact those files are unimportant so I do this online.
It is around 35 seconds because it is a remote backup.
If my vps is down, umm, I cannot do anything other then wait now.
On the other hand, I won't lose anything important even my vps is down, so I can wait.

Lee

Still a big gap in the market for a good automated set and forget backup provider that does not charge the earth.

JetBackup for Linux is a really good option price & feature wise but CentOS 7 only.
Ottomatik.io is amazing but stupid expensive even when you bring your own storage.

Not_Oles

@foxone said:
Here's my log:

What a tantalizing tidbit! Want to share a bit about the software / procedure you are using for the backups, please?

Not_Oles

@Lee said: Still a big gap in the market for a good automated set and forget backup provider that does not charge the earth.

My former neighbor across my former street had a full time job as merely one member of the team doing backups for a Big Corp whose name we'd all recognize. Wondering how much that team cost. . . .

I remember once reading a detailed paper by an engineer at Sun arguing that backups had to be made with dd because tar made too many errors. A couple of years ago I went looking for that paper and couldn't find it. Well, tar always has seemed to work okay for me. Maybe I should have had a backup of the paper!

I don't know whether my neighbor's team used dd or whether they tested restoring. I guess probably they did at least test restoring. One would have to, right?

I've tested restoring from the Proxmox-made backups on my server. In a few dozen tests, restore seems to work okay with basic sanity checks like looking around and just using the restored system for a couple of weeks. I haven't yet tried anything like systematic hashing and confirming all the restored files. I think @yoursunny gets those systematic checks automagically from keeping his files in Github.

Greetings from Mexico!

Not_Oles

@zxrlha said:
I only backup some logs, and in fact those files are unimportant so I do this online.
It is around 35 seconds because it is a remote backup.
If my vps is down, umm, I cannot do anything other then wait now.
On the other hand, I won't lose anything important even my vps is down, so I can wait.

How do you back up your important stuff?

tetech

5 seconds including gzipping the container.

# start=`date +%s` ; ./package.sh proxy ; fin=`date +%s` ; echo "Elapsed: $((${fin} - ${start}))"
Create a snapshot of '/var/lib/lxc/proxy' in '/var/lib/lxc/proxy.tmpl'
At subvol /var/lib/lxc/proxy.tmpl
Delete subvolume (no-commit): '/var/lib/lxc/proxy.tmpl'
Elapsed: 5
# cat package.sh
#!/bin/sh

CNAME=$1

if [ "${CNAME}" == "" ]
then
  echo "Must specify container name as a parameter."
  exit
fi

btrfs sub snap /var/lib/lxc/${CNAME} /var/lib/lxc/${CNAME}.tmpl
btrfs property set -ts /var/lib/lxc/${CNAME}.tmpl ro true
btrfs send /var/lib/lxc/${CNAME}.tmpl | gzip -c > /var/lib/lxc/${CNAME}.gz
btrfs property set -ts /var/lib/lxc/${CNAME}.tmpl ro false
btrfs sub del /var/lib/lxc/${CNAME}.tmpl

JetApps

@Lee said:
Still a big gap in the market for a good automated set and forget backup provider that does not charge the earth.

JetBackup for Linux is a really good option price & feature wise but CentOS 7 only.
Ottomatik.io is amazing but stupid expensive even when you bring your own storage.

Hello,

Although it is true that JetBackup for Linux only supports CentOS 7 installations, I would like to note that JetBackup 5 for Linux supports installation on both CentOS 7 and CentOS 8 systems. This version also utilizes the brand new JetBackup 5 core which has proven to be our fastest, most versatile, secure, and reliable version yet.

Thank you for your support.

Lee

@JetApps said: nstallation on both CentOS 7 and CentOS 8 systems

This is good but the reality is that CentOS is commonly chosen if you are using a control panel. You are now supporting CentOS 8 as you had to given you are so tied to cPanel.

People in the main use Ubuntu/Debian without panels on their VPS. I know many that would use Jetbackup for Linux but OS is the deal-breaker. I mean I would have purchased 15 licences a few months back if it had supported Debian. I had to go elsewhere.

And let's face it, with recent events CentOS is going to be even less popular going forward.

zxrlha

@Not_Oles said:

@zxrlha said:
I only backup some logs, and in fact those files are unimportant so I do this online.
It is around 35 seconds because it is a remote backup.
If my vps is down, umm, I cannot do anything other then wait now.
On the other hand, I won't lose anything important even my vps is down, so I can wait.

How do you back up your important stuff?

Most of my important stuffs are code, and they are on my laptop and synchronized with github.
I also use portable disks to backup some other stuffs on my laptop.

foxone

@Not_Oles said:

@foxone said:
Here's my log:

What a tantalizing tidbit! Want to share a bit about the software / procedure you are using for the backups, please?

Hello!
That's just my proxmox hypervisor syncing backups to the proxmox backup server. The hypervisor is a Ryzen physical server, while the backup server is a small QNAP from 2011 (i think it only has 1g of ram?). They are connected together directly so to not pass through the network, and while that screen is referred to the hypervisor only, there's plenty of clients in the office using pbs client directly.

We also use Duplicati to backup Windows machines.

KidRock

I am just learning to use vps. Installed Centos 8 and configured updates, iptables. Now before installing a control panel and litespeed, I want to take backup of the system and all the configurations and download the backup file to my pc so that if anything goes wrong when installing control panel, I could just upload backup and install centos, firewall and all configurations at once. How do I backup and download for this purpose?

Not_Oles

@KidRock said:
I am just learning to use vps. Installed Centos 8 and configured updates, iptables. Now before installing a control panel and litespeed, I want to take backup of the system and all the configurations and download the backup file to my pc so that if anything goes wrong when installing control panel, I could just upload backup and install centos, firewall and all configurations at once. How do I backup and download for this purpose?

I imagine people might need a bit more information in order to answer your question.

Whatever tools you used from your VPS provider to do the Centos 8 install also might have an option for the backup and restore.

But, without knowing what tools your provider is offering, it's not so easy for us to help you.

Maybe your best option might be to ask your provider how to use their system for the backup and restore?

On Proxmox, one can use the Backup now and Restore buttons:

I can copy the backup file from server local storage to inside your VPS, where you can download it. Uploading a backup and restoring from it can be managed similarly.

Proxmox also supports various other sophisticated automagic backup options.

Greetings from Mexico!

KidRock

@Not_Oles said:
I imagine people might need a bit more information in order to answer your question.

Whatever tools you used from your VPS provider to do the Centos 8 install also might have an option for the backup and restore.

On Proxmox, one can use the Backup now and Restore buttons:

I just checked my VPS dashboard and there is no option for backup/snapshot. Dashboard has the options to install OS and Control Panels like cPanel, Plesk.
I installed Centos 8 from the dashboard and I will install cPanel next. I guess I will be able to take backup from cPanel later. But before installing cPanel, I was just trying to take backup of the system if possible using command line or GUI.
Can I install Proxmox on my VPS and take backup? If yes, any tutorial link?

flips

@KidRock
Full block level/image/snapshot backups of a VPS would have to be done outside the VM, so if the provider does not offer it, you have to look into file level backup.
Which means you backup all the essential files from within the running system and copy off-site. Restore would then have to be something like: Reinstall OS from panel/ISO, then restore files inside the running VPS.
(If you have databases, there's a few special scripts/routines/considerations.)
Full tutorials on VPS backup probably belong in a different thread. (I this there was an article or two in the LES blog a while back.)

Not_Oles

@KidRock Sorry, but, no, installing Proxmox inside your VM will not get you what you want. This is because:

@flips said: Full block level/image/snapshot backups of a VPS would have to be done outside the VM

If our provider offers Rescue Mode, we might get "outside" our VM by booting in Rescue Mode.

Besides doing backups, I've also seen running qemu-kvm within Rescue Mode used to install provider-unsupported OSes such as OpenBSD and Plan 9.

Maybe we also might be able to get outside our VM from within our VM! 😃 We could boot our own OS image by running qemu-kvm inside our VM, perhaps on a memory based file system that we create. 😃

There is a great LES tutorial by Anthony Smith, Migrate a KVM VPS from one host to another - Easy Mode. For the use case here, maybe we could use dd over ssh as explained in Ant's tutorial to copy our VPS' filesystem to our local backup disk instead of to a second VPS.

@KidRock says he is a beginner. This stuff might be somewhat advanced for a beginner. But perhaps it nevertheless might be at least interesting for a beginner to read.

Friendly greetings from Mexico!

Bochi

Never really thought about how long it takes to be honest...
So I had to check my healthchecks.io reports:

They are "online" backups however, using a hacked together bash script and borg to write the backups to a remote repo (maybe 1GB a day in total).