backy2 is a deduplicating block based backup software which encrypts and compresses by default.
The primary usecases for backy are:
rbd diff, backy2 will only read the changed blocks since the last backup. We have virtual machines with 600GB backed up in about 30 seconds with <70MB/s bandwidth.
With a very small set of commands, good
--help and intuitive usage,
backy2 feels mostly like
cp. And that's intentional, because we think,
a restore must be fool-proof and succeed even if you're woken up at 3am
and are drunk.
And it must be hard for you to do stupid things. For example, existing
files or rbd volumes will not be overwritten unless you
deletion of young backups will fail per default.
Every backed up block keeps a checksum with it. When backy scrubs the backup, it reads the block from the backup target storage, calculates it's checksum and compares it to the stored checksum (and size). If the checksum differs, it's most likely that there was an error when storing or reading the block, or by bitrod on the backup target storage.
Then, the block and the backups it belongs to, are marked 'invalid' and the
block will be re-read for the next backup version even if
rbd diff indicates
that it hasn't been changed.
Scrubbing can also take a percentage value for how many blocks of the backup it should scrub. So you can statistically scrub 16% each day and have a full scrub each week (16*7 > 100).
Even invalid backups can be restored!
As backy2 is a long-running process, you will of course not want to wait until something has finished. So there are very few places in backy where a global lock will be applied (especially on a very rarely used full cleanup which you can kill at any time to release the lock).
So you can scrub, backup and restore (multiple times each) on the same machine.
backy.cfg. We have seen ~16GB of RAM usage with large configured queues for 200TB images and a backup performance of 350MB/s to an external s3 storage.
backy2 brings it's own fuse service. So a simple linux command makes backups directly mountable - even on another machine:[email protected]:~# backy2 fuse /mnt
And on another terminal:
[email protected]:~# ls -la /mnt/by_version drwx------ 0 root root 0 Mai 3 16:14 0c44841a-8d47-11ea-8b2d-3dc6919c2aca drwx------ 0 root root 0 Mai 3 16:14 60ae794e-8d46-11ea-8b2d-3dc6919c2aca drwx------ 0 root root 0 Mai 3 16:14 9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca [email protected]:~# ls -la /mnt/by_version_uid/9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca -rw------- 1 root root 280M Mai 3 14:01 data -rw------- 1 root root 0 Mai 3 14:01 expire -rw------- 1 root root 9 Mai 3 14:01 name -rw------- 1 root root 0 Mai 3 14:01 snapshot_name -rw------- 1 root root 51 Mai 3 14:01 tags -rw------- 1 root root 5 Mai 3 14:01 valid [email protected]:~# cat /mnt/by_version_uid/9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca/name sometest1 [email protected]:~# mount /mnt/by_version_uid/9d8cfe80-8d46-11ea-8b2d-3dc6919c2aca/data /mnt You get the idea. The data file (and resulting partitions, mounts) read/write! Writing to them will write to a temporary local file. The original backup version is *not* modified! This means, you may even boot a VM from this file from a remote backup.
You can tag backups with your own tags depending on your usecase. However, backy2 also tags automatically with these tags:
b_daily b_weekly b_monthly
It has a clever algorithm to detect how long the backup for any given image and this tag is ago and then tags again with the given tag. So you'll see a b_weekly every 7 days (if you keep these backups).
By providing a config-value for how old backups need to be in order to be able to delete them, you can't accidentially delete very young backups.
backy protect you can protect versions from being deleted.
This is very important when you need to restore a version which is suspect
to be deleted within the next hours. During restore a lock will prevent
deletion, however, by protecting it, it cannot be deleted until you decide
that it's not needed anymore.
Also, you'll need
--force to overwrite existing files or volumes.