Disks swaps in kid2, btrfs continued...

Disk (end of) life

Kid2 is happily running the urBackup system for 3 windows laptops but doesn't really have the disk space required to continue the lifesize test (i.e. backuping my most disk-intensive Linux systems). So I ordered a 2TB drive from Amazon. Got i/o errors then sent it back. Still waiting confirmation for the refund, not a terribly good experience so far. I recycled two 2.5" hard disks, one from an older laptop and another from a thrift store, so I could continue running urBackup in the meantime. Then I got i/o errors from the PATA drives (both of them, the 6.4GB from an older desktop and also the 200GB from the thrift store). Long story short, I did order another 2TB drive from eBay this time, it arrived yesterday so I mounted and tested it. I removed the PATA drives and currently the 300GB is still in the box but not connected due to lack of SATA power cables. Benchmarked the new drive, seems OK:

Add to btrfs

Like so:

sudo btrfs device add -f /dev/sdd /var/backup/urBackup

Then:

 sudo btrfs filesystem show
Label: none  uuid: 4881245e-bed5-4526-b9ff-3b5ba39b283d
    Total devices 3 FS bytes used 145.29GiB
    devid    1 size 465.76GiB used 74.03GiB path /dev/sdc1
    devid    3 size 596.17GiB used 74.03GiB path /dev/sda1
    devid    4 size 1.82TiB used 0.00B path /dev/sdd

Make some use of the new available space:

sudo btrfs balance start /var/backup/urBackup

 WARNING:

    Full balance without filters requested. This operation is very
    intense and takes potentially very long. It is recommended to
    use the balance filters to narrow down the scope of balance.
    Use 'btrfs balance start --full-balance' option to skip this
    warning. The operation will start in 10 seconds.
    Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting balance without any filters.

....after...a...long...while...

Done, had to relocate 75 out of 75 chunks

Label: none  uuid: 4881245e-bed5-4526-b9ff-3b5ba39b283d
    Total devices 3 FS bytes used 148.21GiB
    devid    1 size 465.76GiB used 50.00GiB path /dev/sdc1
    devid    3 size 596.17GiB used 51.03GiB path /dev/sda1
    devid    4 size 1.82TiB used 51.03GiB path /dev/sdd

Add a new Linux client

Host run12l is a good candidate now that we have more space available.  It has a 300GB HDSSD quite filled up with photos and videos.  It used to have a client already installed, so it was just a matter of reactivating it and defining it in the "Unix" group so that the /home directory becomes the default backup directory instead of the C:\Users windows default.  First full backup started, looks good.

Starting use of btrfs

So I had this "new" 200GB PATA disk to play with.  I started with some reading from howtoforge and created my first btrfs partition on it.  Initially I formatted the entire disk (/dev/sda) like this:

mkfs.btrfs /dev/sda

Then I changed my mind and recreated a partition table and a primary partition just for clarity.  It doesn't seem to help much but it also doesn't seem to be a problem and I reformatted using:

mkfs.btrfs /dev/sda1

So far so good.   186GB available.  I updated /etc/fstab to use the new btrfs partition for urBackup data and activating lzo compression on it:

UUID=5b775601-11bb-42d9-8b43-e106fb288c8e /var/backup/urBackup    btrfs compress=lzo 0 2

I moved the contents of my 300GB ext4 partition there after stopping the urBackup service:

sudo service urbackupsrv stop

Then restarted urBackup and checked it was running fine.  However I had to first change the permissions because urBackup runs as a normal user:

sudo chown urbackup:urbackup /var/backup/urBackup/

At this point the urBackup service seems to be happy from its new home:

sudo service urbackupsrv status
urbackupsrv.service - LSB: Server for doing backups
   Loaded: loaded (/etc/init.d/urbackupsrv; generated; vendor preset: enabled)
   Active: active (running) since Sat 2018-01-20 12:15:56 CET; 14min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1131 ExecStart=/etc/init.d/urbackupsrv start (code=exited, status=0/S
    Tasks: 25 (limit: 4915)
   CGroup: /system.slice/urbackupsrv.service
           └─1301 /usr/bin/urbackupsrv run --config /etc/default/urbackupsrv --d

jan 20 12:15:50 kid2.lan systemd[1]: Starting LSB: Server for doing backups...
jan 20 12:15:56 kid2.lan systemd[1]: Started LSB: Server for doing backups.
jan 20 12:16:51 kid2.lan urbackupsrv[1301]: Login successful for admin from 192.
jan 20 12:19:52 kid2.lan urbackupsrv[1301]: Login successful for admin from 127.

From there I recycled my 300GB SATA hard disk and added it to the btrfs pool, thus making it a multivolume filesystem:

sudo btrfs device add -f /dev/sdb1 /var/backup/urBackup

 Pretty happy at this  point.  Even more available space:

sudo btrfs filesystem show

Label: none  uuid: 5b775601-11bb-42d9-8b43-e106fb288c8e
    Total devices 2 FS bytes used 43.80GiB
    devid    1 size 186.31GiB used 2.01GiB path /dev/sda1
    devid    2 size 298.09GiB used 43.00GiB path /dev/sdb1

Or else:

df -h /dev/sda1
Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
/dev/sda1          485G     44G  440G  10% /var/backup/urBackup

The space above is shown after I had let urBackup run and soon after I had my backups done from the two Windows clients that are defined.  Can't wait now to add a third, bigger hard disk to this filesystem...

urBackup episode 3

• Windows backup ended fine.  Other incremental backups have followed during the few times I boot this laptop under Windows 10 instead of Linux.  So far so good.  I've also tested a restore (one JPEG file), that went fine as well.
• I need to do something to save the config of the server itself I guess.  BackupPC provided the localhost host by default to do that, but the urBackup server doesn't have a default client on the server itself apparently; will check if I can install a Linux client on the same box and make that happen.
• The rtcwake script seems to work fine on kid2 too.  I enhanced it a bit to check for loadavg because it should not shutdown while the load average is anywhere above idle; so i don't need to do specific checks about urBackup processes and so forth
• One day I'll have to look at the email configuration too.  That's definitely broken by default for a home network
• The kid2 hardware looks like a good idea to start migrating to urBackup while not removing BackupPC immediately.  The only problem is I will need one more "big" hard disk before I can add any more clients to urBackup.

urBackup episode 2

I need more space for urBackup and no space for /home (I want to get rid of this home directory anyway). So unmounted /home, remounted it under /mnt, tried to move everything on the /home (in root filesystem), noticed that there wasn't enough room there because of all the cache and trash in there, did some cleanup then moved the rest... ok, I also moved the urBackup data on this 300Gb hard disk. Then changed /etc/fstab, remounted the disk under it's new location, restarted the urbackupsrv service... got some permission denied... changed the owner:group on the mount point, restarted the service again... that appears to have worked. I left the backup for my test Windows client running. It's quite big (lots of photos and videos) but hopefully there will be enough room this time.

urBackup proof of concept

So far I had not tested this solution because it wasn't available by default in the Ubuntu repository. However after my problems with RsyncD on Windows 10, I figured out that maybe another backup solution should be (at least) investigated, just in case backuppc wouldn't do well with Windows backups. I didn't want to go back to SMB backups if I could avoid it. urBackup looks like a nice, open source, polished solution. It is client-based (e.g. a client package has to be installed on each machine to be backed up) but at least it seems to support Windows very well (and I still have to test the Linux client which is essential for my needs). It promises to support backup compression and de-duplication although it is not clear what level of support (at least for compression it relies on the underlying OS and filesystem. Ext4 is not doing that AFAIK). So after adding a PPA, I was able to install a server for testing on kid2. The server configuration was pretty easy (email won't work but that's not urBackup's fault). I downloaded a client on run12l and set it up real quick, then the first backup started. I had to stop it before it could complete, will need to come back to it later.