Messing with backups

Lately, I’ve been exploring the tar command and tinkering with backups. I don’t know why I’ve never looked into it before. I’ve been using rsync to create snapshots of my home directory on my “production” machines. While technically, not a back up, these snapshots have been useful. I’ve also use rsync to copy certain directories to computers across my network. Over the years, I’ve also written scripts that use zip to create compressed archives of my script directory along with some other directories.

About a month ago I created a few scripts to make, daily, weekly, and monthly snapshots of my local repositories using tar. That’s been working well, and it prompted me to look into setting up incremental and differential backups using tar. I found some articles and YouTube videos on the subject to get familiar with the concepts. It wasn’t until I started actually experimenting with it, that it began to gel, and I cobbled together a couple of rudimentary backup scripts. Soon I was able to flesh them out and write scripts for incremental and differential backups.

I created scripts to make incremental backups of my two main repository directories. One contains a local copy of my public repositories that I have on Github, and the other is my private repository that I store on a local server. As of this writing I’ve only done the initial full backup of the repositories, so it will take awhile to see how well it works. I’ll probably still have to deal with a few bugs. Within hours of doing the first backups, I found a couple of minor bugs which I fixed straight away. They didn’t affect anything operation, they were just minor cosmetic issues how the archive files were named.

I have the scripts setup to append the archive name with a six-digit date (yymmdd) followed by the numerical day of week (0-6). A full backup is done on Sunday (day 0) and incrementals are done the next six days. On Sunday the metadata file is renamed using the date for the previous Sunday and a new metadata file is created for the next week. It will be at least a couple of weeks before I know it’s working as expected, but I’m confident I’ve got it right.

If this works out I’m thinking about implementing incremental backups for other important directories and backing up to external drives. Delving into using tar with incremental and differential backups has kind of opened up some new possibilities.