feat(post): Backup compression

gmi-files/blog/backup-compression.gmi

@@ -0,0 +1,84 @@
+# Backup compression
+
+One of those days I was going to double check some information about the backup tool I use (zBackup) when I notice it was not being updated anymore. It still works fine, but I prefer to use an updated tool to avoid any unpatched bugs. Searching for the new options, I ended up with two new candidates: Borg and Restic.
+
+=> https://github.com/ok-borg/borg Borg
+=> https://github.com/restic/restic Restic
+
+They both are deduplication tools, allow encryption, compression and have a check tool to make sure the backup is working. For what I noticed, Restic is better at connecting to different services and Borg is better with local backups, so I chose the last one.
+
+So far, so good. But my first question was: which compression tool to use? With zBackup I didn't have too many options: either I used LZMA or LZO, so my option was LZMA. But Borg offers not only other tools, but also the compression level and I got myself stuck at this. I couldn't just guess a configuration when I need to backup hundreds of GBs every week. After a quick search, I didn't find any result that would satisfy my inner nerd during vacations: I had to test them! Not all of them, of course. I have other things to do during my vacations, but I wanted to run a few personal tests.
+
+For the test data, I just pick one of my folders. One that was not so big, but big enough and with different files types to have a realistic sample. The raw folder had about 16.5GB. So, I started creating a simple backup using zBackup to have a reference and here are the approximate results:
+
+zBackup (LZMA)
+
+```
+Compact time |  40 min
+   Test time |  10 min
+        Size | 9.4 GB
+```
+
+Having that reference, I wanted to know what a fast-archiving option would be and run it using the auto, lz4 option that would only compact compactable files and, if so, using lz4:
+
+Borg auto-lz4
+
+```
+Compact time | 13 min
+   Test time |  6 min
+        Size | 11 GB
+```
+
+Interesting results: way faster than the original tool but using a little bit more of disk space. Next test: using the auto, zlib 6:
+
+Borg auto-zlib-6
+
+```
+Compact time | 13 min
+   Test time |  5 min
+        Size | 11 GB
+```
+
+Almost the same result as before. So, I've tried to remove the auto option, making Borg to compact every single file anyway:
+
+Borg zlib-6
+
+```
+Compact time | 18 min
+   Test time |  6 min
+        Size | 11 GB
+```
+
+As expected, the compact time were considerably longer, and the size was basically the same. So, my next test would be to increase the compression:
+
+Borg zlib-9
+
+```
+Compact time | 21 min
+   Test time |  7 min
+        Size | 11 GB
+```
+
+Still no big surprises here. I thought I could have a very small reduction of the backup size, but that's ok. Next test: using a different configuration: zstd 9
+
+Borg zstd-9
+
+```
+Compact time |  15 min
+   Test time |   7 min
+        Size | 9.9 GB
+```
+
+Now we are talking! A nice compact time with a smaller backup size. To improve it more, I just ran that again but this time, include once more the auto option:
+
+Borg auto-zstd-9
+
+```
+Compact time | 15 min
+   Test time |  3 min
+        Size | 10 GB
+```
+
+Something I've also tested in a separate batch was the backup time after a simple update in the file structure. I am going to resume this here to not make this post so long, but not compacting unnecessary files had a HUGE impact for the subsequent backups sets, what makes me just stick to the auto configuration at the end.
+
+And this was my final configuration: auto, zstd 9, a way fast result than my previous backup tool loosing just a little bit of compression size, what really made me happy at the end of the day. :)

gmi-files/blog/index.gmi

@@ -1,5 +1,6 @@
 # Bruno Fontes Blog
 
+=> backup-compression.gmi Backup compression
 => how-to-remain-online.gmi How to remain online when the big services crash
 => partitions-systemd.gmi Mounting partitions with systemd
 => linux-shell-shortcuts.gmi Linux Shell Shortcuts