STEM with Python – Part 12: Automating Data Compression on Linux

Automate everything!!

Linux’s near-perfect compatibility with Linux is simply awesome! Just like you write code in the shell script, you can do the same with Python – and it’s an even more fun experience than the other way around. Since Pythion is hot at this moment, why don’t you also start learning it if you haven’t yet?

Since I spent a lot of time posting about Kotlin for this week, let’s take a look at Python. Along with my personal Kotlin learning journey, I still always have to interact with my home Linux server for controlling data flow and doing some backups of my personal files.

As I mentioned in a last year’s post, my home Linux server became my Dropbox alternative and it’s even better, I believe. But still, one of its biggest risks of it is its redundancy – it doesn’t have any backups (for now as of this writing).

Code:

So, for my temporary solution, I wrote the below Python code to compress the whole directories and files inside the attached 5TB storage and save it into the server’s other storage. For now, only 5% of the storage is used, which is about 185GB.

Also, since the storage works as a samba server, it’s the center of my commuting life that allows me to save files from multiple different devices – it’s my true media hub.

import os
import subprocess

# Create a log file path in the specified directory
log_dir = '/path/to/seagate_hdd_100gb_home/archiveDir/logs'
log_file = os.path.join(log_dir, 'log5tbTarXz.log')

# Open the log file in append mode
with open(log_file, 'a') as f:
    # Run the command and capture the output
    time = subprocess.run(["date", "+%s"], capture_output=True, text=True).stdout.strip()
    save_dir = '/path/to/seagate_hdd_100gb_home/archiveDir/archives'
    tar_file = os.path.join(save_dir, "data_" + time + ".tar.xz")
    output = subprocess.run(["tar", "-Jcvf", tar_file, "/path/to/seagate_hdd_5tb"], capture_output=True, text=True)

    # Write the command's output to the log file
    f.write(output.stdout)

Crontab:

Additionally, I took advantage of Linux’s crontab. Just by typing “crontab -e”, you can open it in your choice of file editor, including nano or vim. For the record, I chose vim – I mean, seriously, who uses nano?

crontab -e

And this is the line I inserted into crontab that executes the python code by specifying its path. So, basically;y what it means is that the machine will execute the python code every day at 1 am.

0 1 * * * /usr/bin/python3 /path/to/seagate_hdd_5tb/Python/linux_automation/zipAuto5tb.py

Afterthoughts:

What is great about Linux is its near-perfect compatibility with programming languages, including Python. I mean, what other OS can offer us this high level of flexibility and compatibility with programming languages? For example, in this case, I wrote a code in Python that automates a certain computing action, and by inserting it into crontab. the Linux machine will automate it – I mean, this is AWESOME!! And this is a great way to improve your scripting skill on a daily bases, isn’t it?

No Linux, No Life!

Long live, Linux!!

Leave a Reply