Tag Archives: dreamhost

Quick backups with Fabric and Python

Backups… you know you have to make them, but you would so much rather pay someone else to do it. And you know, what? You can outsource it to someone else and you won’t even have to pay for it. That’s right! Let a Python script do the hard work while you do something far more creative.

If you are using a shared hosting provider like DreamHost you are probably hosting a number of domains on there, because it is cheap, because it is easy, and because it makes sense to test things on an inexpensive server before you commit to a dedicated machine or something even more powerful.

This article is also available on Amazon Kindle. You may consider buying it, if you would like to keep it for your reference.

Quite often, those least expensive server options do not come with backup tools, and even if they do, you might want more control and a local copy, or maybe a remote copy that is done your way. For those times, and for many others, Fabric is the tool to go to. What is Fabric? The official documentation states that:

Fabric is a Python (2.5 or higher) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.

There were a lot of long words in there. We’re naught but humble pirates… What is it then?

From my point of view it is the easiest way to write Python scripts that automate system administration, network administration, and software deployment. Those tasks routinely use ssh, scp, and sudo and are very cumbersome to write in Bash. Writing them in Python results in much cleaner and maintainable code. If you ever tried to put off automating certain tasks because it would be a nightmare to write them in Bash, Fabric will make you very happy indeed.

On the surface it is a pointless replacement for custom shell scripts and makefiles, but when you start using it, you realize that it simplifies just what needs to be simplified and leaves you to do what you want. And that’s what the good tools are supposed to do.

Consider the simple task of making backups of directories on a shared server. If you were to do it by hand, you would use ssh to log into the server, make a note of the directory path (OK, you can skip that step if you already have that information), create a .tar.gz or .zip archive, log out, use the information you gathered to issue an scp command to copy the archive to the local machine.

If you were to write a Bash script to do that for you, you would need to test and re-test things and you’d quickly grow discouraged. Fabric lets you get these tasks done quickly and encourages saving them in a library of recipes known as fabfile (an obvious play of words on the old Unix makefile). It is a Python script that the fab will look for by default in the present working directory. Here’s a example of a Fabric file, a script called fabfile.py that the fab command looks for by default

#!/usr/bin/python2.7
from fabric.api import local
from fabric.api import get
from fabric.api import put
from fabric.api import reboot
from fabric.api import run
from fabric.api import sudo
from fabric.context_managers import cd
import time

def remote_get_archived_dir(da):
    """Make a tar.gz archive of da (a directory under ~/)."""
    o = run("cd ~; tar -zcf ~/%s.tar.gz %s" % (da, da))
    ds = "%s_backup_%s" % (da, time.strftime("%Y%m%d-%H%M%S", time.gmtime()))
    sl = local("mkdir %s" % ds)
    sg = get("~/%s.tar.gz" % (da), ds)

You can use it to backup any directory under ~/ in the present working directory on a local machine using the following command:

$ fab -u username -p password -H hostname remote_get_archived_dir:da

For example, if the directory you wanted to archive and make a local copy of that archive was called force located on a host called luke.example.com, you’d use the following command:

$ fab -u hansolo -p pssst -H luke.example.com remote_get_archived_dir:force

And since each fabfile is a Python script, you can use all of the power of Python in such scripts.

Fabric is not a part of the Python distribution, you need to install it using the following command:

$ pip install fabric

Once you have done that, see for yourself how easy it is to do the stuff you had to do by hand.

PS. If you want to learn Python, have a look at these Python programming books.