rsync like backup to Rackspace Cloud File with duplicity

Posted on 2 September, 2009 by chmouel

It seems that there is no much documentation about how to do rsync like backup with duplicty so here it is :

#!/bin/bash
UPLOAD_TO_CONTAINER="backup" #adjust it as you like
export CLOUDFILES_USERNAME=Your Username
export CLOUDFILES_APIKEY=API_KEY_YOU_GOT
export PASSPHRASE=The Passphrase for your encrypted backup
 
duplicity /full/path cf+http://${UPLOAD_TO_CONTAINER}

This should take care to upload the backup files to the backup container. It does that incrementally and detect the changes to your file system to upload. There is much more option for duplicity look at the manpage for more info.

This entry was posted in Rackspace, Scripts. Bookmark the permalink.
  • Tristan

    I’m not sure what I’m doing wrong but I keep receiving syntax errors in my script I used the exact script above with my info filled in. I saved the script as backup.py and run try running with with the following command:

    python backup.py

    but I keep receiving syntax errors.

    • http://www.chmouel.com/ chmouel

      What version of duplicty you have ? you can check it with the command line duplicity -V

      • Tristan

        It says duplicity 0.4.10

        • http://www.chmouel.com/ chmouel

          you probably want to have a newer version than that I got 0.6.05 for me.

  • http://www.chmouel.com/ chmouel

    it’s a shell script so save it as a .sh and run it bash backup.sh for example

    You probably make sure you have to read the manpage as well of duplicity

  • Tristan

    Alright now when I run the shell script I receive the following error:

    Unknown scheme ‘cf+http’

  • http://www.deckerd.com Drew

    Tristan,

    You probably need to update your version of Duplicity. What version do you have?

    duplicity –version

    Let me know.

  • david

    the latest version of duplicity in the latest version of debian (lenny) is 0.4.11

    is there a way to get a more current version that apt can understand?

    • http://www.chmouel.com/ chmouel

      The only way i can see is or to have a custom build package for duplicity or mix debian stable and unstable by pinning the stable distro for everything but duplicity. If you go on irc on freenode.net #cloudfiles some debian gurus would be able to help you.

      There is as well the install from sources.

  • david

    hi there chmouel… how do you install the rackspace cloudfiles once you’ve downloaded the tar? I tried ‘python setup.py’, but that didn’t seem to be the right invocation.

    • http://www.chmouel.com/ chmouel

      like a standard python modules it simple as to do :

      python setup.py install

      python setup.py –help will give you all the option if you want to.

      As always you can find some great ‘live’ help for Rackspace Cloudfiles on the irc.freenode.net #cloudfiles.

  • david

    thx…

  • david

    everything works!

    thanks chmouel.

    first off, the version of duplicity in the lenny repository is something like 0.4.11, and you need something later (0.5.??) to support the ‘cf+http’ backend.

    As chmoel suggested, the #cloudfiles irc channel at freenode was quite helpful… there was not a more recent debian build, and so someone built it for me. This is to soon be worked into the debian backports repo.

    Then you have to get and install the rackspace cloudfiles python bindings; there’s a pointer to them someplace on this blog.

    and the only other thing is, duplicity uses https to talk to rackspace, so if you’re behind a firewall (like I was), you have to open up port 443.

    • http://www.chmouel.com/ chmouel

      cheers for that, I probably going to clarify a bit by a new blog post to explain how to setup of all of this since it seems to create a bit of burden for some people.

  • Pingback: VieuxSteak.ch » Backuper son serveur sur CloudFiles

  • Pingback: 2009 Hosting Summary (or where does Harper host in 2009/2010) » Harper Reed: Tech, Mobile, Yo-yoing and Death Metal

  • Chandraprakash

    Is there any chance duplicity can be installed on cloudsites?
    Any plans of making it available by default? It can be good for customers and good for cloudfiles revenue as this will push the cloudfiles usage to a new level.

  • sam

    Hi. This works flawlessly on my server. Thanks!

    One thing I need to know, is this already make use of the ServiceNet so no bandwidth are charge when I backup my files?

    • http://www.chmouel.com/ chmouel

      Hi Sam,

      It’s not enabled by default you probably want to do a export RACKSPACE_SERVICENET=True before you launch the ftp server to automatically use servicenet.

      Chmouel.

  • david m. chinn

    My firewall bit the dust yesterday after many years, and not having backups, I had to reconfigure stuff.

    Someone mentioned opening port 443 for duplicity to talk to rackspace via https. If you don’t, you get a ‘connection refused’ message.

    The other thing is that DNS (port 53) also has to be opened. This problem will show up before the 443 problem, and the error message will be “no address associated with hostname”

  • mgeary

    i’m trying this with duplicity version 0.6.08b. I don’t see anywhere to manage/set a gpg key, so i’m using –no-encryption. When i do this, i get a “backup statistics” output that appears to say that it copied files, but when i view the container using the rackspace management portal, i don’t see them. Any suggestions?

  • mgeary

    ahh. i see now that duplicity isn’t really designed to put web-usable files onto rackspace, but rather just to use rackspace as an offsite place for your backup. Is there a way to have it just copy the files, and not create an archive?

  • Rdc

    I’m trying to use the screipt above with openstack without any result. Is it possible without modify cloudfiles backend? Variable UPLOAD_TO_CONTAINER is just container name or I must add something like authentication url?

    P.S.: I modified the script adding an enviorment variable CLOUDFILES_AUTHURL pointing to my openstack auth url, UPLOAD_TO_CONTAINER=”my_container_name” and PASSPHRASE=”" with –no-encryption option in the duplicity command.

  • Anonymous

    You need to make sure you have the latest python-cloudfiles which include this change https://github.com/rackspace/python-cloudfiles/commit/23c07dfcbf00733245302c33892965a8bbf2d8de

  • Rdc

    I solved my problem, as you said it was a dependencies problem

  • Tomatelas

    Hi, I wonder… how do you restore the backups?

    Thanks!