Chmouel's blog

How does a PUT to a swift object server look like.

I have been trying lately to get a better understanding of the Swift code base, and I found the best way to know it was to read it from top to bottom and document it along the way. Here is some of my notes, hopefully more will come. I am starting with an object PUT when the request is coming from the proxy server. The request in the log-file will look like this : ...

Audit a swift cluster

Swift integrity tools. There is quite a bit of tools shipped with Swift to ensure you have the right object on your cluster. At first there is the basic : swift-object-info It will take a swift object stored on the filesystem and print some infos about it, like this : swift@storage01:0/016/0b221bab535ac1b8f0d91e394f225016$ swift-object-info 1327991417.01411.data Path: /AUTH_root/foobar/file.txt Account: AUTH_root Container: foobar Object: file.txt Object hash: 0b221bab535ac1b8f0d91e394f225016 Ring locations: 192.168.254.12:6000 - /srv/node/sdb1/objects/0/016/0b221bab535ac1b8f0d91e394f225016/1327991417.01411.data Content-Type: text/plain Timestamp: 2012-01-31 06:30:17.014110 (1327991417.01411) ETag: 053a0f8516a5023b9af76c49ca917d3e (valid) Content-Length: 24 (valid) User Metadata: {‘X-Object-Meta-Mtime’: ‘1327968327.21’} ...

My stats for this year running and cycling

Running 1,266 km (786 miles): Cycling 3865 km (2401 miles): Let’s see if I can improve for next year

Swift and Keystone middleware

[NB: Much things has changed since I have written this article but keeping it here for info] It seems that integrating Swift and Keystone together present some challenges to people and this is absolutely normal as there is a lot of changes going on. This is my attempt to document how everything is plugged together. ...

Debugging python-novaclient on the command line.

I have done lately quite a bit of work with python-novaclient the (nova/keystone) openstack client. I often experiment it with ipython in the console. There is a nice debugging facility in novaclient which you can see while using –debug argument on the command line and if you wanted to use it with ipython you could have that at the beginning of your session : ...

Rackspace CloudDNS python binding

I have released a python binding to Rackspace CloudDNS here which allow you to create/update/delete domains and records. It’s available on github https://github.com/rackspace/python-clouddns/ The binding is pretty simple and have unfortunately no documentation (or even tests) but you can figure out most of it from here : https://github.com/rackspace/python-clouddns/blob/master/tests/t.py ...

Mass editing firewall on Rackspace Cloud.

A lot of our customers in Rackspace cloud has been asking how to mass edit firewalls of servers when you have multiple servers without doing it manually. Part of my cloudservers-api-demo I have written a simple firewall scripts abstracting the Operating System firewall software to allow/enable/disable the firewall and ports/networks. The script has been kept very simple by design and currently allow only to : enable the firewall </li> <li> disable the firewall </li> <li> allow or disallow a port or a network </li> <li> see firewall status </li> </ul> <h3> PREREQUISITES </h3> <ul> <li> A management server under Ubuntu maverick. </li> <li> A supported Operating System for clients which includes : </li> <li> Debian. </li> <li> Ubuntu. </li> <li> RHEL. </li> <li> Fedora </li> <li> My patched python-cloudservers library (see below for installs). </li> <li> Your SSH key installed on all VM for root users. </li> </ul> <h3> Install </h3> <ul> <li> After you have kicked a VM with a Ubuntu maverick and connected to it as root you want first execute intall some prereq packages : </li> </ul> apt-get update && apt-get -y install python-stdeb git checkout my python-cloudservers library : git clone git://github.com/chmouel/python-cloudservers.git after being checked-out you will go into the python-cloudservers directory which has just been created and do this : cd python-cloudservers/ python setup.py install this should automatically install all the dependences. Now you can install my api-demo which include the firewall script : cd ../ git clone git://github.com/chmouel/cloudservers-api-demo You need to configure some environemnt variable first which keep information about your rackspace account. edit your ~/.bashrc (or /etc/environement if you want to make it global) and configure those variable : export RCLOUD_DATACENTER=UK export UK_RCLOUD_USER="MY_USERNAME" export UK_RCLOUD_KEY="MY_API_KEY" export UK_RCLOUD_AURL="https://lon.auth.api.rackspacecloud.com/v1.0" or for the US you would have : export RCLOUD_DATACENTER=US export UK_RCLOUD_USER="MY_USERNAME" export UK_RCLOUD_KEY="MY_API_KEY" export UK_RCLOUD_AURL="https://auth.api.rackspacecloud.com/v1.0" source the ~/.bashrc or relog into your account to have those accounts set-up you can test it to see if that works by going to : ~/cloudservers-api-demo/python and launch the command : ./list-servers.py to test if this is working properly (it should list your servers for your DATACENTER) you are now basically ready to mass update firewall on all servers. Let's say you have two web servers named web1 and web2 and two db servers named db1 and db2 and you would like to allow the 80 port on the web servers and 3306 port on the db servers. You would have to go to this directory : ~/cloudservers-api-demo/firewall/ and first execute this command to see the help/usages : ./fw-control.py --help so let's say to enable the firewall on all the web and db server first you can do : ./fw-control.py -s "web db" enable it will connect and enable the firewall on all the servers which match the name web and db. now let's say we want to enable port 80 on the web : ./fw-control.py -s "web" allow port 80 if you log into the servers you can check with iptables -L -n that it it has been enabled properly. This is simple enough for you to modify the script to your liking to make it more modular for your specific environement.

Howto shutdown your Cloud Server and not getting billed for it.

Currently in Rackspace-Cloud when you are shutting-down your Cloud Servers you are still paying for it. The reason is that when the Cloud Server is shut-down your CloudServer is still sitting on the hyper-visor and still use resources on the Cloud and then get you billed for it. There is a way to get around it by having the CloudServer stored as an image into CloudFiles. The caveat with this solution is that every-time you are creating a server out of the stored image you are getting a new IP and in certain cases you would need to make a change in your application with the new IP. ...

Uploading to Rackspace Cloud Files via FTP

Sometime ago I wrote a FTP proxy to RackSpace Cloud Files which expose Rackspace Cloud Files as a FTP server acting as a proxy. Thanks to the OpenSource community a user on github took it and add support OpenStack and all the latest features available in Cloud Files. It is now pretty robust and works pretty well via nautilus even with the pseudo hierarchical folder feature. The fun part here is that it allow you to effectively have a Cloud Drive where you can easily store your files/backup from your Linux desktop via nautilus built-in ftp support. ...

Installing python-cloudfiles from pypi

I have just uploaded python-cloudfiles to pypi available here This make things easy to add as a dependence of your project like you can have something like this in your setup.py : requirements = ['python-cloudfiles'] and it will automatically download it as part of the dependence with easy_install or pip. cool kids on latest debian/ubuntu can do stuff like this (from python-stdeb package) : pypi-install python-cloudfiles which would automatically download the tarball from pypi and install it as a packages (like the way it should be for prod machine!) ...