Chmouel's Blog » Python

Debugging python-novaclient on the command line.

chmouel — Fri, 02 Sep 2011 03:00:13 +0000

I have done lately quite a bit of work with python-novaclient the (nova/keystone) openstack client. I often experiment it with ipython in the console.

There is a nice debugging facility in novaclient which you can see while using –debug argument on the command line and if you wanted to use it with ipython you could have that at the beginning of your session :

This would give you the details of the session showing you the REST requests and responses including the headers. It even show you the curl commands that you can use on the command line to experiment with it.

Installing python-cloudfiles from pypi

chmouel — Wed, 16 Feb 2011 05:41:33 +0000

I have just uploaded python-cloudfiles to pypi available here

This make things easy to add as a dependence of your project like you can have something like this in your setup.py :

requirements = ['python-cloudfiles']

and it will automatically download it as part of the dependence with easy_install or pip.

cool kids on latest debian/ubuntu can do stuff like this (from python-stdeb package) :

pypi-install python-cloudfiles

which would automatically download the tarball from pypi and install it as a packages (like the way it should be for prod machine!)

If you have a virtualenv environment you can easily do a (needs python-pip package) :

pip -E /usr/local/myvirtualenvroot install python-cloudfiles

and magic would be done to get you on latest python-cloudfiles.

As a bonus side you can browse online the python-cloudfiles library :

http://packages.python.org/python-cloudfiles/

[Update] This has been renamed back to python-cloudfiles please update your setup.py or scripts.

Python decorator to argument checking

chmouel — Thu, 29 Jul 2010 19:03:57 +0000

I am sure there is billions or more people who already done that but I needed this quickly for my project and was not feeling googling around :

Get Facebook albums with python

chmouel — Sat, 09 Jan 2010 14:42:19 +0000

Since I haven’t see much script like this around the web here is a quick script to suck bunch of albums from facebooks (your own) nothing fancy just something to get you started with pyfacebook.

#!/usr/bin/python
import os
import urllib
from facebook import Facebook
 
# see http://developers.facebook.com/get_started.php
# Your API key
API_KEY="YOUR_API_KEY"
# Application secret key
SECRET_KEY="YOUR_SECRET_KEY"
 
cnx = Facebook(API_KEY, SECRET_KEY)
cnx.auth.createToken()
cnx.login()
cnx.auth.getSession()
 
def choose_albums(cnx):
    cnt = 1
    ret={}
    bigthing=cnx.photos.getAlbums(cnx.uid)
 
    for row in bigthing:
        ret[cnt] = row['name'], row['aid'], row['link']
        print "%d) %s - %s" % (cnt, row['name'], row['link'])
        cnt += 1
    ans = raw_input("Choose albums (separated by ,): ")
    return [ret[int(row)] for row in ans.split(', ') ]
 
chosen_albums = choose_albums(cnx)
for album in chosen_albums:
    name, aid, _ =  album
    print "Album: ", (name)
    ddir = "fbgallery/%s" % name
    if not os.path.exists(ddir):
        os.makedirs(ddir)
    for photo in cnx.photos.get(aid=aid):
        url = photo['src_big']
        dest="%s/%s.jpg" % (ddir, photo['pid'])
        if not os.path.exists(dest):
            print "Getting: ", url
            urllib.urlretrieve(url, dest)

Twitter notification of replies by Email

chmouel — Thu, 07 Jan 2010 14:40:25 +0000

I don’t check very often my twitter to know when someone replies and I find it hard to figure out what’s going on when i check a couple of days after even using a client showing only the reply (my client of choice lately is the Emacs twittering mode client)

I have made a script who’s checking your direct reply and email it to you. This is to setup via a cron on a server who has a mail server configured locally. You can get it from here :

http://github.com/chmouel/twitter-reply-notification

XMPP notification for irssi running in a screen on a remote host

chmouel — Sun, 20 Dec 2009 15:41:55 +0000

Like a lot of people I have my irssi on a server in a screen. This has
been working great so far but my only concerns are the notifications
on the desktop when something happening.

Over the time I have found some different solution with mitigated
results for me :

- Use fanotify script with the libnotify-bin and SSH like mentioned here.

- Setup your irssi (or other) as irc proxy bouncer and connect with
your desktop client (like xchat) to get notification.

The fanotify is kind of very hacky on a laptop with intermittent
connection and having a cron doing a ssh every minutes or so is not
ideal, not talking about no passphrase ssh key or having to snoop the
SSH_AGENT variable to connect without password.

The via proxy method is not my thing and I don’t feel like having
xchat open all the time just for it and I anyway usually forget to
launch it.

My solution is to have a plugin for irssi notify me via XMPP if there
is a direct message addressed to me. I usually get my pidgin or gmail
alway open and if i don’t since it goes to a gmail account I got gmail
sending me an email about it.

You can find all the information about the install and configuration
here :

http://github.com/chmouel/irssi-xmpp-notify

Accessing to Rackspace Cloud Files via servicenet (update)

chmouel — Tue, 20 Oct 2009 18:47:49 +0000

Last week I have posted an article explaining how to connect to Rackspace Cloud Files via Rackspace ServiceNET but I actually got it wrong as pointed by my great colleague exlt so I had to take it down until figured out how to fix it.

I have add that feature properly to the PHP and Python API in version 1.5.0 to add a ‘servicenet’ argument to the connection and updated the blog post here :

http://blog.chmouel.com/2009/10/14/how-to-connect-to-rackspace-cloud-files-via-servicenet/

It should give you all the information for the howto use that feature.

I have released as well a minor update in 1.5.1 to allow you to define the environment variable RACKSPACE_SERVICENET to force the use of the Rackspace ServiceNET this allow you to don’t have to modify the tools and have a clean code between prod and testing.

How to connect to Rackspace Cloud Files via ServiceNET

chmouel — Wed, 14 Oct 2009 00:47:27 +0000

If you are a Rackspace customer and you are are planning to use Rackspace Cloud Files via it’s internal network (ServiceNet) so you don’t get billed for the bandwidth going over Cloud Files this is how you can do.

The first thing is to make sure with your support team if your servers are connected to ServiceNet and if you have that connection then there is a small change to do in your code.

The second thing is to use the just released 1.5.0 version on GitHUB for PHP :

http://github.com/rackspace/php-cloudfiles/tree/1.5.0

and for python :

http://github.com/rackspace/python-cloudfiles/tree/1.5.0

(you need to click on the download link at the top to download the tarball of the release).

Afer this is just a matter to set the argument servicenet=True, for example in PHP :

$user='username';
$api_key='theapi_key';
 
$auth = new CF_Authentication($user, $api_key);
$auth->authenticate();
$conn = new CF_Connection($auth, $servicenet=true);

In Python you can do like this :

username='username'
api_key='api_key'
 
cnx = cloudfiles.get_connection(username, api_key, servicenet=True)

python syntax warning in emacs

chmouel — Mon, 31 Aug 2009 05:04:28 +0000

One of the best feature to have with Emacs when doing python development is to have a real time syntax error/warning check highlighted in your code to avoid many errors or superfluous code.

This code is taken from the brillant Emacswiki python page.

You need to install pyflakes first which should be available on your linux distro by default in a package or for the other OS you may follow the procedure from the pyflakes webpage.

and add this configuration to your .emacs :

(when (load "flymake" t)
(defun flymake-pyflakes-init ()
  (let* ((temp-file (flymake-init-create-temp-buffer-copy
                     'flymake-create-temp-inplace))
         (local-file (file-relative-name
                      temp-file
                      (file-name-directory buffer-file-name))))
    (list "pyflakes" (list local-file))))
(add-to-list 'flymake-allowed-file-name-masks
             '("\\.py\\'" flymake-pyflakes-init)))

make sure pyflakes is in your path and enable flymake-mode by default if you like for all python mode :

(add-hook 'python-mode-hook 'my-python-mode-hook)

go by M-x customize flymake as well if you like to customise some variables.

Yum Force Exclude List

chmouel — Thu, 20 Mar 2008 09:36:06 +0000

While talking with my fellow colleague Darren Birkett about what seems a design limitation
of yum to not be able to force listing the excludes from yum. I had a
shoot to make a yum plugin to force listing the excludes.

Here is how it works :

root@centos5:~> grep exclude /etc/yum.conf
exclude=rpm*
root@centos5:~> yum install rpm-devel
Loading “installonlyn” plugin
Loading “changelog” plugin
Loading “chmouel” plugin
Loading “priorities” plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
Excluding Packages in global exclude list
Finished
0 packages excluded due to repository priority protections
Parsing package install arguments
Nothing to do

rpm* is excluded, but if we use the environment variable FORCE_EXCLUDE
it will force it.

root@centos5:~> FORCE_EXCLUDE=true yum install rpm-devel
Loading “installonlyn” plugin
Loading “changelog” plugin
Loading “chmouel” plugin
Loading “priorities” plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
0 packages excluded due to repository priority protections
Parsing package install arguments
Resolving Dependencies
–> Populating transaction set with selected packages. Please wait.
—> Downloading header for rpm-devel to pack into transaction set.
rpm-devel-4.4.2-47.el5.i3 100% |=========================| 17 kB 00:00
—> Package rpm-devel.i386 0:4.4.2-47.el5 set to be updated
–> Running transaction check
[.....]

It will allow you to list the excluded rpm as well :

root@centos5:~> FORCE_EXCLUDE=true yum list|grep rpm
rpm.i386 4.4.2-47.el5 installed
rpm-libs.i386 4.4.2-47.el5 installed
rpm-python.i386 4.4.2-47.el5 installed
redhat-rpm-config.noarch 8.0.45-22.el5.centos base
rpm-build.i386 4.4.2-47.el5 base
rpm-devel.i386 4.4.2-47.el5 base
root@centos5:~> yum list|grep rpm
rpm.i386 4.4.2-47.el5 installed
rpm-libs.i386 4.4.2-47.el5 installed
rpm-python.i386 4.4.2-47.el5 installed
redhat-rpm-config.noarch 8.0.45-22.el5.centos base

See the README.txt in the rpm file to see how to use/install it.

You can download the rpm here and the src.rpm here