How to connect to Rackspace Cloud Files via ServiceNET

If you are a Rackspace customer and you are are planning to use Rackspace Cloud Files via it’s internal network (ServiceNet) so you don’t get billed for the bandwidth going over Cloud Files this is how you can do.

The first thing is to make sure with your support team if your servers are connected to ServiceNet and if you have that connection then there is a small change to do in your code.

The second thing is to use the just released 1.5.0 version on GitHUB for PHP :

http://github.com/rackspace/php-cloudfiles/tree/1.5.0

and for python :

http://github.com/rackspace/python-cloudfiles/tree/1.5.0

(you need to click on the download link at the top to download the tarball of the release).

Afer this is just a matter to set the argument servicenet=True, for example in PHP :

$user='username';
$api_key='theapi_key';

$auth = new CF_Authentication($user, $api_key);
$auth->authenticate();
$conn = new CF_Connection($auth, $servicenet=true);

In Python you can do like this :

username='username'
api_key='api_key'

cnx = cloudfiles.get_connection(username, api_key, servicenet=True)

python syntax warning in emacs

One of the best feature to have with Emacs when doing python development is to have a real time syntax error/warning check highlighted in your code to avoid many errors or superfluous code.

This code is taken from the brillant Emacswiki python page.

You need to install pyflakes first which should be available on your linux distro by default in a package or for the other OS you may follow the procedure from the pyflakes webpage.

and add this configuration to your .emacs :

(when (load "flymake" t)
(defun flymake-pyflakes-init ()
  (let* ((temp-file (flymake-init-create-temp-buffer-copy
                     'flymake-create-temp-inplace))
         (local-file (file-relative-name
                      temp-file
                      (file-name-directory buffer-file-name))))
    (list "pyflakes" (list local-file))))
(add-to-list 'flymake-allowed-file-name-masks
             '("\\.py\\'" flymake-pyflakes-init)))

make sure pyflakes is in your path and enable flymake-mode by default if you like for all python mode :


(add-hook 'python-mode-hook 'my-python-mode-hook)

go by M-x customize flymake as well if you like to customise some variables.

Yum Force Exclude List

While talking with my fellow colleague Darren Birkett about what seems a design limitation
of yum to not be able to force listing the excludes from yum. I had a
shoot to make a yum plugin to force listing the excludes.

Here is how it works :

root@centos5:~> grep exclude /etc/yum.conf
exclude=rpm*
root@centos5:~> yum install rpm-devel
Loading “installonlyn” plugin
Loading “changelog” plugin
Loading “chmouel” plugin
Loading “priorities” plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
Excluding Packages in global exclude list
Finished
0 packages excluded due to repository priority protections
Parsing package install arguments
Nothing to do

rpm* is excluded, but if we use the environment variable FORCE_EXCLUDE
it will force it.

 root@centos5:~> FORCE_EXCLUDE=true yum install rpm-devel
Loading “installonlyn” plugin
Loading “changelog” plugin
Loading “chmouel” plugin
Loading “priorities” plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
0 packages excluded due to repository priority protections
Parsing package install arguments
Resolving Dependencies
–> Populating transaction set with selected packages. Please wait.
—> Downloading header for rpm-devel to pack into transaction set.
rpm-devel-4.4.2-47.el5.i3 100% |=========================| 17 kB 00:00
—> Package rpm-devel.i386 0:4.4.2-47.el5 set to be updated
–> Running transaction check
[…..]

It will allow you to list the excluded rpm as well :

 root@centos5:~> FORCE_EXCLUDE=true yum list|grep rpm
rpm.i386 4.4.2-47.el5 installed
rpm-libs.i386 4.4.2-47.el5 installed
rpm-python.i386 4.4.2-47.el5 installed
redhat-rpm-config.noarch 8.0.45-22.el5.centos base
rpm-build.i386 4.4.2-47.el5 base
rpm-devel.i386 4.4.2-47.el5 base
root@centos5:~> yum list|grep rpm
rpm.i386 4.4.2-47.el5 installed
rpm-libs.i386 4.4.2-47.el5 installed
rpm-python.i386 4.4.2-47.el5 installed
redhat-rpm-config.noarch 8.0.45-22.el5.centos base

See the README.txt in the rpm file to see how to use/install it.

You can download the rpm here and the src.rpm here

Generating md5 encrypted password for chpasswd

If you want to generate properly encrypted password to feed to chpasswd, the most easier and proper way is to do that from command line :
[code lang=”bash”]
echo "encryptedpassword"|openssl passwd -1 -stdin
[/code]
If you want to generate in pure python you can do it like that :
[code lang=”python”]
def md5crypt(password, salt, magic=’$1$’):
import md5
m = md5.new()
m.update(password + magic + salt)

# /* Then just as many characters of the MD5(pw,salt,pw) */
mixin = md5.md5(password + salt + password).digest()
for i in range(0, len(password)):
m.update(mixin[i % 16])

# /* Then something really weird… */
# Also really broken, as far as I can tell. -m
i = len(password)
while i:
if i & 1:
m.update(‘x00’)
else:
m.update(password[0])
i >>= 1

final = m.digest()
# /* and now, just to make sure things don’t run too fast */
for i in range(1000):
m2 = md5.md5()
if i & 1:
m2.update(password)
else:
m2.update(final)
if i % 3:
m2.update(salt)
if i % 7:
m2.update(password)

if i & 1:
m2.update(final)
else:
m2.update(password)

final = m2.digest()

# This is the bit that uses to64() in the original code.
itoa64 = ‘./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz’
rearranged = ”
for a, b, c in ((0, 6, 12), (1, 7, 13), (2, 8, 14), (3, 9, 15), (4, 10, 5)):
v = ord(final[a]) < < 16 | ord(final[b]) << 8 | ord(final[c]) for i in range(4): rearranged += itoa64[v & 0x3f]; v >>= 6

v = ord(final[11])
for i in range(2):
rearranged += itoa64[v & 0x3f]; v >>= 6

return magic + salt + ‘$’ + rearranged

[/code]

You need to feed it up with a salt, like this :
[code lang=”python”]
def generate_salt(count):
import random, string
char = string.ascii_letters + string.digits + string.punctuation.replace(‘:’, ”)
return string.join(map(lambda x,v=char: random.choice(v), range(count)), ”)
[/code]

Cheetah Mode for Emacs

Here is a simple html derived mode for Cheetah templates files. The font-locking regexp can be improved thought but that’s a start.

(define-derived-mode cheetah-mode html-mode "Cheetah"
  (make-face 'cheetah-variable-face)
  (font-lock-add-keywords
   nil
   '(
     ("\\(#\\(from\\|else\\|include\\|set\\|import\\|for\\|if\\|end\\)+\\)\\>" 1 font-lock-type-face)
     ("\\(#\\(from\\|for\\|end\\)\\).*\\<\\(for\\|import\\|if\\|in\\)\\>" 3 font-lock-type-face)
     ("\\(\\$\\(?:\\sw\\|}\\|{\\|\\s_\\)+\\)" 1 font-lock-variable-name-face))
   )
  (font-lock-mode 1)
  )
(setq auto-mode-alist (cons '( "\\.tmpl\\'" . cheetah-mode ) auto-mode-alist ))

Assignement in Python with list.extend()

This is weird for me :

d = ['foo', 'bar', 'ba', 'c']
print d
f = d
f.extend(d)
print d

give me the result

-*- mode: compilation; default-directory: “/tmp/” -*-
Compilation started at Mon Jul 31 16:49:41

python “/tmp/a.py”
[‘foo’, ‘bar’, ‘ba’, ‘c’]
[‘foo’, ‘bar’, ‘ba’, ‘c’, ‘foo’, ‘bar’, ‘ba’, ‘c’]

Compilation finished at Mon Jul 31 16:49:4

It seems that extend assign as well the non extended part (d) which is confusing because to merge list i need to use temporary variable.

Python 2.5 Beta

I have been looking at the What’s new of Python 2.5. There is some cool features inside it :

Conditional Expressions:

This stuff basically allow to do standard C idiom (that we found in every kind of derivative language) like

a = condition ? “true” : “false”

the weird part is that Guido Van-Rossum implemented this syntax :

x = true_value if condition else false_value

which we will have to get use to. I am not a big fan of this feature since it makes the code more obscure. But still could be useful for little scripts or quick hack.

Unified try/except/finally:

I like the inclusion of else in a try finally. I use to put everything in the try which was not really very good looked when now i can really try or else something. Make me easier to port some Java code to python as well and respect the logic of the java program.

The with statement:

This one is weird for me it’s basically a closure of variable allowing to do stuff like this :

with expression as variable:

stuff

#expression not here anymore

I am still wondering how and why to use that. But i guess it will come us when i will do some coding in Python 2.5.

Other interesting stuff:

Sqlite3 library default inclusion. The cool thing about this for Linux vendors is that it does not require the libsqlite3 library but link dynamically via the ctypes module (i guess).

Ctypes allow you to call C dynamic functions via python.

A fast new XML parser.

Sounds like fun for the new python and there is a lot of other bugfixes so go read it.