Accessing to Rackspace Cloud Files via servicenet (update)

Last week I have posted an article explaining how to connect to Rackspace Cloud Files via Rackspace ServiceNET but I actually got it wrong as pointed by my great colleague exlt so I had to take it down until figured out how to fix it.

I have add that feature properly to the PHP and Python API in version 1.5.0 to add a ‘servicenet’ argument to the connection and updated the blog post here :

http://blog.chmouel.com/2009/10/14/how-to-connect-to-rackspace-cloud-files-via-servicenet/

It should give you all the information for the howto use that feature.

I have released as well a minor update in 1.5.1 to allow you to define the environment variable RACKSPACE_SERVICENET to force the use of the Rackspace ServiceNET this allow you to don’t have to modify the tools and have a clean code between prod and testing.

How to connect to Rackspace Cloud Files via ServiceNET

If you are a Rackspace customer and you are are planning to use Rackspace Cloud Files via it’s internal network (ServiceNet) so you don’t get billed for the bandwidth going over Cloud Files this is how you can do.

The first thing is to make sure with your support team if your servers are connected to ServiceNet and if you have that connection then there is a small change to do in your code.

The second thing is to use the just released 1.5.0 version on GitHUB for PHP :

http://github.com/rackspace/php-cloudfiles/tree/1.5.0

and for python :

http://github.com/rackspace/python-cloudfiles/tree/1.5.0

(you need to click on the download link at the top to download the tarball of the release).

Afer this is just a matter to set the argument servicenet=True, for example in PHP :

$user='username';
$api_key='theapi_key';

$auth = new CF_Authentication($user, $api_key);
$auth->authenticate();
$conn = new CF_Connection($auth, $servicenet=true);

In Python you can do like this :

username='username'
api_key='api_key'

cnx = cloudfiles.get_connection(username, api_key, servicenet=True)

Rackspace Cloud Files helper scripts

Since working a lot with Rackspace Cloud Files I have put some quick
scripts using the python-cloudfiles API to do some ls rm and cp of containers or objects..

It’s really a collection of quickly written stuff put in the same file
but at least if not useful for you it would give you an idea how the
python-cloudfiles works. It is all available here :

http://github.com/chmouel/cloud-files-helper

Ruby XMLRPC over a Self Certified SSL with a warning

If you use the XMLRPC client in ruby over a self certified SSL you have this warning :

warning: peer certificate won’t be verified in this SSL session

You can get override that warning cleanly (i have seen some people who just comment the message in the standard library) like that :

require 'xmlrpc/client'

require 'net/https'
require 'openssl'
require 'pp'

module SELF_SSL
  class Net_HTTP < Net::HTTP
    def initialize(*args)
      super
      @ssl_context = OpenSSL::SSL::SSLContext.new
      @ssl_context.verify_mode = OpenSSL::SSL::VERIFY_NONE
    end
  end

  class XMLRPC_Client < XMLRPC::Client
    def initialize(*args)
      super
      @http = SELF_SSL::Net_HTTP.new( @host, @port,
                                      @proxy_host,@proxy_port )
      @http.use_ssl = @use_ssl if @use_ssl
      @http.read_timeout = @timeout
      @http.open_timeout = @timeout
    end

  end

end

if __FILE__ == $0
  f = SELF_SSL::XMLRPC_Client.new2("https://url")
  puts f.call("method", 'arg')
end

Yum Force Exclude List

While talking with my fellow colleague Darren Birkett about what seems a design limitation
of yum to not be able to force listing the excludes from yum. I had a
shoot to make a yum plugin to force listing the excludes.

Here is how it works :

root@centos5:~> grep exclude /etc/yum.conf
exclude=rpm*
root@centos5:~> yum install rpm-devel
Loading “installonlyn” plugin
Loading “changelog” plugin
Loading “chmouel” plugin
Loading “priorities” plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
Excluding Packages in global exclude list
Finished
0 packages excluded due to repository priority protections
Parsing package install arguments
Nothing to do

rpm* is excluded, but if we use the environment variable FORCE_EXCLUDE
it will force it.

 root@centos5:~> FORCE_EXCLUDE=true yum install rpm-devel
Loading “installonlyn” plugin
Loading “changelog” plugin
Loading “chmouel” plugin
Loading “priorities” plugin
Setting up Install Process
Setting up repositories
Reading repository metadata in from local files
0 packages excluded due to repository priority protections
Parsing package install arguments
Resolving Dependencies
–> Populating transaction set with selected packages. Please wait.
—> Downloading header for rpm-devel to pack into transaction set.
rpm-devel-4.4.2-47.el5.i3 100% |=========================| 17 kB 00:00
—> Package rpm-devel.i386 0:4.4.2-47.el5 set to be updated
–> Running transaction check
[…..]

It will allow you to list the excluded rpm as well :

 root@centos5:~> FORCE_EXCLUDE=true yum list|grep rpm
rpm.i386 4.4.2-47.el5 installed
rpm-libs.i386 4.4.2-47.el5 installed
rpm-python.i386 4.4.2-47.el5 installed
redhat-rpm-config.noarch 8.0.45-22.el5.centos base
rpm-build.i386 4.4.2-47.el5 base
rpm-devel.i386 4.4.2-47.el5 base
root@centos5:~> yum list|grep rpm
rpm.i386 4.4.2-47.el5 installed
rpm-libs.i386 4.4.2-47.el5 installed
rpm-python.i386 4.4.2-47.el5 installed
redhat-rpm-config.noarch 8.0.45-22.el5.centos base

See the README.txt in the rpm file to see how to use/install it.

You can download the rpm here and the src.rpm here

Moving from Australia and going back to Paris.

Finally after three years (almost i arrived on 29 September 2004 here) i am following all my french friends (except Aurelien whos staying here) and i am moving back to France. It is a heartbreak since this is one (if not the one) of the best country i ever lived. I made a lot of good friends in Sydney and for the last 3 month in Melbourne. Well anyway i am sorta looking forward to come back to France to be near my kid and see all my friends there.

And as always on my way back to France i am going to travel a bit in South America until december, i have made a website which show all places i am going to which i would update on the go.

The website is here :

http://whereis.chmouel.com/

It is probably only interesting to my mum but it as well give me some good understanding of all thoses new Ajax technology and others MVC Framework (via Django).

For all my french friends i hope to see you soon in Paris!

Generating md5 encrypted password for chpasswd

If you want to generate properly encrypted password to feed to chpasswd, the most easier and proper way is to do that from command line :
[code lang=”bash”]
echo "encryptedpassword"|openssl passwd -1 -stdin
[/code]
If you want to generate in pure python you can do it like that :
[code lang=”python”]
def md5crypt(password, salt, magic=’$1$’):
import md5
m = md5.new()
m.update(password + magic + salt)

# /* Then just as many characters of the MD5(pw,salt,pw) */
mixin = md5.md5(password + salt + password).digest()
for i in range(0, len(password)):
m.update(mixin[i % 16])

# /* Then something really weird… */
# Also really broken, as far as I can tell. -m
i = len(password)
while i:
if i & 1:
m.update(‘x00’)
else:
m.update(password[0])
i >>= 1

final = m.digest()
# /* and now, just to make sure things don’t run too fast */
for i in range(1000):
m2 = md5.md5()
if i & 1:
m2.update(password)
else:
m2.update(final)
if i % 3:
m2.update(salt)
if i % 7:
m2.update(password)

if i & 1:
m2.update(final)
else:
m2.update(password)

final = m2.digest()

# This is the bit that uses to64() in the original code.
itoa64 = ‘./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz’
rearranged = ”
for a, b, c in ((0, 6, 12), (1, 7, 13), (2, 8, 14), (3, 9, 15), (4, 10, 5)):
v = ord(final[a]) < < 16 | ord(final[b]) << 8 | ord(final[c]) for i in range(4): rearranged += itoa64[v & 0x3f]; v >>= 6

v = ord(final[11])
for i in range(2):
rearranged += itoa64[v & 0x3f]; v >>= 6

return magic + salt + ‘$’ + rearranged

[/code]

You need to feed it up with a salt, like this :
[code lang=”python”]
def generate_salt(count):
import random, string
char = string.ascii_letters + string.digits + string.punctuation.replace(‘:’, ”)
return string.join(map(lambda x,v=char: random.choice(v), range(count)), ”)
[/code]