Django, metaWeblog and Amazon S3 06 Jun 2007

This is a quick write up about adding support for Amazon S3 to your Django weblog. I added support for metaWeblog API by using the this write up from All Your Pixel. Most, if not all, blogging clients support the metaWeblog API so I think it was a good choice.

Why use Amazon S3?

• It's super cheap (probably cost you less than $1/mo)


• Performance is improved by moving static media away from your Django/Apache instance.


• Amazon worries about managing and scaling the storage back end.

First off, let's edit the settings.py that is in your Django project's directory. We will want to add your Amazon access information. Note: You can get your access information by signing up for S3 at the Amazon site. Add the following to your settings.py file:

AWS_ACCESS_KEY = 'Your Key'
AWS_SECRET_ACCESS_KEY = 'Your Secret Key'
BUCKET_NAME = 'Your Bucket Name'
AWS_S3_URL = 'http://s3.amazonaws.com'

So, I am going to assume you are using the xmlrpc.py and metaweblog.py that was provided in the All Your Pixel posting (linked above). Be sure you are importing the Django project settings file by using:

from django.conf import settings

This will also require the Python S3 module that is provided by Amazon. Get this file and add it to your project, or your Python path. Import that module, and the mimetypes module which is used to guess the type of the file being uploaded.

from yourproject import S3
import mimetypes

Now we will create, or edit, the metaWeblog_newMediaObject method. This is what is called when your blogging client attaches, or uploads, a media file to your blog post.

def metaWeblog_newMediaObject(user, blogid, struct):
  ret = {}
  fext = os.path.splitext(struct['name'])[1].lower()
  fname = generate_fname() + fext
  try:
    conn = S3.AWSAuthConnection(settings.AWS_ACCESS_KEY, 
                                settings.AWS_SECRET_ACCESS_KEY)
    buckets = conn.list_all_my_buckets()
    if not settings.BUCKET_NAME in [b.name for b in buckets.entries]:
      # BUCKET_NAME doesn't exist, create it!
      res = conn.create_bucket(settings.BUCKET_NAME)
      if res.http_response.status != 200:
        raise
      filename = 'uploads/' + fname
      res = conn.get(settings.BUCKET_NAME, filename)
      while res.http_response.status == 200:
        # File exists, generate new filename
        fname = generate_fname() + fext
        filename = 'uploads/' + fname
        res = conn.get(settings.BUCKET_NAME, filename)
      content_type = mimetypes.guess_type(filename)[0]
      if not content_type:
        content_type = 'text/plain'
      res = conn.put(settings.BUCKET_NAME,
                     filename,
                     S3.S3Object(struct['bits'].__str__()),
                     {'x-amz-acl': 'public-read',
                      'Content-Type': content_type}
      )
      if res.http_response.status == 200:
        ret['url'] = '%s/%s/%s' % (settings.AWS_S3_URL,
                                   settings.BUCKET_NAME,
                                   filename)
    except:
      pass
    return ret

I should mention that the method generate_fname(), which is used above, is just a function to generate a MD5 hash which will be used as a file name of the new file being added. It is not required, but you may want to use something similar. A quick run down of what this code does:

1 - Generates a file name to use.
2 - Creates an S3 instance.
3 - Gets a list of all your S3 buckets.
4 - Checks to see that the bucket you want to use exists. If not, it creates it.
5 - Checks to see if the file already exists. If so, it generates a new file name.
6 - Guesses the file type.
7 - Uploads the file to S3.
8 - Returns the file URL to your blogging client.

That's it! There are a few other articles written about this. Check them out as well!