James Tauber : James Tauber's Blog 2009/01

blog > 2009 >

Moving to Distutils

reposted from http://code.google.com/p/django-hotclub/wiki/MovingToDistutils

the how and why of Pinax's move to distutils

Pinax is changing the way that external dependencies are brought in during development on trunk. Note that this document is only talking about changes in how things work and will work on trunk, NOT necessarily how they will work with a released version of Pinax.

Until recently, Pinax had two choices for a given external dependency:

use svn:externals and point to the external dependency's svn repository
include the external dependency code in the Pinax codebase

However, there are problems with this approach:

it largely relies on external dependencies being in svn and this is increasingly not the case (although it was when Pinax started)
it makes it difficult for Pinax itself to move away from svn
there is no management of dependencies between external dependencies, nor between particular projects in Pinax and their individual dependencies

To solve these problems and more, Pinax is switching to a distutils-based approach. This means:

externals dependencies are encouraged to be released as distutil-compliant packages with a valid setup.py and put on PyPI
development versions of dependencies can be pulled in in a variety of different ways including from git, hg or bzr repositories

In order to develop from the Pinax trunk, you will need to use pip. Because some external dependencies are retrieved via git and bzr you will also need those if using Pinax trunk.

Although we will eventually have per-project requirements files, there are currently two requirements files that describe to pip what dependencies to bring in and how:

pinax/requirements/libs.txt
pinax/requirements/external_apps.txt

The former is actually a requirement of the latter so you can bring in all external dependencies with:

pip install -r pinax/requirements/external_apps.txt

We strongly recommend the use of virtualenv in conjunction with pip to allow isolated environments to be set up without Pinax having to hack PYTHONPATH.

by : Created on Jan. 31, 2009 : Last modified Jan. 31, 2009 : (permalink)

Retweet

@gruber: Theory: the people who send out “RT” tweets are the same fuckers who came in and ruined Usenet a decade ago.

@jtauber: couldn't resist: RT @gruber Theory: the people who send out “RT” tweets are the same fuckers who came in and ruined Usenet a decade ago.

@akuchling: @jtauber ouch! Now I'll feel guilty every time.

@pydanny: @jtauber So very true!

@jtauber: @akuchling I disagree with him but couldn't resist the irony

@jtauber: @pydanny I was just trying to be ironic :)

@curtclifton: @jtauber LOL

@edwelker: Damn straight.

@TokyoDan: @jtauber How’s that? I RT when I think people who follow me might find the subject interesting. And it gets the originator more followers.

@jtauber: okay, I think I'm going to have to spell this out: I don't mind RTs. I was just RTing @gruber's dislike of RTs to be funny

@evilrob: @jtauber: you don't like RTs?

@brosner: RT @evilrob: @jtauber: you don't like RTs?

@20seven: RT @brosner: RT @evilrob: @jtauber: you don't like RTs?

@jtauber: okay, this whole RT thread needs to go on xkcd right now

@mak1e: RT @brosner: @evilrob: @jtauber: you don't like RTs?

@ericflo: RT @20seven RT @brosner: RT @evilrob: @jtauber: you don't like RTs?

@bryanveloso: RT @ericflo: RT @20seven RT @brosner: RT @evilrob: @jtauber: you don't like RTs?

UPDATE: Okay, I now agree with @gruber

by : Created on Jan. 28, 2009 : Last modified Jan. 28, 2009 : (permalink)

Serving Up User Contributed Media From A Separate Server

One commonly recommended practice in Django (although applicable elsewhere) is to serve up your static media from a different server than the one running Django for dynamic pages.

This becomes a slight challenge when you have user-contributed media (like allowing users to upload photos).

Here are some possibilities I can think of:

mount the media server from the Django server via NFS so write from Django directly to the media server
have the Django server write locally but then subsequently move the files to the media server via rsync (or similar)
like the previous case but to avoid 404s before the file has been moved, the relevant model has a flag for whether the file is ready or not and return an "in progress" message until the file is done (not sure how Django would know the file has successfully been moved, though, unless a Django-based worker process was doing it instead of just rsync)
similar to the previous two cases but initially store the image on (and serve it from) the Django server until the flag says the URL to the media server can be used
have Django running on the media server just for the purpose of receiving media POSTs (but no dynamic page generation)
having some other WSGI (or similar) process running on the media server to receive media (and then communicate with the Django server on the backend)

I guess S3-based solutions add some extra issues but ideas 2, 3 and 4 would be applicable.

Anyone have experiences (good or bad) with any of these? Any possibilities I'm missing?

by : Created on Jan. 26, 2009 : Last modified Jan. 26, 2009 : (permalink)

Pinax 0.5.1 Out

Pinax 0.5.1 has now been released. This is a minor bug fix release. Worth getting if you're on 0.5.0 but save the real excitement for the upcoming 0.7 :-)

You can read the release notes and download it from http://pinaxproject.com/download/

by : Created on Jan. 10, 2009 : Last modified Jan. 10, 2009 : (permalink)