Simple Django, Celery, RabbitMQ start

(Originally posted July 12, 2013)

Considering email is a big part in reminding people to do things, we need a way to distribute the sending of emails somewhat asynchronously. To do this, the consensus way in Django is to use Celery with RabbitMQ. After sifting through a bunch of different posts on the subject, I decided to write a little about how I got the initial setup going. I’m using a mac with homebrew installed.

The first thing to do is install and get RabbitMQ going.


brew install rabbitmq

With rabbitMQ installed, we now want to get the rabbit server running.

1
sudo rabbitmq-server -detached

will start the server in the background.

To make sure it’s running, run the command,

1
sudo rabbitmqctl status

You should see a long nested bracket output with information regarding the status of the server. If you see that, you can be sure it is running. To stop the server, never just kill the pid, rather run

1
sudo rabbitmqctl stop

and this will stop the server. Note that it’s important to run all these commands with sudo. Otherwise, you might see a message of an erlang dump meaning there was an error.

Celery

1
$ brew install celery

will install celery! We then want to use pip to install the django connector

1
$ pip install django-cellery

Settings.py

Add ‘djcellery’ to installed apps and run (since you’re hopefully using South),

1
$ python manage.py migrate djcellery

We also need to add a few other things to get the django connection finished. In settings.py, add the following lines

1
2
3
4
5
6
import djcelery
djcelery.setup_loader()
CELERY_IMPORTS = ('<project_name>.tasks',)

where <project_name> is the name of your projects. The CELERY_IMPORTS line can actually be the path to any file that has tasks defined, whether it is a tasks.py file or not.

Tasks.py

Conventionally, it tasks for cellery are in a file called tasks.py which can reside anywhere in the project tree. For this simple starter task, the task will run every minute, and print a simple string to the command line. This takes no extra configuration and is simple to see if it is all working.

1
2
3
4
5
6
from celery.task.schedules import crontab
from celery.decorators import periodic_task
@periodic_task(run_every=crontab())
def project_tasks():
    print 'hello world!'

Wherever you put this, just make sure that settings.py has the correct path so that it can find the definition.

Running celery

The last thing to do is to run celery and see the output!

1
python manage.py celeryd -v 2 -B -s celery -E -l INFO

After letting it run for a minute, you should see, along with some other celery information, hello world! being printed!

Obviously this isn’t that impressive, but now you know you’re able to run python code periodically. Now go do something more impressive than hello world.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s