Youtube Together 1 — Start to Link Upload

(Originally posted July 20, 2013)

Having been working with Django for a while now, I felt like now would be a good time to learn a different python framework — Flask. Flask seems to boil all the things down to a fantastic minimal development experience. It seems perfect for quick apis, or apps. Django is fantastic for more complex apps with its large community and many extensions. Flask may be great for that too, it’s just not something I’ve seen right off the bat.

In order to learn Flask, I’ve decided to solve a mini-problem that I’ve encountered — managing music at a party. From my experience, many of the parties have just one computer hooked up to the sound system, and it’s somewhat of a free for all as to who can play the music and what they can play. And since this computer may or may not have a specific song the person wants to play. As a result, most of the music is played straight from youtube. This leads to having a terrible, or even non existant, queue system. The vision of this project is to have people be able to join a “room” where anyone can search youtube and upload a link to a song they want to hear and have it be put into the queue for playing. This is a rough outline and surely the project is able to change paths, but for the beginning, this seems like a good path.

Like all good python projects, this one starts with firing up a virtualenv.

1
virtualenv --distribute youtube-parties

Then

1
source youtube-parties/bin/activate

to activate the environment. Then you’re going to want to move to another directory and

1
2
3
mkdir youtube-parties
cd youtube-parties
pip install Flask

The last line there installs the framework we’re going to use to our pip environment. Now we want to start the app. Obviously we need to begin with the standard hello world app.

1
2
3
4
5
6
7
8
9
from flask import Flask
app = Flask(__name__)
@app.route("/")
  def hello():
  return "Hello World!"
if __name__ == "__main__":
  app.run()

You should see a message saying that there is a local server running on localhost:5000 and if you navigate there you should see “Hello World!”. Very nice and very pointless.

Now to templating. Create a directory for the templates and change to that directory.

1
2
mkdir templates
cd templates/

The templating system Flask uses is called Jinja2. Looking at the docs  and running through a few examples (the ones here), it seems very similar to Django’s templating system and can easily be picked up by those unfamiliar. Now we want to add the two files to template. The first is layout.html, which will be the base of the whole site, and index.html, which will be the base of the site.

layout.html

1
2
3
4
5
6
7
8
9
10
<!doctype html>
<html lang="en">
  <head>
    <title>Youtube for Parties</title>
  </head>
  <body>
    {% block content %}
    {% endblock %}
  </body>
</html>

index.html

1
2
3
4
5
6
7
{% extends "layout.html" %}
{% block content %}
<h1>Youtube Party!</h1>
{% endblock %}

The gist of is simple where the content block in index get put into the content block in layout. Now change app.py to

1
2
3
4
5
6
7
8
9
from flask import Flask, render_template
app = Flask(__name__)
@app.route('/')
def index():
 return render_template('index.html')
if __name__ == '__main__':
 app.run()

and run the app again, navigate to localhost:5000 and see the new html.

The next step is to allow uploads of urls to the site. Right now, since we just want to get something going for us, we don’t need to worry about malicious text or anything, and we’re even going to skip the database for the moment. After changes, the new app.py should look like

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
from flask import Flask, render_template, url_for, request, redirect
app = Flask(__name__)
dumb_db = []
@app.route('/', methods=['GET''POST'])
def index():
  if request.method == 'POST':
  dumb_db.append(request.form['youtube-url'])
  return render_template('index.html', queue=dumb_db)
@app.route('/upload-url', methods=['POST'])
def upload_url():
  dumb_db.append(request.form['youtube-url'])
  return redirect(url_for('index'))
if __name__ == '__main__':
  app.debug = True
  app.run()

index.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
{% extends "layout.html" %}
{% block content %}
<h1>Youtube Party!</h1>
{% for q in queue %}
 <div>
   {{q}}
 </div>
{% endfor %}
<form method="post">
  <input type="text" name="youtube-url">
  <button type="submit">Upload</button>
</form>
{% endblock %}

If you restart the app, you should be able to see a form, and if you type things into the form, and submit, you should see a list of the printed above the form. Some notes about the new code.

1) We need some more imports for app.py.

2) dumb_db is just a list where the values from the submit form are appended. If you restart the app, the values go away and you get a new, blank list.

3) The render_tempate call in index has another variable at the end. This passes the value on to the template for use.

4) The app route decorator for index has a method keyword. This allows for a POST method to go through when the default is only GET request. Note that if you didn’t mention GET along with POST, you wouldn’t be able to process a GET request.

5) We added debug = True in the app module at the bottom. This allows for printouts if there are server errors. Very important if you don’t want to use a whole bunch of pdb.

6) In index.html, we loop the queue in a very pythonic way, printing the item each loop

The last thing we want to do for the time is to make the site look halfway decent. In production, you’d want a webserver (nginx or apache) to do this for you. In development though, all you have to do is create a folder called static in the same folder as app.py, and the development server will serve the files automatically at the uri /static/*. Throughout the tutorial, I’m going to be using the css framework Foundation, made by Zurb. The reason I’m doing this is because I’m getting pretty sick of seeing Bootstrap. Bootstrap is fantastic, but I’m looking for a little change up in scenery. Go to http://foundation.zurb.com/ and click the download button. Unzip the file and move the contents to that newly created static/ folder.

To review, we started with nothing, created a virtualenv, added flask, created a first route, added templates, added support for POSTing urls, and marked up the html into something that looks halfway decent.

Next time, we’ll add a database to the url’s have persistance. We’ll also add some error checking to make sure that the links being added are valid.

Python datetimes and Django’s humanize

(Originally posted July 16, 2013)

One of the important things to know when dealing with deadlines is how long you have left to complete them. The complete, but very annoying to deal with, python datetime module is able to calculate these times. But displaying them on a webpage isn’t exactly pleasing to the eye. For example:

5 days, 10:14:57.121807

is what it would look like if you just printed it out. This is fine if your looking for pure information, but not good at all for anything in production since it takes longer than half a second to figure out, and we don’t want users to have to think very hard about something like this. Luckily, Django has template filters in their “humanize” library that converts unpleasing text like that into something human readable. To do this, all you do is throw a filter after the time and you should get the humanized time instead. If I returned the time difference between the final time and the current time,

1
2
time_difference = end_time - cur_time
return time_difference

I should be able to run that through the filter and get the desired format.

{{time_difference|naturaltime}}

Unfortunately, for the code I was using, this wasn’t working. The filter didn’t have any effect on the time object. Time to dig into Django’s source code. After finding the humanize file, the first thing I noticed was

1
2
if not isinstance(value, date): # datetime is a subclass of date
    return value

This check makes sure that the value being passed in the filter is of type date, or datetime. Since I was returning a difference of datetimes, I was actually passing a timedelta object, which would fail this check and return the original value, which was the error I was getting. Like I said, python datetimes are annoying.

To fix this error, I simply had to make sure that my function returned a datetime. Since a datetime – timedelta returns a datetime, I changed my code to

1
2
time_difference = end_time - cur_time
return cur_time + time_difference

And just like that, the filter was working as expected. Kind of a silly error considering that when you use the naturaltime filter, you’re always going to end up with a timedelta, so you should be able to use one off the bat. Either way, humanizing the times is way better than the clunky normal time display.

Random Primary Keys

(Originally posted July 15, 2013)

When you create an object in Django and save it, Django automatically assigns it a primary key (pk) to the object. This is the

There are two ways to do this in general.

One way to do this is to obfuscate the standard, auto-incrementing integer primary key values that Django’s model ORM provides out of the box. One way to do this method is to use something like AES encrypting that is provided up pycrypto. From there, you can create encrypt and decrypt methods that you wrap when you go to retrieve models from the database. Unfortunately, this method requires you remember to call this function every time you access an object. Hopefully you remember, and that would probably get really annoying.

The other method is to create a primary key yourself on object creation, and then use that throughout the app. To do this, you need to override the model’s save method which, when the object is first saved, will generate a random value to use as the primary key, check to see if there is something with the value previously assigned, and if not, use that as the value’s key.

The code is as follows:

1
2
3
4
5
6
7
8
9
10
11
def save(self*args, **kwargs):
    if not self.pk:
        while True:
            self.pk = int(str(uuid.uuid4().int)[0:6])
            try:
                super(Goal,self).save(*args, **kwargs)
                break
            except IntegrityError:
                continue
    else:
        super(Goal,self).save(*args, **kwargs)

I prefer overriding the save method to generate a random id. It’s a one stop definition where you write the code once and don’t have to think about it again, which is always a good thing. Also, it allows you to get a very large number of objects just by changing the length of the slice you use to generate the key.

Simple Django, Celery, RabbitMQ start

(Originally posted July 12, 2013)

Considering email is a big part in reminding people to do things, we need a way to distribute the sending of emails somewhat asynchronously. To do this, the consensus way in Django is to use Celery with RabbitMQ. After sifting through a bunch of different posts on the subject, I decided to write a little about how I got the initial setup going. I’m using a mac with homebrew installed.

The first thing to do is install and get RabbitMQ going.


brew install rabbitmq

With rabbitMQ installed, we now want to get the rabbit server running.

1
sudo rabbitmq-server -detached

will start the server in the background.

To make sure it’s running, run the command,

1
sudo rabbitmqctl status

You should see a long nested bracket output with information regarding the status of the server. If you see that, you can be sure it is running. To stop the server, never just kill the pid, rather run

1
sudo rabbitmqctl stop

and this will stop the server. Note that it’s important to run all these commands with sudo. Otherwise, you might see a message of an erlang dump meaning there was an error.

Celery

1
$ brew install celery

will install celery! We then want to use pip to install the django connector

1
$ pip install django-cellery

Settings.py

Add ‘djcellery’ to installed apps and run (since you’re hopefully using South),

1
$ python manage.py migrate djcellery

We also need to add a few other things to get the django connection finished. In settings.py, add the following lines

1
2
3
4
5
6
import djcelery
djcelery.setup_loader()
CELERY_IMPORTS = ('<project_name>.tasks',)

where <project_name> is the name of your projects. The CELERY_IMPORTS line can actually be the path to any file that has tasks defined, whether it is a tasks.py file or not.

Tasks.py

Conventionally, it tasks for cellery are in a file called tasks.py which can reside anywhere in the project tree. For this simple starter task, the task will run every minute, and print a simple string to the command line. This takes no extra configuration and is simple to see if it is all working.

1
2
3
4
5
6
from celery.task.schedules import crontab
from celery.decorators import periodic_task
@periodic_task(run_every=crontab())
def project_tasks():
    print 'hello world!'

Wherever you put this, just make sure that settings.py has the correct path so that it can find the definition.

Running celery

The last thing to do is to run celery and see the output!

1
python manage.py celeryd -v 2 -B -s celery -E -l INFO

After letting it run for a minute, you should see, along with some other celery information, hello world! being printed!

Obviously this isn’t that impressive, but now you know you’re able to run python code periodically. Now go do something more impressive than hello world.

Developer, Golf on the Mind writer, Packers, Brewers, Bucks fan