Nathan Ostgard

Game Objects is a collection of 2D and 3D maths classes, and algorithms for helping in the creation of games with Python.

Will McGugan’s Game Objects library has been invaluable for my dabblings in Python game development. It provides optimized objects for common graphics math (3D vector class, matrix class, etc.). The objects are flexible and framework independent, so you can use them with the platform of your choice, be it pygame, PyOpenGL, or DirectPython.

It’s a somewhat new project, and the documentation a bit sparse, but the library is pretty straightforward and he’s been updating it regularly.

Relevant links:

• Project Page
• Documentation
• Read more about Game Objects on Will’s blog.

Decided to move some of my vertex array code over to ARB_vertex_buffer_object, to see if I could get a bit of a speed boost out of PyOpenGL. Much to my dismay, these functions don’t have the sexy extension wrappers I talked about before, so it was a bit of a chore to get it all working.

Here’s a little wrapper class I wrote to make things easier:

from OpenGL.GL import *
from OpenGL.raw import GL
from OpenGL.arrays import ArrayDatatype as ADT

class VertexBuffer(object):

  def __init__(self, data, usage):
    self.buffer = GL.GLuint(0)
    glGenBuffers(1, self.buffer)
    self.buffer = self.buffer.value
    glBindBuffer(GL_ARRAY_BUFFER_ARB, self.buffer)
    glBufferData(GL_ARRAY_BUFFER_ARB, ADT.arrayByteCount(data), \
                 ADT.voidDataPointer(data), usage)

  def __del__(self):
    glDeleteBuffers(1, GL.GLuint(self.buffer))

  def bind(self):
    glBindBuffer(GL_ARRAY_BUFFER_ARB, self.buffer)

  def bind_vertexes(self, size, type, stride=0):
    self.bind()
    glVertexPointer(size, type, stride, None)

  ... snipped for length ...

Download buffers.py.

So, how to use it? Let’s say you have a Python list with a bunch of vertexes with float x, y, and z components:

verts = [[x, y, z], [x, y, z], [x, y, z], ...]

You’ll have to use NumPy or an equivalent to convert it to a PyOpenGL compatible array:

import numpy
numpy_verts = numpy.array(verts, dtype=numpy.float32)

Create the VertexBuffer object with it:

buffer = VertexBuffer(numpy_verts, GL_STATIC_DRAW)

Use it in your day to day rendering:

glEnableClientState(GL_VERTEX_ARRAY)
buffer.bind_vertexes(3, GL_FLOAT)
glDrawElementsui(GL_TRIANGLES, indexes)

They’re not shown in the snippet, but I’ve also defined bind_colors, bind_edgeflags, bind_indexes, bind_normals, and bind_texcoords, as shortcuts for the rest of the GL array functions.

When I have Django URL patterns like:

urlpatterns = patterns('',
  (r'^(?P<foo_id>\w+)/$', 'myproject.myapp.views.show'),
  (r'^(?P<foo_id>\w+)/edit/$', 'myproject.myapp.views.edit'),
  (r'^(?P<foo_id>\w+)/delete/$', 'myproject.myapp.views.delete'),
)

I always end up with views like:

def show(self, foo_id):
  foo = get_object_or_404(Foo, id=foo_id)
  ...

def edit(self, foo_id):
  foo = get_object_or_404(Foo, id=foo_id)
  ...

def delete(self, foo_id):
  foo = get_object_or_404(Foo, id=foo_id)
  ...

Having the same few lines at the top of every function in makes me feel dirty. You can clean this up with a Middleware class, replacing foo_id with the actual object before calling the view.

myproject/myapp/middleware.py looks like this:

from django.shortcuts import get_object_or_404
from someapp.models import Foo

class FindObjects:
  def process_view(self, request, view_func, view_args, view_kwargs):
    if 'foo_id' in view_kwargs:
      view_kwargs['foo'] = get_object_or_404(Foo, id=view_kwargs['foo_id'])
      del view_kwargs['foo_id']

I include it in settings.py:

MIDDLEWARE_CLASSES = (
  ...
  'myproject.myapp.middleware.FindObjects',
)

And now my views look like this:

def show(request, foo):
  ... yay! do stuff with foo! ...

Very useful for views with many parameters, or views with multiple optional parameters. In one project, I have views which process data for three different situations: data in a metropolitan area, data in a city, or data in a city within a metro. My middleware for that looks like this:

from django.shortcuts import get_object_or_404
from phuce.metros.models import City, Metro

class FindObjects:
  def process_view(self, request, view_func, view_args, view_kwargs):
    city = metro = None
    if 'metro_link' in view_kwargs:
      metro = get_object_or_404(Metro, link=view_kwargs['metro_link'])
      del view_kwargs['metro_link']
      view_kwargs['metro'] = metro
    if 'city_link' in view_kwargs:
      if metro:
        city = get_object_or_404(City, metro=metro, \
                                 metro_link=view_kwargs['city_link'])
      else:
        city = get_object_or_404(City, link=view_kwargs['city_link'])
      del view_kwargs['city_link']
      view_kwargs['city'] = city

It’s much tidier without that monster in my views.

Normally if you wanted to get access to an OpenGL extension, you’d have to do something like:

from ctypes import *    
from OpenGL import platform
gl = platform.OpenGL
glGetShaderiv = gl.glGetShaderiv
glGetShaderiv.argtypes = [c_int, c_int, POINTER(c_int)]

Gross. Luckily, PyOpenGL version 3.0.0a5 and later will automatically map ARB standard functions for you when they are available. Now all you have to do is:

from OpenGL.GL import *

And everything will be magically available to you, wrapped so you don’t have to use ctypes to create c_char_p or c_int variables to pass them.

Watch out, though! The functions won’t be mapped correctly if your display context hasn’t been created yet. Always make sure to import OpenGL after the pygame display has been initialized.

# first initialize pygame
import pygame
from pygame.locals import *
pygame.init()
pygame.display.set_mode((800, 600), OPENGL | DOUBLEBUF)

# now you can import it
from OpenGL.GL import *

I would love to know how to manually tell PyOpenGL to re-map everything after initializing pygame, though. I don’t like having to reposition the import like that. Nathan Gray has a deep reload replacement for the reload builtin, which might work, but I’m not sure if that’s the best solution to this problem…

There’s no way in Django (that I’ve found) to render a field’s name in a template. This means you end up with <th>Field Name</th> all over your templates. Why hello there, DRY violation!

The fields are stored in model._meta.fields, but templates don’t allow you to access variables which start with an underscore. I’ve got two little utility functions I wrote for myself to generate a dict of labels I can use in my templates:

def get_labels_for(model, cap=True, esc=True):
  from django.template.defaultfilters import capfirst
  from django.utils.html import escape
  labels = {}
  for field in model._meta.fields:
    label = field.verbose_name
    if cap:
      label = capfirst(label)
    if esc:
      label = escape(label)
    labels[field.name] = label
  return labels

def with_labels(context, cap=True, esc=True):
  from django.db.models import Model
  result = context.copy()
  for k, v in context.iteritems():
    if isinstance(v, Model):
      result[k + '_labels'] = get_labels_for(v, cap, esc)
    elif hasattr(v, '__getitem__') and len(v) > 0:
      if isinstance(v[0], Model):
        result[k + '_labels'] = get_labels_for(v[0], cap, esc)
  return result

The parameters:

• model can be a model class or a model instance.
• If cap is True, the first letter of each label will be capitalized.
• If esc is True, the labels will be escaped for HTML.

So, in your view:

def some_view(request, foo_id):
  foo = get_object_or_404(Foo, id=foo_id)
  context = {'foo': foo, 'foo_labels': get_labels_for(foo)}
  return render_to_response('foo.html', context)

And in your template:

{{ foo_labels.bar }}: {{ foo.bar }}

with_labels works the same way, except you can just surround your context with it:

def some_view(request, foo_id):
  foo = get_object_or_404(Foo, id=foo_id)
  bars = Bars.objects.all()
  context = {'foo': foo, 'bars': bars}
  return render_to_response('foo.html', with_labels(context))

It will detect the models and lists of models in the context and add foo_labels and bars_labels to the context.

In Python:

import os.path, random, string
def random_filename(chars=string.hexchars, length=16, prefix='', suffix='', \
                    verify=True, attempts=10):
  for attempt in range(attempts):
    filename = ''.join([random.choice(chars) for i in range(length)])
    filename = prefix + filename + suffix
    if not verify or not os.path.exists(filename):
      return filename

>>> random_filename()
'DC53e02B08eF47e9'
>>> random_filename(chars='hi', length=32)
'hhhihihhhiiihhhihhiiiiiihhhhiihh'
>>> random_filename(prefix='username', suffix='.txt')
'username7dbd29aBdD25BeB9.txt'

# returns None if it can't find a filename
>>> open('xxx', 'w').close()
>>> str(random_filename(chars='x', length=3))
'None'

In Ruby:

def random_filename(opts={})
  opts = {:chars => ('0'..'9').to_a + ('A'..'F').to_a + ('a'..'f').to_a,
          :length => 16, :prefix => '', :suffix => '',
          :verify => true, :attempts => 10}.merge(opts)
  opts[:attempts].times do
    filename = ''
    opts[:length].times { filename << opts[:chars][rand(opts[:chars].size)] }
    filename = opts[:prefix] + filename + opts[:suffix]
    return filename unless opts[:verify] && File.exists?(filename)
  end
  nil
end

When most pages in a site require authentication, decorating all the views with @login_required can be annoying. You can reverse the default behavior by creating a custom middleware class:

import urllib
from django.conf import settings
from django.contrib.auth import REDIRECT_FIELD_NAME
from django.http import HttpResponseRedirect

def allow_anonymous(view_func):
  view_func.allow_anonymous = True
  return view_func

class RequireLogin:
  def process_view(self, request, view_func, view_args, view_kwargs):
    if request.path != settings.LOGIN_URL and \
      not request.user.is_authenticated() and \
      not getattr(view_func, 'allow_anonymous', False):
      url = '%s?%s=%s' % (settings.LOGIN_URL, REDIRECT_FIELD_NAME, \
                          urllib.quote(request.get_full_path()))
      return HttpResponseRedirect(url)

That’s an ugly block of code but it’s not too complex. allow_anonymous is a function decorator, like login_required. It just tags the function to tell the middleware that authentication isn’t required. The RequireLogin class verifies that the user is logged in. If not, and if the function is not decorated with allow_anonymous, it redirects to settings.LOGIN_URL.

So put that code in a file in your project – let’s say, yourproject/yourapp/middleware.py. Then open settings.py and add "yourproject.yourapp.middleware.RequireLogin" to MIDDLEWARE_CLASSES. This tells Django about your new middleware class, and RequireLogin.process_view will be called any time a view is about to be rendered.

Now you can use it in your view:

from yourproject.yourapp.middleware import allow_anonymous

def some_private_view(request):
  # won't be accessible unless user is logged in
  return HttpResponse('Hello, user!')

@allow_anonymous
def some_public_view(request):
  return HttpResponse('Hello, world!')

If all is working correctly, some_private_view should ask for a login, but some_public_view will allow viewing without it.

Update: I’ve updated the code to fix a bug when using django.contrib.auth.views.login for your login view. As you can’t mark this view function with @allow_anonymous, it would infinitely redirect back to it. Oops! Thanks for pointing it out, Phil.

Simon Willison linked to an article which argues:

Warnings cause us to lose our work, to mistrust our computers, and to blame ourselves. A simple but foolproof design methodology solves the problem: “Never use a warning when you mean undo.” And when a user is deleting their work, you always mean undo.

The post spawned a discussion on undo techniques for Django. I decided to implement one method and post the results here. It only offers undo for deleting, and not for editing. Other than that, I like it.

How it works

It’s a pretty simple concept: add a trashed_at field to your model, with the default value of None. When delete() is called on an object, if trashed_at is None, set it to the current time but don’t delete it. If it’s not None, actually delete it from the database.

Continue reading...

Sometimes users want to bring text from an editor like Word into your web forms. You will often find nasty little characters hiding in the text, like ’\u2022’ (a.k.a. the notorious bullet). These characters will normally throw errors if you try to convert them to ASCII:

>>> u'\u2022'.encode('ascii')
Traceback (most recent call last):
  File "<console>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode ... (yadda yadda)

To sanitize these strings and make them XML/HTML safe:

>>> u'\u2022'.encode('ascii', 'xmlcharrefreplace')
'•'

It translates the invalid characters into their XML equivalents. Woo! You can also use 'ignore' or 'replace' (replaces with ?):

>>> u'\u2022'.encode('ascii', 'ignore')
''
>>> u'\u2022'.encode('ascii', 'replace')
'?'

If you’re getting nasty Unicode errors from your templates in Django now that they’ve merged the Unicode branch, this might help as a quick fix.

Foo.objects.get(keywords__contains='bar') will make your database cry when you run it against a 30 million row table. At times it becomes necessary to segment a large table into many smaller tables.

Say you have these models:

class City(models.Model):
  city = models.CharField(maxlength=255)
  state = models.USStateField()

class Listings(models.Model):
  name = models.CharField(maxlength=100)
  street = models.CharField(maxlength=100)
  city = models.ForeignKey(City)
  zip = models.CharField(maxlength=10)
  keywords = models.CharField(maxlength=255)

The listings table has gotten huge. Users only search within a single city, so you want to break the listings table into a table for each city. Instead of yourapp_listing, you now have yourapp_listing_1, yourapp_listing_2, etc. All the listings are in the matching table for their city.

I couldn’t find a documented way to make db_table dynamic in Django. So, how do you get Django to use the right table when you’re querying for listings? Here’s how I did it:

class ListingManager(models.Manager):
  def get_table_for(self, city):
    '''
    someapp_listing if city == None
    someapp_listing_012 if city.id == 12
    '''
    table = '_'.join((self.model._meta.app_label,
                      self.model._meta.module_name))
    if city:
      table += '_%03d' % city.id
    return table

  def in_city(self, city):
    self.city = city
    self.model._meta.db_table = self.get_table_for(city)
    return self

class Listing(models.Model):
  city = models.ForeignKey(City)
  objects = ListingManager()

  def delete(self):
    Listing.objects.in_city(self.city)
    super(Listing, self).delete()

  def save(self):
    Listing.objects.in_city(self.city)
    super(Listing, self).save()

It’s simple to use:

>>> city = City.objects.get(id=2)
>>> Listings.objects.in_city(city).all()

Edit 2007/08: Simplified the code. When I first wrote this I replaced db_table with a subclass of str that called ListingManager.get_table_for. It was overkill, and didn’t always work as expected (e.g., 'foo' + Listing._meta.db_table didn’t work).

Did a bit of running around today to get Django sending email via Gmail. It’s simple once you figure it out.

If you’re running 0.96, upgrade to the latest development version or apply the patch from ticket #2897. 0.96 does not support TLS, which Gmail requires. Then add the appropriate values to settings.py:

EMAIL_USE_TLS = True
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = 'youremail@gmail.com'
EMAIL_HOST_PASSWORD = 'yourpassword'
EMAIL_PORT = 587

You can use the shell to test it:

>>> from django.core.mail import send_mail
>>> send_mail('Test', 'This is a test', to = ['youremail@somewhere.com'])

Edit: Bryan commented that send_mail is deprecated. Use EmailMessage instead:

>>> from django.core.mail import EmailMessage
>>> email = EmailMessage('Hello', 'World', to = ['youremail@somewhere.com'])
>>> email.send()

To install Python, pygame, and PyOpenGL on Windows:

1. Download Python here and install it.
2. Download pygame here and install it. The “Windows” section has an executable installer (e.g. pygame-1.7.1release.win32-py2.5.exe).
3. You should also download and install Numeric from the pygame page.
4. Download ez_setup.py and save it to a folder outside your Python installation (e.g. C:\).
5. Open a command prompt, change to the folder you saved ez_setup.py to, and run it:
   c:\python\python c:\ez_setup.py
   It will download the easy_install executables and put them in your Python\Scripts folder.
6. Now install PyOpenGL using easy_install:
   c:\python\scripts\easy_install PyOpenGL

All done!