Custom Search

Tuesday, March 6, 2012

Configuration with Python class objects

Another old bit of writing. While the space used for the demonstration - web applications - is now thoroughly dominated by template engines and frameworks, the core concepts are still valid. Resurrected by request.

Abstract

The requirements for configuring a complex program include such things as separate namespaces for different parts of the program; the ability to configure an object as like that one, only... and the ability to create new configurations on the fly. In general, the configuration facility should provide programming language constructs for advanced users. All this should be possible while keeping the language simple enough for unsophisticated users.

Python provides a modern, object oriented language for building configuration values, with a regular and straightforward syntax. Its class objects can hold configuration information in class variables. Classes can inherit from each other, and hence can look be made to resemble each other, or prototype. Finally, classes can be instantiated to create an object that acts like a configuration object created at run time.

The configuration problem

Writing software configuration tools has always been a balancing act between the needs for power and flexibility, and the need to make program configuration simple enough for non-technical users. Much of this has been solved on the desktop with the appearance of standardized APIs and configuration wizards, but it has resurfaced with a vengeance in software for the WWW.

A typical web program consists of a collection of interfaces to data objects, which have similar but not identical configurations, accessed by users who have different capabilities - some can write, some can only read, and others can create - and different views of the same objects. Worse yet, some of the configured objects may need slightly different configurations, created dynamically at run time. This is all coupled with the need for the HTML generated by the application to be created by a nontechnical designer. Thus the HTML must be part of the configuration, not part of the source code. While a configuration wizard might be able to handle these needs, many WWW applications are custom built in a highly competitive environment that does not allow time for such frills.

Python provides a very effective tool for configuring such programs. While it may seem like overkill to use an object-oriented programming language as a configuration language, Python has a history of use as an embedded scripting language for non-technical users. See [Anderson], [Beazley] and [Hinsen] for recent examples. A little artful work with class statements allows the namespace to be partitioned. This leaves having to quote strings that would be unquoted tokens in a custom configuration language as the only serious wart.

Configuration objects as a solution

The fundamental goal of configuration is to allow the user to control the appearance and properties of the programs objects without having to write program code. In an object-oriented programming environment, the simple way to do this is to attach a configuration object to each program object, where the attributes of the configuration object are the values the program object uses to determine it's appearance and behavior.

This leads to a parallel object structure, with the programmer creating one class hierarchy and the configuration creating the second. The objects created by the programmer define behavior. The objects the configuration creates select a behavior, and control the presentation.

The configuration objects connect to each programmed object in the system in a pattern related to both the State and Command patterns [Gamma]. This pattern, which I refer to as the Configuration pattern, when combined with Python's object model, neatly solves the configuration problem.

The problem then becomes how the configuration file is used to create these objects. In general, the only objects that the python interpreter creates without programming are class objects. However, class objects have attributes that can be read just like object attributes. Since each class creates a separate namespace with a simple syntax, this makes them an ideal solution to this problem.

Details from a working example

Having described the solution, we now present it in detail with extracts from a working program that uses this method for program configuration. The program in question is a web-based chat server. All of the HTML presented by the system comes from the configuration files.

The chat servers basic interaction with the user is to present a list of rooms, each of which represents an area with some set of discussion rules. The user selects one of the rooms and is then shown the contents of the room. The first part of the room is a control panel. Part of this is used to control how the user sees the room, and part to input messages to the room and control the appearance of the input messages. The second part of the room is the list of messages that user sees.

The basic objects for this application are obvious, and implemented as you would expect. There are user objects that reflect the users control panel settings. There are message objects that are used to hold messages. There are room objects that hold specifics about the room, such as a list of active users and messages they can see.

As described above, parallel to this class hierarchy is a configuration class hierarchy. Every object in the system has a configuration object attached that control's it's behavior and presentation. Behavior is controlled by the configuration objects attribute settings. Presentation is controlled by a configuration object attribute with a string value, which uses the Python % operator, usually with the object being presented as the right operand. This means that each class needs to provide the Python dictionary interface for that operator.

The simplest object in this application is a message. The message class definition is:

class message:
 "A message in a room."
 def __init__(my, dispatch, room):
  my._user = dispatch.request.session
  my.time = time.time()
  my.fromid = my._user['userid']
  my.nickname = my._user['nickname']

  # Message text
  my.message = string.strip(dispatch.form['message'].value)
  my.text = config.message.format % my
  del my._user

 def __str__(my): return my.text

 def __getitem__(my, key):
  if key == 'time':
   return time.ctime(my.time)
  if hasattr(my,  key):
   return getattr(my, key)
  return my._user[key]

The one atypical feature of the message class is that the HTML to display is created when the object is created and then returned by the __str__ method. It does this because the HTML representation of the message never changes, and it could be displayed thousands of times. Most other objects change between displays, and hence rebuild the HTML when they are displayed.

The other features - an __str__ method to return the HTML, and __getitem__ that calculates some values, uses attributes if they exist, and otherwise pulls values from a component object - are standard for objects in this system. A typical configuration for a message is:

class message:
 format = '''%(format_image)s%(format_status)s%(face)s<br>
   %(message)s<br>From %(location)s on %(time)s
   Back to <a href="#top">top</a>.'''
 to_wrapper = '<font color=red>%s</font>'
 from_wrapper = '%s'

The format attribute is used in the class and completely controls the format of messages. Most of the values used in it come from the _user object. The two _wrapper attributes are used to wrap messages to and from a user so they can be distinguished without having to rebuild the HTML every time the message is displayed.

As can be seen, the text used to build configuration objects is relatively simple, and doesn't require programming skills. In practice, a few strings need to be quoted where they wouldn't in a custom configuration language, and the comma required to distinguish a singleton tuple from an expression are the only real drawbacks in the syntax.

Showing the flexibility of this technic calls for a larger example. In a system of chat rooms, the rooms themselves are the most important objects. They are where messages are displayed, and users move in and out of them. This particular system also provides a couple of different types of dynamically created rooms, all of which have to be dealt with.

The configuration objects to deal with a collection of rooms looks like:

class room:
 # Format to display the body of a room. Does not
 # include messages! This has title &/ name as
 # formatting items.
 body_format = '<h3 align=center id="top">%(title)s</h3>%(header)s'

 # Description of the room, displayed in body_format
 # as %(header)
 header = ""

 # Format to display the head of a room. Same
 # formatting items as above.
 head_format = '<TITLE>Chat Room - %(title)s</TITLE>'

 # Maximum number of lines/chars in a message in
 # this room. 0 or None means no limit
 maxlines = 20
 maxchars = 2000

 # How long a room will hang around unused.
 timeout = 15 * 60

# Public rooms timeout VERY SLOWLY
class public(room):
    timeout = 24 * 60 * 60

# Each room in rooms must have an entry here with a title.
# Other values for the room config can be overridden in the
# class statement for each room
class friends(room): title = "The Friends Room"
class lounge(room):
 title = "The Lounge"
 header = """<P>This room is the <b>beta</b> chat room.
  It's where we test new features before putting them
  online everywhere.</P>"""
class poetry(room):
 title = "The Poetry Room"
 maxlines = 100
 maxchars = 10000

# List rooms created at startup
rooms = (lounge, poetry, friends)

As with the message configuration object, the bulk of this configuration object is taken up with formatting the resulting HTML. A few attributes at the end control the behavior of each room. Public rooms are created on the fly, and timeout slower than the default - 24 hours instead of 5 minutes - and hence override the timeout attribute. The first real room is the Friends room, which sets the title for the room. The Lounge adds a note to the top of the room display by overriding the header attribute. The Poetry room allows very long messages by overriding both the maxlines and the maxchars attributes.

The last feature this example can illustrate is the ability to create new configurations on the fly. The chat system lets users enter a room name to create a new room dynamically. The configuration for such a room is public room configuration, with a title added. So the configuration object can be created by creating an instance of a config.public object, and then setting it's title attribute:

 newroom = room()
 newroom.config = config.public()
 newroom.config.title = newtitle

Which creates a new room and attaches a configuration object with the new title.

Summary

As can be seen, a configuration pattern combined with Pythons object model provides a solution to each element of the configuration problem. Python itself provides the configuration language and programming constructs. The Python class construct provides namespaces and the ability to create configurations that inherit from other configurations. The ability to instantiate configuration classes and change them provides this same ability dynamically.

References

[Anderson]
Charles Anderson. "Experiences with Extension Programing and Scripting in Python" In Proceedings of the Sixth International Python Conference. 1997
[Beazley]
David M. Beazley and Peter S. Lomdahl. "Feeding a Large-scale Physics Application to Python" In Proceedings of the Sixth International Python Conference. 1997
[Gamma]
Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns - Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.
[Hinsen]
Konrad Hinsen. "The Molecular Modeling Toolkit: as case study of a large scientific application in Python" In Proceedings of the Sixth International Python Conference. 1997