In other news, Bob is busy working on his refactor of aeve now that he knows a lot more about AppleEvents than he used to. Hopefully he will get it to the point where he will give me SVN access to the code right around the time I finish embedding Python in an OSA component, and I can start helping him out by really banging on it and making sure it is possible to do everything that needs to be done.
Twisted stores the 'host' header sent by the browser in the request, and it stores the path segments that were used to locate the current Resource object in request.prepath. request.prepath is a list of strings indicating the URL segments leading up to the current Resource object. request.postpath is a list of path segments that have not yet been handled, but this is usually not terribly interesting.
So, it is possible, for example, to construct sibling urls, parent urls, and child urls by making a copy of request.prepath and performing list operations on it:
new = request.prepath[:]
new[-1] = 'someSibling.html'
Now that we have a list of path segments leading to a sibling Resource, we can construct a URL:
url = 'http://%s/%s' % (request.getHeader('host'), '/'.join(new))
While this works for simple cases, it breaks down when things get more complex. For example, the browser could be talking to the server over https, and while it is possible, it is non-trivial to detect this and construct a URL with the proper method. Duplicating this code throughout user-level code leads to many buggy implementations, with subtle problems that don't show up unless under very specific configurations.
For example, it is often desirable to run twisted.web behind a reverse proxy, such as using apache and the ProxyPass directive. (Someone please correct me if this is not the correct terminology.) In this case, the user browses to a URL served by the Apache server, and Apache forwards the request to the port upon which Twisted Web is listening. However, while it is doing this, it changes the "host" header from the header originally sent by the browser, to the host upon which Apache is running. Which means if we rely on the host header to construct our new URLs, they will reference the private proxy server rather than the public Apache server.
When I first began working on the Twisted project, one of the classes I wrote was called PathReferenceAcqusitionContext. It had this long name for historical reasons, and I shall simply refer to it as PathRef. One of the abilities of this object, which you could construct by calling the pathRef method on the Request, was to generate other PathRef objects by calling methods such as child, sibling, parent, etc. Then, once you had navigated to the conceptual URL location you desire, you could convert it to a URL string.
This implementation of PathRef was never documented, and had some other, unfortunate, unrelated properties which made it a perpetual pain in the ass. Glyph, finally fed up with the unused and problem-causing PathRef, removed it recently, and it is no longer in Twisted 1.0.6. Which is a good thing.
However, Glyph, finally seeing the need for a way to talk about URLs abstractly in a secure and convenient way, has kindly written the very good (in my opinion) Twisted/twisted/sandbox/paths.py, an implementation of all that was good about PathRef. I hope this gets moved into Twisted fairly soon, and a method for generating a URLPath representing the current Resource is added to the Request.
However, this method needs to be able to understand that the 'host' header is not necessarily the address to which the outside world refers. I decided to do a little experiment to see what Apache would do when performing a ProxyPass, and if we could determine where the outside world considers our application to live. When I was running Apache on port 8081 on my machine, and it was set up to forward requests to Twisted Web, I was able to observe that Apache had inserted an additional header in the process of forwarding the request:
The factory method on the request which generates a URLPath object should check to see if 'x-forwarded-host' has been set, and compensate accordingly. This should be a good solution to a long standing problem with using Twisted Web.
m = mail.mail()
for msg in m.messages:
That's just pseudocode, but you get the idea. All of the necessary moving parts are already in place to do this, as Jack Jansen noted in this message from 1999:
The bits that I'm doing differently are thanks to Python 2.2's generators, which make it very easy to write support for things like "for foo in bar".
Here's what I'm doing:
I added a __getattr__ to TalkTo that uses the _moduleName attribute I added last year to look for the given name in the correct module. If one is found, I then return a DelayedComponentItem with an additional parameter passed -- self, or the TalkTo instance itself. This allows the DelayedComponentItem to call count, which is a generic AppleEvent which returns the number of items of that type in the specified container (which can be None, to mean the application itself)
Then, I added a __len__ method to DelayedComponentItem that calls count to see how many items of that type there are.
Finally, I added an __iter__ method that does len(self) and returns actual ComponentItem instances for each item in the container. It works perfectly!
The only thing that tripped me up was the fact that AppleEvents like to count things starting with 1 and Python counts starting with 0. So the first ComponentItem returned won't actually be useful for anything and will generate an error if you try to use it. I realized this while I was driving home, and I look forward to fixing it tomorrow morning!
Good things are afoot.
I have been aware for some time now that IModel is an interface that is about preparing a model for display in a web browser -- much like Zope3's IView. It converts from whatever raw object interface is present on the original model, to an interface which exposes data suitable for traversal and insertion into a web template file.
So, tonight I went through and added the resource object as the first parameter to lookupSubmodel, getSubmodel, setSubmodel, getData, and setData. Changing the signature of methods which are widely used in a wide variety of situations is painful -- especially when you're changing an interface which is as fundamental to a system as IModel is to woven. However, thanks to the inspect module and some fancy use of positional optional arguments, I was able to come up with a solution that logs DeprecationWarnings whenever a method doesn't expect the request, and passes the request when a method does.
mktap web --path=/path/to/web/files --flashconduit=4321
I'll edit the post below to mention this.
For people who are curious about the bleeding edge of Woven development, I have been working on LivePage a bit more recently. LivePage allows asynchronous events both from the browser, and to the browser after the page has already been loaded. The current implementation in CVS requires Flash, but it is also technically possible to get this to work with an IFrame connection that the server never closes as the Output Conduit. I whipped up a little example so you can get a feel for where I am going with this, and give me feedback if you desire.
If you want to try this, save it as an rpy and view it with twisted.web using a tap created like this: (Latest CVS is probably required)
mktap web --path=/path/to/web/resources --flashconduit 4321
Make sure you put a slash after the .rpy, so the relative urls spit out by the webConduitGlue resolve correctly. For example, http://localhost:8080/simpleLivePageExample.rpy/
from twisted.web.woven import model, page, input, widgets
self.value = 0
def setUp(self, request, node, data):
def clicked(self, request, widget):
self.model.setData(self.model.getData() + 1)
resource = page.LivePage(ANumber())
resource.template = """<html>
<div model="value" view="aView">
<img src="http://www.google.com/images/logo.gif" model="value" view="Widget" controller="aController" />
<div view="webConduitGlue" />
bin/trial -m twisted.test.test_woven
Do not read this.
from __future__ import generators
def __init__(self, it):
self.it = it
if isinstance(item, str):
item = Text(item)
elif isinstance(item, tuple):
item = lmx2(item)
assert isinstance(item, Text) or
isinstance(item, lmx2), "wtf did you add %s to me for?" % item
def __init__(self, argument, **kw):
if isinstance(argument, str):
self._name = argument
self._children = 
self._attributes = kw
assert not kw, "Either use a fourple or a tag name and set of keyword arguments"
self._name, self._attributes, self._children, spare = argument
def __lshift__(self, other):
def __rshift__(self, other):
other.__dict__ = self.__dict__
other.__class__ = self.__class__
for item in self._children:
def __getitem__(self, item):
if isinstance(item, int):
def __setitem__(self, item, value):
if isinstance(item, int):
self._children[item] = value
self._attributes[item] = value
def __delitem__(self, item):
if isinstance(item, int):
def __getattr__(self, name):
if name == '_':
raise AttributeError("no private attrs")
return lambda **kw: self._children.append(lmx2(name, **kw))
return ('<%s ' % self._name) + ' '.join(['%s="%s"' % (key, value)
for key, value in self._attributes.items()]) + ('>...%s>' % self._name)
Man, I _told_ you not to read this. Why you always trippin like that?
from twisted.python import log
If you wish, you can also use any file in place of sys.stdout.