Amara XML Tools is a collection of Pythonic tools for XML data binding. Not just tools that happen to be written in Python, but tools built from the ground up to use Python idioms and take advantage of the many advantages of Python over other programming languages.
Amara builds on 4Suite, but whereas 4Suite focuses more on literal implementation of XML standards in Python, Amara adds a much more Pythonic face to these capabilities. The combination ensures standards compliance within expressive Python form.
The main component of Amara is:
Other components are:
If you do not have 4Suite, you can still install Amara in one go from one package, called Amara-allinone. This package has all the dependencies you need, besides Python itself.
If you do have 4Suite you can just download the basic Amara package.
In either case, Amara uses distutils so installation is a simple matter of
python setup.py install.
The following example shows how to create a binding from a simple XML file, monty.xml. (All example files mentioned are available in the demo directory of the Amara package.)
First, the contents of monty.xml:
<?xml version="1.0" encoding="utf-8"?>
<monty>
<python spam="eggs">
What do you mean "bleh"
</python>
<python ministry="abuse">
But I was looking for argument
</python>
</monty>
Now the code to create the binding:
import amara
doc = amara.parse('monty.xml')
doc is the data binding result, an object representing the XML. Since
I fed the binder a full XML document, I get back an object
representing the document itself which then has a member representing
the top-level element.
It's that simple. In order to get the value "eggs" (as a Python Unicode object) you can write:
doc.monty.python.spam
Or in order to get the value "But I was looking for argument" you can write:
doc.monty.python[1]
The parse function can be passed a file-like object (stream) as well
as a string. There is also a parse_path function for parsing
XML retrieved from local files and URLs.
You can pass amara.parse a string (not Unicode object) with the
XML content, an open-file-like object, a file path or a URI.
The following example shows how to create a binding from an XBEL file (a popular XML format for bookmarks that was developed by Python's very own XML-SIG). There is a sample XBEL file you can use in the demo directory.
doc = amara.parse('xbel.xml')
The following sample code prints the first and second bookmark titles:
print doc.xbel.folder.bookmark.title
print doc.xbel.folder.bookmark[1].title
Note that Bindery tries to make things as natural as possible. You can access child elements by just using their name, usually. If there is more than one with the same name it grabs the first one. You can use list indices to specify one of multiple child elements with the same name. Naturally, a more explicit way of getting the first bookmark's title is:
print doc.xbel.folder.bookmark[0].title
title is an element containing only text. the expression doc.xbel.folder.bookmark[0].title returns the binding object representing the element. Such objects have unicode conversion methods so that you can get their descendant text (all text nodes in the element's subtree), which is printed in the above line. The following four lines are all equivalent:
print doc.xbel.folder.bookmark.title
print doc.xbel.folder.bookmark[0].title
print unicode(doc.xbel.folder.bookmark.title)
print unicode(doc.xbel.folder.bookmark[0].title)
Calling the unicode conversion on an element node in Amara is very similar to
calling the string conversion on an element node in XPath.
The following snippet is a recursive function that prints out all bookmark URLs in the file:
def all_titles_in_folder(folder):
#Warning: folder.bookmark will raise an AttributeError if there are no bookmarks
for bookmark in folder.bookmark:
print bookmark.href
if hasattr(folder, "folder"):
#There are sub-folders
for folder in folder.folder:
all_titles_in_folder(folder)
return
for folder in doc.xbel.folder:
all_titles_in_folder(folder)
You would probably not do this in actual usage, though. You can perform this task with much less code, without recursion, and with a speed boost using XPath, which is covered in a later section.
If you want to count the number of elements of the same name, use len.
len(doc.xbel.folder)
This gives the number of top-level folders.
In the default binding XML elements turn into specialized objects. For
each generic identifier (element name) a class is generated that is
derived from bindery.element_base. Attributes become simple data
members whose value is a Unicode object containing the attribute
value.
Element objects are specially constructed so they can be treated as single objects (in which case the first child element of the corresponding name is selected, or one can use list item access notation or even iterate.
Going back to the example, binding.xbel.folder.bookmark is the same
as binding.xbel.folder.bookmark[0]. Both return the first bookmark
in the first folder. To get the second bookmark in the first folder,
use binding.xbel.folder.bookmark[1].
Bindery by default preserves ordering information from the source XML. You can access this through the children list of element and document objects:
folder.xml_children
Within the children list, child elements are represented using the
corresponding binding objects, child text becomes simple Unicode
objects. Notice that in the default binding text children are
normalized, meaning that the binding will never place two text nodes
next to each other in xml_children.
If an element node contains text as well as child elements, be aware of
the how descendant text nodes are accessed. You can get the accumulated
text children of an element using the xml_child_text property. Given
the document <a>1<b>2</b>3<c/></a>, a.xml_child_text would return
u'13'. On the other hand, converting to Unicode (unicode(a))
would return u'123'.
You can get some information about the structure of most bindery objects
by calling the xml_doc method.
>>> import amara
>>> doc = amara.parse('monty.xml')
>>> print doc.xml_doc()
Object references based on XML child elements:
monty (1 element) based on 'monty' in XML
>>> print doc.monty.xml_doc()
Object references based on XML child elements:
python (2 elements) based on 'python' in XML
>>> print doc.monty.python.xml_doc()
Object references based on XML attributes:
spam based on 'spam' in XML
Object references based on XML child elements:
>>> print doc.monty.python.spam
eggs
This is human-readable material, for convenience, but you can also get
node structure information in machine-readable form. xml_properties
returns a dictionary whose keys represent the object reference names from
the XML attributes and elements.
>>> import amara
>>> doc = amara.parse("<a x='1'>hello<b/>lovely<c/>world</a>")
>>> doc.a.xml_properties
{u'x': u'1', u'c': <amara.bindery.c object at 0xb7bbcdcc>, u'b': <amara.bindery.b object at 0xb7bbcb8c>}
xml_child_elements returns similar dictionary, but reduced to only the child element.
>>> #Continuing from the above snippet
>>> doc.a.xml_child_elements
{u'c': <amara.bindery.c object at 0xb7bbcdcc>, u'b': <amara.bindery.b object at 0xb7bbcb8c>}
The preservation of ordering information means that Bindery does a
pretty good job of allowing you to render binding objects back to XML
form. Use the xml method for this.
print doc.xml()
The xml method returns encoded text, not Unicode. The default encoding
is UTF-8. You can also serialize a portion of the document.
print doc.xbel.folder.xml() #Just the first folder
You can pass in a stream for the output:
doc.xml(sys.stdout)
You can control such matters as the output encoding, whether the
output is pretty-printed, whether there is an output XML declaration,
etc. by using parameters that control the output, based on the XSLT output
control attributes. As an example, if you pas in omitXmlDeclaration=u"yes"
the output of the XML declaration is suppressed. Here are the other
parameters you can set.
Bindery supports an XPath subset that covers almost all of that standard. The XPath features that are not supported are very rare, and for pracctical purposes you can assume it's complete XPath support. The folowing example retrieves all top-level folders:
tl_folders = doc.xbel.xml_xpath(u'folder')
for folder in tl_folders:
print folder.title
You invoke the xml_xpath method on the object you wish to serve as
the context for the XPath query. To get the first element child
(regardless of node name) of the first bookmark of the first folder,
use:
doc.xbel.folder.bookmark.xml_xpath(u'*[1]')
or
doc.xbel.xml_xpath(u'folder[1]/bookmark[1]/*[1]')
or
doc.xbel.xml_xpath(u'/folder[1]/bookmark[1]/*[1]')
or
doc.xml_xpath(u'xbel/folder[1]/bookmark[1]/*[1]')
etc.
Remember: in Python, lists indices start with 0 while they start with 1 in XPath.
Notice: this XPath returns a node set, rendered in Python as a list of nodes. It happens to be a list of one node, but you still have to extract it with [0].
The return value depends on the XPath expression (expr)
The following example prints out all bookmark URLs in the file, but is much simpler and more compact than the equivalent code earlier in this document:
bookmarks = doc.xml_xpath(u'//bookmark')
for bookmark in bookmarks:
print bookmark.href
The following just returns all hrefs wherever they appear in the document, using an attribute query:
hrefs = doc.xml_xpath(u'//@href')
for href in hrefs:
print unicode(href)
The following prints the title of the bookmark for the 4Suite project:
url = u"http://4suite.org/"
title_elements = doc.xml_xpath('//bookmark[@href="%s"]/title'%url)
#XPath node set expression always returns a list
print unicode(title_elements[0])
Bindery supports documents with namespaces. The following example displays a summary of the contents of an RSS 1.0 feed:
import amara
#Set up customary namespace bindings for RSS
#These are used in XPath query and XPattern rules
RSS10_NSS = {
u'rdf': u'http://www.w3.org/1999/02/22-rdf-syntax-ns#',
u'dc': u'http://purl.org/dc/elements/1.1/',
u'rss': u'http://purl.org/rss/1.0/',
}
doc = amara.parse('rss10.rdf', prefixes=RSS10_NSS)
#Create a dictionary of RSS items
items = {}
item_list = doc.xml_xpath(u'//rss:item')
items = dict( [ ( item.about, item) for item in item_list ] )
print items
for channel in doc.RDF.channel:
print "Channel:", channel.about
print "Title:", channel.title
print "Items:"
for item_ref in channel.items.Seq.li:
item = items[item_ref.resource]
print "\t", item.link
print "\t", item.title
The following illustrates how namespace details are maintained in bindery objects:
#Show the namespace particulars of the rdf:RDF element
print doc.RDF.namespaceURI
print doc.RDF.localName
print doc.RDF.prefix
For attributes, xml_attributes is a dictionary containing namespace
information for all of an element's attributes. See the Attributes
section below for more details.
Namespaces work naturally with XPath as well:
#Get the RSS item with a given URL
item_url = u'http://www.oreillynet.com/cs/weblog/view/wlg/532'
matching_items = doc.RDF.xml_xpath(u'//rss:item[@rdf:about="%s"]'%item_url)
print matching_items
assert matching_items[0].about == item_url
In the above example I manually set the mapping from namespace prefixes to namespace names, but if your use of XML namespaces is "sane", you might not need this step. Bindery document objects automatically remember the namespace declarations made on the top level element of any document you parse.
The Python object reference and class names used in Amara bindings are
based on the corresponding XML IDs, but there are limits to such mappings.
For one thing, XML allows characters that are not allowed in Python IDs,
such as -. In such cases Amara mangles the name. In the case of a document
such as <a-1 b-1=""/> you would access the element using doc.a_1, and the attribute using doc.a_1.b_1.
This can be problematic when writing generic code to address XML, because any element's name might be mangled in a different way depending on document context. You always have the option of accessing objects using their proper XML names in XPath:
doc.xml_xpath(u"a-1/b-1)
You can get the original XML name information from any object using properties named after DOMs:
element.nodeName
element.localName
element.namespaceURI
element.prefix
And you can get similar information on attributes by reading the
xml_attributes dictionary on the element node.
If you're dealing with a large XML file, you may not want the entire
data binding in memory at the same time. You may want to instantiate
it bit by bit. If you have a clear pattern for how you want to break
up the document, you can use the function amara.pushbind. See
the following example:
import amara
for folder in amara.pushbind('xbel.xml', u'/xbel/folder'):
title = folder.title
bm_count = len(list(folder.bookmark))
print "Folder", title, "has", bm_count, "top level bookmarks"
The neat thing is that this program doesn't have the entire binding in
memory at time. In each iteration it loads just enough XML to represent
each top-level folder element. pushbind is a generator that yields each
little subtree each time it's invoked.
In general pushbind subtrees are Amara bindery
objects based on the XSLT pattern that is passed in as its first argument.
You also pass in an XML source, either a string (use the string= keyword)
or a URI or file-name (use the source= keyword).
An interactive session helps illustrate:
>>> XML="""\
... <doc>
... <one><a>0</a><a>1</a></one>
... <two><a>10</a><a>11</a></two>
... </doc>
... """
>>> import amara
>>> chunks = amara.pushbind(XML, u'a')
>>> a = chunks.next()
>>> print a
0
>>> print a.xml()
<a>0</a>
>>> a = chunks.next()
>>> print a.xml()
<a>1</a>
>>> a = chunks.next()
>>> print a.xml()
<a>10</a>
>>> a = chunks.next()
>>> print a.xml()
<a>11</a>
>>> a = chunks.next()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
StopIteration
In the above case the XSLT pattern "a" matched all the elements with this name in the file. All the XML outside the "a" elements is essentially discarded by this code, so you have to tailor the XSLT pattern to include all the material you're interested in. In the following example, only the first two "a" elements are included in the generator yield.
>>> chunks = amara.pushbind(XML, u'one/a')
>>> a = chunks.next()
>>> print a.xml()
<a>0</a>
>>> a = chunks.next()
>>> print a.xml()
<a>1</a>
>>> a = chunks.next()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
StopIteration
The pattern /doc/one/a would result in similar behavior for the sample document.
You can use namespaces in these patterns by using prefixes defined in a dictionary passed to pushbind.
import amara
#Set up customary namespace bindings for RSS
#These are used in XPath query and XPattern rules
RSS10_NSS = {
u'rdf': u'http://www.w3.org/1999/02/22-rdf-syntax-ns#',
u'dc': u'http://purl.org/dc/elements/1.1/',
u'rss': u'http://purl.org/rss/1.0/',
}
#Print out all titles in the RSS feed
items = amara.pushbind('demo/rss10.rdf', u'rss:item',
prefixes=RSS10_NSS)
for item in items:
print item.title
#Print out all titles in the RSS feed, slightly different approach
chunks = amara.pushbind('demo/rss10.rdf', u'rss:item/rss:title',
prefixes=RSS10_NSS)
for title in chunks:
print title
Modifying a binding is pretty straightforward. You can create or modify an attribute by simple assignment:
import amara
doc = amara.parse('monty.xml')
doc.monty.foo = u'bar'
doc.monty.spam = u'[attr modified]'
You can also replace an element's contents with a single text node, using similar notation:
doc.monty.python = u'[elem 1 modified]\n'
doc.monty.python[1] = u'[elem 2 modified]\n'
The resut of the above code is:
<monty>
<python spam="[attr modified]" foo="bar">[elem 1 modified]
</python>
<python ministry="abuse">[elem 2 modified]
</python>
</monty>
You can empty out all children of an element:
doc.monty.python.xml_clear()
And add new elements and text. Create a new, empty element named new
and append it as the last childof the first python element.
doc.monty.python.xml_append(doc.xml_create_element(u'new'))
Append a text node.
doc.monty.python.new.append(u'New Content')
You can insert new elements or text into specific locations, either before or after an existing child.
#Create a new `python` element as the second element child of `monty`
doc.monty.xml_insert_after(doc.monty.python,
doc.xml_create_element(u'python'))
#Create a new `python` element as the first element child of `monty`
doc.monty.xml_insert_before(doc.monty.python,
doc.xml_create_element(u'python'))
You can also delete a specific element, attribute, or text child:
del doc.monty.python
which does the same thing as
del doc.monty.python[0]
You can also use the xml_remove_child method:
child = doc.monty.python
doc.monty.xml_remove_child(child)
Just to exhaustively list the approaches to deletion, you can use the
position of an object among its siblings to remove it from its parent.
You can get this position using the xml_index_on_parent property.
ix = doc.monty.python.xml_index_on_parent
This assigns ix the value 1 since the first python element is the second child of monty.
ix = doc.monty.python[1].xml_index_on_parent
This assigns ix the value 3 since the second python element is the
fourth child of monty. Once you have an index, you can use that index
in order to delete a specific child by passing it to the
xml_remove_child_at method:
doc.monty.xml_remove_child_at(3)
This removes the second python element, using the index determined above. This works for text as well:
doc.monty.xml_remove_child_at(0)
Removes the first text node. You can omit the index in the call to this method. By default it removes the last child:
doc.monty.xml_remove_child_at()
You can create a new element with a namespace:
e = doc.xml_create_element(element_qname, element_ns)
which is equivalent to
e = doc.xml_create_element(element_qname, namespace=element_ns)
You can easily add attributes while you're creating elements.
import amara
doc = amara.parse('monty.xml')
#Create a third python element
e = doc.xml_create_element(u'python', attributes={u'life': u'brian'})
doc.monty.xml_append(e)
print doc.xml()
This gives the following output:
<?xml version="1.0" encoding="UTF-8"?>
<monty>
<python spam="eggs">
What do you mean "bleh"
</python>
<python ministry="abuse">
But I was looking for argument
</python>
<python life="brian"/></monty>
You can see how the attribute is manifested on the new element.
The dictionary attributes={u'life': u'brian'} is actually a
short-hand. The full form is:
{
(<attr1_qname>, <attr1_namespace>): attr1_value},
(<attr2_qname>, <attr2_namespace>): attr2_value},
...
(<attrN_qname>, <attrN_namespace>): attrN_value},
}
But you can abbreviate (<attrN_qname>, <attrN_namespace>) to
just <attrN_qname> if the namespace is None (i.e. the attribute is
not in a namespace).
So you could add a namespace qualified attribute as follows:
import amara
NS = u'urn:bogus'
doc = amara.parse('monty.xml')
#Create a third python element
e = doc.xml_create_element(
u'python',
attributes={(u'ns:life', NS): u'brian'},
content=u'unfortunate'
)
doc.monty.xml_append(e)
print doc.xml()
Notice that this time I threw in some content as well. The result:
<?xml version="1.0" encoding="UTF-8"?>
<monty>
<python spam="eggs">
What do you mean "bleh"
</python>
<python ministry="abuse">
But I was looking for argument
</python>
<python xmlns:ns="urn:bogus" ns:life="brian">unfortunate</python></monty>
To create an attribute after the element is created, use the
xml_set_attribute method. To add the namespace
qualified attribute from the example above, use the following:
import amara
NS = u'urn:bogus'
doc = amara.parse('monty.xml')
#Create a third python element
e = doc.xml_create_element(
u'python',
content=u'unfortunate'
)
doc.monty.xml_append(e)
doc.monty.python[2].xml_set_attribute((u'ns:life', NS), u'brian')
print doc.xml()
This produces the same XML as above. If no namespace is required, the
first argument to xml_set_attribute is just the attribute name.
The xml_set_attribute method returns the name of the resulting
attribute.
You can set an attribute's value using Python idiom as well:
doc.monty.python[2].life = u'Pi'
Information about an element's attributes, including namespace
information, is kept in xml_attributes. This is a dictionary keyed
by the local name of each attribute, with the value being a tuple of
the namespace qualified attribute name and the namespace URL. For
example, given the following code:
import amara
NS = u'urn:bogus'
doc = amara.parse('monty.xml')
#Create a third python element
e = doc.xml_create_element(
u'python',
content=u'unfortunate'
)
doc.monty.xml_append(e)
#Add a namespace qualified attribute
doc.monty.python[2].xml_set_attribute((u'ns:life', NS), u'brian')
#Add an attribute with no namespace
doc.monty.python[2].xml_set_attribute(u'foo', u'bar')
doc.monty.python[2].xml_attributes gives the value:
{u'life': (u'ns:life', u'urn:bogus'), u'foo': (u'foo', None)}
Just for fun, here's an interesting variation that illustrates the special status of the XML namespace.
import amara
from xml.dom import XML_NAMESPACE as XML_NS
doc = amara.parse('monty.xml')
#Create a third python element
e = doc.xml_create_element(
u'python',
attributes={(u'xml:lang', XML_NS): u'en'},
content=u'Ni!'
)
doc.monty.xml_append(e)
print doc.xml()
This gives the following output:
<?xml version="1.0" encoding="UTF-8"?>
<monty>
<python spam="eggs">
What do you mean "bleh"
</python>
<python ministry="abuse">
But I was looking for argument
</python>
<python xml:lang="en">Ni!</python></monty>
Notice that there's no declaration for the XML namespace, as allowed by the standard.
Amara offers a powerful facility for adding to XML documents through the
xml_append_fragment method on elements. You pass this method a string
(not Unicode, since this is a parse operation) with a fragment of
literal XML which is parsed and added to the element. The XML fragment
must be a well formed external parsed entity. Basically multiple root
elements are allowed, but they must be properly balanced, special
characters escaped, and so on. Doctype declaration is
prohibited. According to XML rules, the encoded string is
assumed to be UTF-8 or UTF-16, but you can override this with
an XML text declaration (<?xml version="1.0" encoding="ENC"?>)
or by passing in an encoding parameter to this function.
import amara
doc = amara.parse('monty.xml')
doc.monty.xml_append_fragment('<py3 x="1">p</py3><py4 y="2">q</py4>')
print doc.monty.xml_child_elements.keys()
The output is [u'python', u'py3', u'py4'], as two elements are added in
the xml_append_fragment call.
The optional encoding is a string with the encoding to be used in parsing the XML fragment. If this parameter is specified, it overrrides any text declaration in the XML fragment.
#Latin-1 ordinal 230 is the a-e ligature
doc.monty.xml_append_fragment('<q>P%an</q>'%chr(230), 'latin-1')
You can create entire documents from scratch
doc = amara.create_document()
doc.xml_append(doc.xml_create_element(u"hello"))
doc.xml()
Yields
<?xml version="1.0" encoding="UTF-8"?>
<hello/>
Fleshing it out a bit the following code:
doc = amara.create_document()
doc.xml_append(doc.xml_create_element(u"hello"))
doc.hello.xml_append(doc.xml_create_element(u"world"))
doc.xml()
Yields
<?xml version="1.0" encoding="UTF-8"?>
<hello><world/></hello>
And so on.
If you're familiar with XSLT, you might wonder how to do the equivalent of xsl:copy-of in Amara. You can create such a deep copy using the copy module in the Python standard library.
import copy
import amara
doc = amara.parse('monty.xml')
#Clone the document
doc2 = copy.deepcopy(doc)
#This modification only affects the clone
doc2.monty.python.spam = u"abcd"
#This output will just like what was read in
print doc.xml()
#This output will show the change from "eggs" to "abcd"
print doc2.xml()
Bindery supports these XML constructs in a fairly natural way. If you have PIs or comments in a source document parsed into a binding, it will have objects representing the PIs and comments:
import amara
DOC = """\
<?xml-stylesheet href="xxx.css" type="text/css"?>
<!--A greeting for all-->
<hello-world/>
"""
doc = amara.parse(DOC)
print doc.xml_children
shows the following list:
[
<amara.bindery.pi_base instance at 0x433f6c>,
<amara.bindery.comment_base instance at 0x433fac>,
<amara.bindery.hello_world object at 0x433fec>
]
The first item is a bound PI object and the second a bound comment. You can dig deeper if you like:
pi = doc.xml_children[0]
comment = doc.xml_children[1]
print repr(pi.target) #shows u'xml-stylesheet'
print repr(pi.data) #shows u'href="xxx.css" type="text/css"'
print repr(comment.body) #shows u'A greeting for all'
You can also create or mutate these objects.
doc = amara.create_document()
pi = bindery.pi_base(u"xml-stylesheet", u'href="xxx.css" type="text/css"')
doc.xml_append(pi)
doc.xml_append(doc.xml_create_element(u"A"))
There is also an API for document type declarations (DTDecls).
To create a document with a DTDecl, do something like the following:
doc = amara.create_document(
u"xsa",
pubid=u"-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML",
sysid=u"http://www.garshol.priv.no/download/xsa/xsa.dtd"
)
Which results in a tree equivalent to:
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE xsa PUBLIC "-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML"
"http://www.garshol.priv.no/download/xsa/xsa.dtd">
<xsa/>
Notice how this automatically creates the document element for you (no
need for a separate xml_append). You can expand on this to create
attributes and content for the document element:
from xml.dom import XML_NAMESPACE as XML_NS
doc = amara.create_document(
u"xsa",
attributes={(u'xml:lang', XML_NS): u'en'},
content=u' ',
pubid=u"-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML",
sysid=u"http://www.garshol.priv.no/download/xsa/xsa.dtd"
)
Which results in:
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE xsa PUBLIC "-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML"
"http://www.garshol.priv.no/download/xsa/xsa.dtd">
<xsa xml:lang="en"> </xsa>
You can then access the DTDecl details:
assert doc.xml_pubid == u"-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML"
assert doc.xml_sysid == u"http://www.garshol.priv.no/download/xsa/xsa.dtd"
assert doc.xml_doctype_name == u"xsa"
The idea is that if you parse a document with a DTDecl, the root node contains the document element QName, the public ID and the system ID. This does not work yet because of idiosyncracies of the Python/XML libraries. This should be fixed when I remove dependencies on these libraries in a coming version. When it does work, you should be able to do:
#DOES NOT YET WORK
XSA = """\
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE xsa PUBLIC "-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML"
"http://www.garshol.priv.no/download/xsa/xsa.dtd">
<xsa>
<vendor>
<name>Fourthought, Inc.</name>
<email>info@fourthought.com</email>
<url>http://fourthought.com</url>
</vendor>
<product id="FourSuite">
<name>4Suite</name>
<version>1.0a1</version>
<last-release>20030327</last-release>
<info-url>http://4suite.org</info-url>
<changes>
- Begin the 1.0 release cycle
</changes>
</product>
</xsa>
"""
doc = amara.parse(XSA)
assert doc.xml_pubid == u"-//LM Garshol//DTD XML Software Autoupdate 1.0//EN//XML"
assert doc.xml_sysid == u"http://www.garshol.priv.no/download/xsa/xsa.dtd"
assert doc.xml_doctype_name == u"xsa"
Internal DTD subset constructs are not preserved in the binding.
Bindery works by iterating over XML nodes and firing off a set of rules triggered by the node type and other details. The default binding is the result of the default rules that are registered for each node type, but Bindery makes this easy to tweak by letting you register your own rules.
Bindery comes bundled with 3 easy rule frameworks to handle some common binding needs.
The title elements in XBEL are always simple text, and creating full
Python objects for them is overkill in most cases. They could be just
as easily simple data members with the Unicode value of the element's
content. To make this adjustment in Bindery, register an instance of
the simple_string_element_rule rule. This rule takes an list of
XSLT pattern expressions which indicate which elements are to be
simplified. So to simplify all title elements:
import amara
from amara import binderytools
#Specify (using XSLT patterns) elements to be treated similarly to attributes
rules = [
binderytools.simple_string_element_rule(u'title')
]
#Execute the binding
doc = amara.parse('xbel.xml', rules=rules)
#title is now simple unicode
print doc.xbel.folder.bookmark.title.__class__
Perhaps you want to focus on only part of a document, and to save
memory and hassle, you want to omit certain elements that are not of
interest in the binding. You can use the omit_element_rule in this
case.
The following example does not create bindings for folder titles at all (but bookmark titles are preserved):
import amara
from amara import binderytools
#Specify (using XSLT patterns) elements to be ignored
rules = [
binderytools.omit_element_rule(u'folder/title')
]
#Execute the binding
doc = amara.parse('xbel.xml', rules=rules)
#Following would now raise an exception:
#print doc.xbel.folder.title
A common need is to strip out pure whitespace nodes so that they don't
clutter up "children" lists. Bindery bundles the ws_strip_element_rule
rule for this purpose. Elements that match the pattern are stripped of whitespace.
import amara
from amara import binderytools
#Specify (using XSLT patterns) elements to be stripped
#In this case select all top-level elements for stripping
rules = [
binderytools.ws_strip_element_rule(u'/*')
]
#Execute the binding
doc = amara.parse('xbel.xml', rules=rules)
You can combine rules, such as stripping white space while still omitting certain elements.
If all you care about is the structure of elements and attributes, and
not the text content you can use the element_skeleton_rule.
Elements that match the pattern have all character data stripped.
import amara
from amara import binderytools
#Specify (using XSLT patterns) elements to be bound as skeletons
#In this case select all elements
rules = [
binderytools.element_skeleton_rule(u'*')
]
#Execute the binding
doc = amara.parse('xbel.xml', rules=rules)
The basic idea behind data binding is translating XML into native data types. Amara provides a rule that looks at each XML node to see if it can infer a native Python data type for the value, in particular int, float or datetime.
TYPE_MIX = """\
<?xml version="1.0" encoding="utf-8"?>
<a a1="1">
<b b1="2.1"/>
<c c1="2005-01-31">
<d>5</d>
<e>2003-01-30T17:48:07.848769Z</e>
</c>
<g>good</g>
</a>"""
import amara
from amara import binderytools
rules=[binderytools.type_inference()]
doc = amara.parse(TYPE_MIX, rules=rules)
doc.a.a1 == 1 #type int
doc.a.b.b1 == 2.1 #type float
doc.a.c.c1 == datetime.datetime(2005, 1, 31) #type datetime.
The built-in custom rules use XSLT patterns, which use prefixes to specify namespaces. You may have to let the binder tools know what namespace bindings are in effect:
import amara
from amara import binderytools
#Set up customary namespace bindings for RSS
#These are used in XPath query and XPattern rules
RSS10_NSS = {
'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#',
'dc': 'http://purl.org/dc/elements/1.1/',
'rss': 'http://purl.org/rss/1.0/',
}
rules = [
binderytools.simple_string_element_rule(u'title')
]
#Execute the binding
doc = amara.parse('rss10.rdf', prefixes=RSS10_NSS, rules=rules)
Remember, however, as dicussed above, if you declare all the namespaces you use on top-level elements, you do not need to repeat them in the explicit prefixes dictionary.
You can also use rules with pushbind. If all you ever want from "a" elements is to extract the text content, you could do something like the following. Look carefully: the sample document is slightly different this time.
>>> XML="""\
... <doc>
... <one><a>0</a><b>1</b></one>
... <two><a>10</a><b>11</b></two>
... </doc>
... """
>>> import amara
>>> from amara import binderytools
>>> #This rule says "treat all elements at the third level of depth as simple strings"
>>> rule = binderytools.simple_string_element_rule(u'/*/*/*')
>>> #Push back bindings of all second level elements ('one' and 'two')
>>> chunks = amara.pushbind(XML, u'/*/*', rules=[rule])
>>> elem = chunks.next()
>>> print elem.a.__class__
<type 'unicode'>
>>> print elem.a
u'0'
>>> print elem.b.__class__
<type 'unicode'>
>>> print elem.b
u'1'
If you need more sophisticated tweaking, you proably want to register
your own customized binding class. The following example gives
bookmark elements a method, retrieve, which retrieves the body of
the Web page:
import urllib
from xml.dom import Node
import amara
from amara import bindery
from Ft.Xml import InputSource
from Ft.Lib import Uri
#Subclass from the default binding class
#We're adding a specialized method for accessing a bookmark on the net
class specialized_bookmark(bindery.element_base):
def retrieve(self):
try:
stream = urllib.urlopen(self.href)
content = stream.read()
stream.close()
return content
except IOError:
import sys; sys.stderr.write("Unable to access %s\n"%self.href)
#Explicitly create a binder instance, in order to customize rules
binder = bindery.binder()
#associate specialized_bookmark class with elements not in an XML
#namespace and having a GI of "bookmark"
binder.set_binding_class(None, "bookmark", specialized_bookmark)
#Execute the binding
doc = amara.parse('xbel.xml', binderobj=binder)
#Show specialized instance
print doc.xbel.folder.bookmark.__class__
#Exercise the custom method
print "Content of first bookmark:"
print doc.xbel.folder.bookmark.retrieve()
Focus on the line:
binder.set_binding_class(None, "bookmark", specialized_bookmark)
When you register classes to use in binding a given elements type you
do so by specifying namespace URI and local name of the element. If
you know that the element is not in a namespace, as in the XBEL
example, you use None. None is the Right Way to signal "not in a
namespace" in most Python/XML tools, and not the empty string "".
Bindery tries to manage things so that writing back the XML from a binding makes sense, and that XPath gives expected results, but it is easy to bring about odd results if you customize the binding.
As an exampe, if you use simple_string_element_rule and then
reserialize using the xml method, the elements that were simplified
will be written back out as XML attributes rather than child
elements. If you do run into such artifacts after customizing a
binding the usual remedy is to write a custoized xml method or add
specialized XPath wrapper code (see bonderyxpath.xpath_wrapper_mixin for the
default XPath wrappering).
Bindery is designed to be extensible, but this is not a simple proposition given the huge flexibility of XML expression, and the many different ways developpers might want to generate resulting Python objects (and vice versa). You can pretty much do whatever you need to by writing Bindery extensions, but in order to keep things basically manageable, there are some ground rules.
[TODO: more on this section to come. If you try tweaking bindery extensions and have some useful notes, please pitch in by sending them along.]
Scimitar is an implementation of ISO Schematron that compiles a Schematron schema into a Python validator script.
Scimitar supports all of the draft ISO Schematron specification. See the TODO file for known gaps in Scimitar convenience.
You typically use scimitar in two phases. Say you have a schematron schema schema1.stron and you want to validate multiple XML files against it, instance1.xml, instance2.xml, instance3.xml.
First you run schema1.stron through the scimitar compiler script, scimitar.py:
scimitar.py schema1.stron
A file, schema1.py (same file stem with the "py" extension sunstituted), is generated in the current working directory. If you'd prefer a different location or file name, use the "-o" option. The generated file is a validator script in Python. It checks the schematron rules specified in schema1.stron.
You now run the generated validator script on each XML file you wish to validate:
python schema1.py instance1.xml
The validation report is generated on standard output by default, or you can use the "-o" option to redirect it to a file.
The validation report is an XML external parsed entity, in other words a file that is much like a well-formed XML document, but with some restrictions loosened so that it's effectively text with possible embedded tags.
To elaborate using the example from the schematron 1.5 specification:
$ cat simple1.stron
<?xml version="1.0" encoding="UTF-8"?>
<sch:schema xmlns:sch="http://www.ascc.net/xml/schematron" version="ISO">
<sch:title>Example Schematron Schema</sch:title>
<sch:pattern>
<sch:rule context="dog">
<sch:assert test="count(ear) = 2"
>A 'dog' element should contain two 'ear' elements.</sch:assert>
<sch:report test="bone"
>This dog has a bone.</sch:report>
</sch:rule>
</sch:pattern>
</sch:schema>
$ scimitar.py simple1.stron
$ ls simple*.py
simple1-stron.py
$ cat instance2.xml
<dog><ear/></dog>
$ python simple1-stron.py instance2.xml
<?xml version="1.0" encoding="UTF-8"?>
Processing schema: Example Schematron Schema
Processing pattern: [unnamed]
Assertion failure:
A 'dog' element should contain two 'ear' elements.
DOM came from the Java world, and hardly the most Pythonic API possible (see the Bindery above for a good step forward). Some DOM-like implementations such as 4Suite's Domlettes mix in some Pythonic idiom. Amara DOM Tools goes even further in this regard.
You're probably familiar with xml.dom.pulldom, which offers a nice
hybrid between SAX and DOM, allowing you to efficiently isolate
imporant parts of a document in a SAX-like manner, and then using DOM
for finer-grained manipulation. Amara's pushdom makes this process
even more convenient You give it a set of XPatterns, and it provides a
generator yielding a series of DOM chunks according to the patterns.
In this way you can process huge files with very little memory usage, but most of the convenience of DOM.
for docfrag in domtools.pushdom('demo/labels.xml', u'/labels/label'):
label = docfrag.firstChild
name = label.xpath('string(name)')
city = label.xpath('string(address/city)')
if name.lower().find('eliot') != -1:
print city.encode('utf-8')
Prints "Stamford".
See also this XML-DEV message.
For more on the generator tools see the article "Generating DOM Magic".
domtools.abs_path allows you to get the absolute path for a
node. The following code:
from amara import domtools
from Ft.Xml.Domlette import NonvalidatingReader
from Ft.Lib import Uri
file_uri = Uri.OsPathToUri('labels.xml', attemptAbsolute=1)
doc = NonvalidatingReader.parseUri(file_uri)
print domtools.abs_path(doc)
print domtools.abs_path(doc.documentElement)
for node in doc.documentElement.childNodes:
print domtools.abs_path(node)
Displays:
/
/labels[1]
/labels[1]/text()[1]
/labels[1]/label[1]
/labels[1]/text()[2]
/labels[1]/label[2]
/labels[1]/text()[3]
/labels[1]/label[3]
/labels[1]/text()[4]
/labels[1]/label[4]
/labels[1]/text()[5]
For more on abs_path tools see the article "Location, Location, Location".
Tenorsax (amara.saxtools.tenorsax) is a framework for "linerarizing"
SAX logic so that it flows a bit more naturally, and needs a lot less
state machine wizardry.
I haven't yet had time to document it heavily, but see test/saxtools/xhtmlsummary.py for an example.
See also this XML-DEV message.
Flextyper is an implementation of Jeni Tennison's Data Type Library Language (DTLL) (on track to become part 5 of ISO Document Schema Definition Languages (DSDL). You can use Flextyper to generate Python modules containing data types classes that can be used with 4Suite's RELAX NG library
Flextyper is currently experimental. It won't come into its full usefulness until the next release of 4Suite, although you can use it with current CVS releases of 4Suite.
Flextyper compiles a DTLL file into a collection of Python modules implementing the contained data types. Run it as follows:
flextyper.py dtll.xml
A set of files, one per data type namespace defined in the DTLL, is created. By default the output file names are based on the input, e.g. dtll-datatypes1.py, dtll-datatypes2.py, etc.
You can now register these data types modules with a processor instance of 4Suite's RELAX NG implementation.