[Tutor] Parsing an XML document using ElementTree

"Sithembewena Lloyd Dube" <zebra05 at gmail.com> wrote

Looks like you are getting an empty list back.
Try printing list(xml_content.getiterator('contest'))

And if thats empty try checking the case of your tag?
I'm pretty sure it will be case sensitive?

HTH,

Well, yes, it does return an XML document, but not what you expect:
"<response>\r\n <error-message>you must add 'accept-encoding' as
'gzip,deflate' to the header of your request</error-message>\r
\n</response>"

Meaning, the server forces you to pass an HTTP header to the request in
order to receive gzip compressed data. Once you have that, you must
decompress it before passing it into ElementTree's parser. See the
documentation on the gzip and urllib modules in the standard library.

Stefan

Hi Everyone,

Thanks for all your suggestions. I read up on gzip and urllib and also
learned in the process that I could use urllib2 as its the latest form of
that library.

Herewith my solution: I don't know how elegant it is, but it works just
fine.

def get_contests():
url = '
http://xml.matchbook.com/xmlfeed/feed?sport-id=&vendor=TEST&sport-name=&short-name=Po
'
req = urllib2.Request(url)
req.add_header('accept-encoding','gzip/deflate')
opener = urllib2.build_opener()
response = opener.open(req)
compressed_data = response.read()
compressed_stream = StringIO.StringIO(compressed_data)
gzipper = gzip.GzipFile(fileobj=compressed_stream)
data = gzipper.read()
current_path = os.path.realpath(MEDIA_ROOT + '/xml-files/d.xml')
data_file = open(current_path, 'w')
data_file.write(data)
data_file.close()
xml_data = ET.parse(open(current_path, 'r'))
contest_list = []
for contest_parent_node in xml_data.getiterator('contest'):
contest = Contest()
for contest_child_node in contest_parent_node:
if (contest_child_node.tag == "name" and
contest_child_node.text is not None and contest_child_node.text != ""):
contest.name = contest_child_node.text
if (contest_child_node.tag == "league" and
contest_child_node.text is not None and contest_child_node.text != ""):
contest.league = contest_child_node.text
if (contest_child_node.tag == "acro" and
contest_child_node.text is not None and contest_child_node.text != ""):
contest.acro = contest_child_node.text
if (contest_child_node.tag == "time" and
contest_child_node.text is not None and contest_child_node.text != ""):
contest.time = contest_child_node.text
if (contest_child_node.tag == "home" and
contest_child_node.text is not None and contest_child_node.text != ""):
contest.home = contest_child_node.text
if (contest_child_node.tag == "away" and
contest_child_node.text is not None and contest_child_node.text != ""):
contest.away = contest_child_node.text
contest_list.append(contest)
try:
os.remove(current_path)
except:
pass
return contest_list

Many thanks!

This is ok.

This should be simplifiable to

uncompressed_stream = gzip.GzipFile(fileobj=response)

And this subsequently becomes

xml_data = ET.parse(uncompressed_stream)

Take a look at ET.iterparse().

This is screaming for a simplification, such as

for child in contest_parent_node:
if child.tag in ('name', 'league', ...): # etc.
if child.text:
setattr(context, child.tag, child.text)


Stefan

Hi Stefan,

Thanks for the code review :) Only just noticed this.