[Hadoop-common-user] parsing open xml

On Fri, Jun 12, 2009 at 1:31 PM, Alexandre Jaquet wrote:

Hi,

Does hadoop and map / reduce will allow me to parse large quantity of open
xml files distributed inside the same filesystem but using multipe jobs ?

Thx

Alexandre Jaquet

When you refer to "filesystem," do you mean HDFS?

It's very common to store lots of text files in HDFS and run multiple jobs
to process / learn about those text files. As for XML support, you can use
Java libraries (or Python libraries if you're using Hadoop streaming) to
parse the XML; Hadoop itself doesn't have much XML support. I hope this
answers your question.

Alex

On Fri, Jun 12, 2009 at 1:31 PM, Alexandre Jaquet <alexjaquet@gmail.com

On Sat, Jun 13, 2009 at 1:42 AM, Alexandre Jaquet wrote:

Thanks Alex,

Parsing the documents is a task done within the reducer ? we collect the
datas (document input) within a mapper and then parse it ?

Thanks in advance

Alexandre Jaquet

2009/6/13 Alex Loddengaard <alex@cloudera.com>
?

Well, you define what your job does, but I expect that nearly all MR jobs
do
their parsing in the mapper, not in the reducer. You may find these two
videos useful:

<http://www.cloudera.com/hadoop-training-mapreduce-hdfs>
<http://www.cloudera.com/hadoop-training-programming-with-hadoop>

Hope this helps!

Alex

On Sat, Jun 13, 2009 at 1:42 AM, Alexandre Jaquet <alexjaquet@gmail.com
to
this
alexjaquet@gmail.com
jobs