This isn't a HBase question. This belongs on
mapreduce[email protected] lists. I'm moving it there, and lets
carry on on that list. I've added you to cc in case you are not
subscribed to the mentioned list :)
Also, to reply to your original question - No, there isn't anything in
Hadoop core's Java APIs that lets you do this 'essential' task. It
would be useful to have a delimited text input format, if you'd like
to contribute one. Perhaps use OpenCSV or such a library for good
extensibility over delimited files.
For the record, Pig and Hive have support for such needs in them
natively, and you can use these downstream libraries to get down and
dirty with your data quickly.
On Fri, Sep 30, 2011 at 5:07 PM, Arsalan Bilal wrote:
I am asking about such* **InputFormat* and *RecordReader* that read strings
of text separated by (;) semicolon characters
--
Best Regards,
Arsalan Bilal
I am asking about such* **InputFormat* and *RecordReader* that read strings
of text separated by (;) semicolon characters
On Fri, Sep 30, 2011 at 11:11 AM, Sonal Goyal wrote:
Sorry, I think I got confused by the question and talked about the
OutputFormat not the input format, which is apparently what you are looking
for. Please ignore my answer. Apologies!
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Sep 30, 2011 at 12:32 PM, Sonal Goyal <[email protected]>
wrote:
Guava's
--
Best Regards,
Arsalan Bilal
Sorry, I think I got confused by the question and talked about the
OutputFormat not the input format, which is apparently what you are looking
for. Please ignore my answer. Apologies!
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Sep 30, 2011 at 12:32 PM, Sonal Goyal <[email protected]>
wrote:
Hi Arsalan,
Are you trying to insert this data into HBase or are you trying to just
process this log file using Hadoop? I am not sure how your question is
related to HBase, so if it is unrelated, you can seek help on the mapreduce
user lists.
For a MR job, you can use TextInputFormat and specify the custom
separator.
See https://issues.apache.org/jira/browse/HADOOP-3295.
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Sep 30, 2011 at 12:26 PM, Arsalan Bilal <
[email protected]>wrote:Are you trying to insert this data into HBase or are you trying to just
process this log file using Hadoop? I am not sure how your question is
related to HBase, so if it is unrelated, you can seek help on the mapreduce
user lists.
For a MR job, you can use TextInputFormat and specify the custom
separator.
See https://issues.apache.org/jira/browse/HADOOP-3295.
Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>
<http://in.linkedin.com/in/sonalgoyal>
On Fri, Sep 30, 2011 at 12:26 PM, Arsalan Bilal <
No , i did not try Guava's Splitter
I am asking about such input format class that takes also separator.
As example, job.setInputFormatClass(<Class Format>, <Seperator>);
What will be inputFormatClass here that support separator?
On Thu, Sep 29, 2011 at 8:39 PM, Buttler, David <[email protected]>
wrote:
I am asking about such input format class that takes also separator.
As example, job.setInputFormatClass(<Class Format>, <Seperator>);
What will be inputFormatClass here that support separator?
On Thu, Sep 29, 2011 at 8:39 PM, Buttler, David <[email protected]>
wrote:
Have you considered just taking the line of text as is and using
Splitter?
Not sure how this is related to HBase
-----Original Message-----
From: Arsalan Bilal
Sent: Thursday, September 29, 2011 7:24 AM
To: [email protected]
Subject: Problem in Input Format Class
i want to read text file (sample given below) separated with
semicolon(;)
using mapper. Each one record is separated with semicolon(;).
Should i write my own custom input format class? OR
Is there exist any input format class that ask about separator?
input File look
1;00000003;310:012:8001:01;-05:00;04:04;2010;45;56164773;3;1;0;1;
Not sure how this is related to HBase
-----Original Message-----
From: Arsalan Bilal
Sent: Thursday, September 29, 2011 7:24 AM
To: [email protected]
Subject: Problem in Input Format Class
i want to read text file (sample given below) separated with
semicolon(;)
using mapper. Each one record is separated with semicolon(;).
Should i write my own custom input format class? OR
Is there exist any input format class that ask about separator?
input File look
1;00000003;310:012:8001:01;-05:00;04:04;2010;45;56164773;3;1;0;1;
--
Best Regards,
Arsalan Bilal
--
Best Regards,
Arsalan Bilal
--
Harsh J