FAQ
Hi Folks

I am wondering if anyone has the answer of this question. I am processing
log files using Map reduce and get data to put some part into mysql and rest
of hbase. At the moment, i am running two separate jobs to do this so
reading single file for 2 times to dump the data. My questions is that can
it be possible that I run single job to achieve it??

--
Regards
Shuja-ur-Rehman Baig
http://pk.linkedin.com/in/shujamughal
Cell: +92 3214207445

Search Discussions

  • Jean-Daniel Cryans at Oct 27, 2010 at 8:47 pm
    Do both insertions in your reducer by either not using the output
    formats at all or use one of them and do the other insert by hand.

    J-D
    On Wed, Oct 27, 2010 at 1:44 PM, Shuja Rehman wrote:
    Hi Folks

    I am wondering if anyone has the answer of this question. I am processing
    log files using Map reduce and get data to put some part into mysql and rest
    of hbase. At the moment, i am running two separate jobs to do this so
    reading single file for 2 times to dump the data. My questions is that can
    it be possible that I run single job to achieve it??

    --
    Regards
    Shuja-ur-Rehman Baig
    http://pk.linkedin.com/in/shujamughal
    Cell: +92 3214207445
  • Shuja Rehman at Oct 27, 2010 at 9:11 pm
    Actually I am not using reducers. Only mappers work for me
    Secondly the procedure was mapper output saved in files which then
    transferred to mysql using sqoop. so here i need to save output to files +
    sent data to hbase. suppose i use output format to save data into file then
    how to send data to hbase in map reduce manually?
    On Thu, Oct 28, 2010 at 1:47 AM, Jean-Daniel Cryans wrote:

    Do both insertions in your reducer by either not using the output
    formats at all or use one of them and do the other insert by hand.

    J-D
    On Wed, Oct 27, 2010 at 1:44 PM, Shuja Rehman wrote:
    Hi Folks

    I am wondering if anyone has the answer of this question. I am processing
    log files using Map reduce and get data to put some part into mysql and rest
    of hbase. At the moment, i am running two separate jobs to do this so
    reading single file for 2 times to dump the data. My questions is that can
    it be possible that I run single job to achieve it??

    --
    Regards
    Shuja-ur-Rehman Baig
    http://pk.linkedin.com/in/shujamughal
    Cell: +92 3214207445


    --
    Regards
    Shuja-ur-Rehman Baig
    http://pk.linkedin.com/in/shujamughal
    Cell: +92 3214207445
  • Jean-Daniel Cryans at Oct 27, 2010 at 9:17 pm
    Using the HBase API in your mapper:
    http://hbase.apache.org/docs/current/api/org/apache/hadoop/hbase/client/HTable.html#put(java.util.List)

    J-D
    On Wed, Oct 27, 2010 at 2:10 PM, Shuja Rehman wrote:
    Actually I am not using reducers. Only mappers work for me
    Secondly the procedure was mapper output saved in files which then
    transferred to mysql using sqoop. so here i need to save output to files +
    sent data to hbase. suppose i use output format to save data into file then
    how to send data to hbase in map reduce manually?
    On Thu, Oct 28, 2010 at 1:47 AM, Jean-Daniel Cryans wrote:

    Do both insertions in your reducer by either not using the output
    formats at all or use one of them and do the other insert by hand.

    J-D

    On Wed, Oct 27, 2010 at 1:44 PM, Shuja Rehman <shujamughal@gmail.com>
    wrote:
    Hi Folks

    I am wondering if anyone has the answer of this question. I am
    processing
    log files using Map reduce and get data to put some part into mysql and
    rest
    of hbase. At the moment, i am running two separate jobs to do this so
    reading single file for 2 times to dump the data. My questions is that
    can
    it be possible that I run single job to achieve it??

    --
    Regards
    Shuja-ur-Rehman Baig
    http://pk.linkedin.com/in/shujamughal
    Cell: +92 3214207445


    --
    Regards
    Shuja-ur-Rehman Baig
    http://pk.linkedin.com/in/shujamughal
    Cell: +92 3214207445

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshbase, hadoop
postedOct 27, '10 at 8:45p
activeOct 27, '10 at 9:17p
posts4
users2
websitehbase.apache.org

People

Translate

site design / logo © 2018 Grokbase