FAQ
Hi,
Can anybody please give an example of a streaming mapper/reducer written in C++? I don't seem to find even a single example on web. Thanks.
Upendra

Search Discussions

  • Allen Wittenauer at Dec 4, 2009 at 11:19 pm
    For C/C++, you should be using the pipes interface.


    On 12/4/09 3:09 PM, "Upendra Dadi" wrote:

    Hi,
    Can anybody please give an example of a streaming mapper/reducer written in
    C++? I don't seem to find even a single example on web. Thanks.
    Upendra
  • Upendra Dadi at Dec 4, 2009 at 11:30 pm
    Thank you Allen for you reply. I am trying to use MapReduce on Amazon EC2.
    EC2 don't seem to support Pipes using their simple web GUI interface (is it
    possible to use Pipes using their CLI or API?!). What is the problem with
    using C++ with streaming?

    Upendra

    ----- Original Message -----
    From: "Allen Wittenauer" <awittenauer@linkedin.com>
    To: <common-user@hadoop.apache.org>
    Sent: Friday, December 04, 2009 6:17 PM
    Subject: Re: streaming job written in c++

    For C/C++, you should be using the pipes interface.


    On 12/4/09 3:09 PM, "Upendra Dadi" wrote:

    Hi,
    Can anybody please give an example of a streaming mapper/reducer
    written in
    C++? I don't seem to find even a single example on web. Thanks.
    Upendra
  • Allen Wittenauer at Dec 4, 2009 at 11:33 pm
    Absolutely none. You just read stdin/stdout as you would in any other
    language. But since C/C++ has its own interface, it is just more common to
    use that interface than using the one built for 'everything else'.

    On 12/4/09 3:29 PM, "Upendra Dadi" wrote:

    Thank you Allen for you reply. I am trying to use MapReduce on Amazon EC2.
    EC2 don't seem to support Pipes using their simple web GUI interface (is it
    possible to use Pipes using their CLI or API?!). What is the problem with
    using C++ with streaming?

    Upendra

    ----- Original Message -----
    From: "Allen Wittenauer" <awittenauer@linkedin.com>
    To: <common-user@hadoop.apache.org>
    Sent: Friday, December 04, 2009 6:17 PM
    Subject: Re: streaming job written in c++

    For C/C++, you should be using the pipes interface.


    On 12/4/09 3:09 PM, "Upendra Dadi" wrote:

    Hi,
    Can anybody please give an example of a streaming mapper/reducer
    written in
    C++? I don't seem to find even a single example on web. Thanks.
    Upendra
  • Chris Dyer at Dec 4, 2009 at 11:41 pm
    I've written plenty of apps in c++ using the streaming interface-- you
    just need to read from std::cin and write to std::cout. Keys and
    Values are separated by a tab.

    Since Pipes doesn't give you access to very much of the Hadoop/MR
    runtime (e.g., HDFS), and it's not a very idiomatic c++ interface
    (e.g., it passes around vectors of strings by value) and doesn't
    provide any of the usual JobConf convenience functions, I haven't
    found reason to use it.

    Chris

    On Fri, Dec 4, 2009 at 6:17 PM, Allen Wittenauer
    wrote:
    For C/C++, you should be using the pipes interface.


    On 12/4/09 3:09 PM, "Upendra Dadi" wrote:

    Hi,
    Can anybody please give an example of a streaming mapper/reducer written in
    C++? I don't seem to find even a single example on web. Thanks.
    Upendra

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-user @
categorieshadoop
postedDec 4, '09 at 11:10p
activeDec 4, '09 at 11:41p
posts5
users3
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase