FAQ
Hi,

I am using CDH4.2 and Cloudera Manager Free edition 4.5 based on parcels. I
would like to use sqoop2 but when I try to install the server component
using "yum install sqoop2-server", it tries to install
"hadoop-hdfs","hadoop-mapreduce" etc packages.

Not sure whether installing them would override the parcel based links and
configuration.What is the best way to use sqoop2 alongwith parcel based
cloudera installations ?

Does the instructions here
"https://ccp.cloudera.com/display/CDH4DOC/Sqoop+2+Installation" work with
parcel based installations ?

Thanks
Himanish

Search Discussions

  • Mark Grover at Mar 7, 2013 at 10:52 pm
    Hi Himanish,
    Our apologies for the inconvenience. Sqoop2 introduces a new
    client-server model; however, the functionality to manage Sqoop2
    server is not presently available through Cloudera Manager. While we
    work on adding support for managing sqoop2 server in Cloudera Manager,
    in the meanwhile, you will have to set some properties before running
    Sqoop2 server. The sqoop2 client would work out of the box with
    parcels.

    Last time I tried, I had to set these properties before running the
    server (replace "/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140"
    with the location of your parcel in a similar location):
    PIDFILE="/var/run/sqoop2/sqoop-server-sqoop2.pid"
    LIB_DIR="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/sqoop"
    SQOOP_LOG="/var/log/sqoop2"
    export CATALINA_BASE="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/sqoop2/sqoop-server-0.20"
    export CATALINA_HOME="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/bigtop-tomcat"
    export CATALINA_BIN="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/bigtop-tomcat/bin"
    export CATALINA_PID="$PIDFILE"
    export CATALINA_TMPDIR="/var/tmp/sqoop2"
    export CATALINA_OPTS="-Xmx1024m -cp ${LIB_DIR}:${LIB_DIR}/lib"
    export CATALINA_OUT="${SQOOP_LOG}/sqoop-tomcat.log"

    Then, start sqoop2 server.

    While I am not the best person to comment on this, it is possible to
    co-locate parcels and packages on the same machine. So, technically
    you could run Sqoop2 server from within the sqoop2-server package
    (which will work without having to explicitly set any properties) and
    use the client from within the parcel or the package. If you would
    prefer this approach, please post back and one of us could provide you
    with more details on how to do that.

    Please let us know if you have any further questions,
    Mark

    On Thu, Mar 7, 2013 at 1:34 PM, Himanish Kushary wrote:
    Hi,

    I am using CDH4.2 and Cloudera Manager Free edition 4.5 based on parcels. I
    would like to use sqoop2 but when I try to install the server component
    using "yum install sqoop2-server", it tries to install
    "hadoop-hdfs","hadoop-mapreduce" etc packages.

    Not sure whether installing them would override the parcel based links and
    configuration.What is the best way to use sqoop2 alongwith parcel based
    cloudera installations ?

    Does the instructions here
    "https://ccp.cloudera.com/display/CDH4DOC/Sqoop+2+Installation" work with
    parcel based installations ?

    Thanks
    Himanish
  • Himanish Kushary at Mar 8, 2013 at 10:47 am
    Thanks a lot Mark for the detailed answer. I will give it a try.

    On Thu, Mar 7, 2013 at 5:52 PM, Mark Grover wrote:

    Hi Himanish,
    Our apologies for the inconvenience. Sqoop2 introduces a new
    client-server model; however, the functionality to manage Sqoop2
    server is not presently available through Cloudera Manager. While we
    work on adding support for managing sqoop2 server in Cloudera Manager,
    in the meanwhile, you will have to set some properties before running
    Sqoop2 server. The sqoop2 client would work out of the box with
    parcels.

    Last time I tried, I had to set these properties before running the
    server (replace "/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140"
    with the location of your parcel in a similar location):
    PIDFILE="/var/run/sqoop2/sqoop-server-sqoop2.pid"
    LIB_DIR="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/sqoop"
    SQOOP_LOG="/var/log/sqoop2"
    export
    CATALINA_BASE="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/sqoop2/sqoop-server-0.20"
    export
    CATALINA_HOME="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/bigtop-tomcat"
    export
    CATALINA_BIN="/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.140/lib/bigtop-tomcat/bin"
    export CATALINA_PID="$PIDFILE"
    export CATALINA_TMPDIR="/var/tmp/sqoop2"
    export CATALINA_OPTS="-Xmx1024m -cp ${LIB_DIR}:${LIB_DIR}/lib"
    export CATALINA_OUT="${SQOOP_LOG}/sqoop-tomcat.log"

    Then, start sqoop2 server.

    While I am not the best person to comment on this, it is possible to
    co-locate parcels and packages on the same machine. So, technically
    you could run Sqoop2 server from within the sqoop2-server package
    (which will work without having to explicitly set any properties) and
    use the client from within the parcel or the package. If you would
    prefer this approach, please post back and one of us could provide you
    with more details on how to do that.

    Please let us know if you have any further questions,
    Mark

    On Thu, Mar 7, 2013 at 1:34 PM, Himanish Kushary wrote:
    Hi,

    I am using CDH4.2 and Cloudera Manager Free edition 4.5 based on
    parcels. I
    would like to use sqoop2 but when I try to install the server component
    using "yum install sqoop2-server", it tries to install
    "hadoop-hdfs","hadoop-mapreduce" etc packages.

    Not sure whether installing them would override the parcel based links and
    configuration.What is the best way to use sqoop2 alongwith parcel based
    cloudera installations ?

    Does the instructions here
    "https://ccp.cloudera.com/display/CDH4DOC/Sqoop+2+Installation" work with
    parcel based installations ?

    Thanks
    Himanish


    --
    Thanks & Regards
    Himanish

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupscm-users @
categorieshadoop
postedMar 7, '13 at 9:34p
activeMar 8, '13 at 10:47a
posts3
users2
websitecloudera.com
irc#hadoop

2 users in discussion

Himanish Kushary: 2 posts Mark Grover: 1 post

People

Translate

site design / logo © 2022 Grokbase