Search Discussions
-
Hi, While importing data from Oracle, using Sqoop version: 1.4.6, is it possible to define Table Owner/Schema? Thanks.
Sam Joe
Jun 15, 2016 at 9:57 pm
Jun 15, 2016 at 9:57 pm -
At the very end of my Sqoop job, after all the data has been transferred from my db with read-only access, my mappers fail in connection.commit() because that is causing some kind of write to the ...
Mark Libucha
Jun 10, 2016 at 7:03 pm
Jun 10, 2016 at 7:03 pm -
Hi, I can’t keep from running out of JVM heap when trying to import a large Oracle table with the direct flag set. I can import successfully with smaller tables. Stack trace in the mapper log shows ...
Mark Libucha
Jun 9, 2016 at 3:51 pm
Jun 10, 2016 at 1:55 pm -
Hello Sqoop Community- I am excited to serve as the release manager for Sqoop 1.99.7. I wanted to reach out to the community and make sure that there is sufficient opportunity to discuss what we ...
Abraham Fine
May 31, 2016 at 5:58 pm
May 31, 2016 at 5:58 pm -
Hi to all, I would like to ask if is it possible to use Sqoop with Kafka, Avro and Confluent Schema Registry. I saw a lot of activity on this topic on the JIRA site, but, as newbie of Sqoop, I did ...
Giampaolo Trapasso
May 30, 2016 at 9:13 am
May 30, 2016 at 9:13 am -
Hi guys, I'm using MapR 4.1, Sqoop 1.4.6. I can successfully run a sqoop import with free-form sql query and without bulkload: sqoop import --query 'SELECT COLUMN1, COLUMN2 FROM MY_TABLE WHERE ...
Ruslan Al-Fakikh
May 24, 2016 at 6:52 pm
May 24, 2016 at 6:52 pm -
hi,I encounter a problem , when I import data from mysql to hdfs ,I found the values of each column was surround values with quotes : my question is why and how to stop sqoop2 do this ? thank you! ...
Zengbaitang
May 20, 2016 at 8:22 am
May 20, 2016 at 8:22 am -
Hi, i am doing sqoop import from oracle to Hive data is loading into Hive successfully, but i am seeing lot of NULL values, i am handling NULL values during sqoop import but in Hive i cannot get rid ...
Yeshwanth kumar
May 19, 2016 at 8:55 am
May 19, 2016 at 8:55 am -
Hi All, We have added lot of new enhancements in Apache Sqoop. We got review comments and we have pushed the changes accordingly.But from last 3-6 months , we are not getting any update on them.Since ...
Shashank Tandon
May 10, 2016 at 8:28 am
May 10, 2016 at 8:28 am -
Hi all, Could somebody say when will be released Sqoop 1.99.7? Thanks, Sergey!
Ssvinarchuk
May 10, 2016 at 7:34 am
May 25, 2016 at 11:39 pm -
Hi, We're having problems with connections not being released by sqoop (particularly with Oracle). Are there any ways to configure the connection pool (I believe it's c3p0) to release connections at ...
Per Ullberg
May 6, 2016 at 9:32 am
May 6, 2016 at 9:32 am -
0
How can I control the impact to the production environment when I use Sqoop to load data into hdfs
Hi, How big impact to the database does Sqoop load data from a database ? How will the Sqoop access and dump data from database ? Is it different among different connector or driver? If it has big ...Chen dong
May 4, 2016 at 8:20 am
May 4, 2016 at 8:20 am -
Hi, I have a bunch of sqoop jobs that import data from the same RDBMS. We have a max number of allowed connection on the RDBMS and want to make sure that we at any given time don't exceed that ...
Per Ullberg
May 3, 2016 at 7:26 am
May 3, 2016 at 7:26 am -
Hi Everyone, I have a table which has few columns as blob types with huge data. Is there any best way to 'sqoop import' it to hive tables with out losing any data ? Any help is highly appreciated ...
Ajay Chander
Apr 27, 2016 at 8:13 pm
Apr 27, 2016 at 10:44 pm -
3
Re:
sqoop import into HDFS. Which database is used? -------------------------------------------- On Mon, 4/25/16, Aleksey Ponkin wrote: Subject: To: <span class="m_body_email_addr" ...Deepak Vohra
Apr 25, 2016 at 8:57 pm
Apr 26, 2016 at 12:32 pm -
HI, I have the following use case: We have several SQL databases in different locations. And we need to load them in HDFS. The problem is that due to security concerns we do not have direct access ...
Aleksey Ponkin
Apr 25, 2016 at 6:59 pm
Apr 25, 2016 at 6:59 pm -
I am using java api to call sqoop. Is there a way to get counters or number of rows downloaded? or is there a way to capture log generated from sqoop. final int ret = Sqoop.runTool(new String[] { ...
Sura t
Apr 21, 2016 at 9:21 pm
Apr 21, 2016 at 9:21 pm -
Hi, I've been able to import data from Oracle in Sqoop 1.99.6 by creating links and *jobs*. However, I was wondering if the following syntax can be used to import data: sqoop import \ --connect ...
Sam Joe
Apr 21, 2016 at 3:26 pm
Apr 21, 2016 at 3:26 pm -
I am trying to sqoop data from an oracle database and the fieldnames contain the / character. e.g. (/BEV1/LULEINH). This causes the sqoop to fail issuing the error text listed bellow: Is it possible ...
Scott Shaw
Apr 21, 2016 at 2:24 pm
Apr 21, 2016 at 2:24 pm -
Hi, I'm facing some issues while trying to create a link in Sqoop 1.99.6. sqoop:000 update link --lid 1 Updating link with id 1 Please update link: Name: OracleLinkqa Link configuration JDBC Driver ...
Sam Joe
Apr 19, 2016 at 5:50 pm
Apr 19, 2016 at 6:27 pm -
Hi Everyone, I am just trying to understand if there is any default 'data encryption/decryption' involved when we sqoop the data from rdbms to hive. If there is so, can someone point me to a material ...
Ajay Chander
Apr 19, 2016 at 3:17 pm
Apr 19, 2016 at 5:09 pm -
Hi Everyone, I have used Sqoop to import the data from rdbms to Hadoop. I was just wondering if Sqoop provides the flexibility to import the .csv files stored on a local or remote file system to ...
Ajay Chander
Apr 18, 2016 at 8:28 pm
Apr 19, 2016 at 4:07 am -
Hi Zuo, this kind of question is best asked on our user mailing list. Please see [1] for instructions how to sign up. To answer your question, SQOOP-1493 is not changing the mapping by default as ...
Jarek Jarcec Cecho
Apr 18, 2016 at 2:15 pm
Apr 18, 2016 at 2:15 pm -
Hi, I am using CDH 5.7.0 and using sqoop client api to import a table from MySQL to HIVE. Am using the sqoop-connector-kite-1.99.5-cdh5.7.0.jar The import fails with the following message ...
Suresh Moorthi
Apr 15, 2016 at 10:29 am
Apr 15, 2016 at 10:42 am -
hi i've just started to trying sqoop. i want to try connect sqoop to teradata but i've fail. this is a message that i've got error message FYI this is what i do: 1. i'm using sqoop 1.4.6. (not cdh ...
Tkg_cangkul
Apr 15, 2016 at 4:02 am
Apr 15, 2016 at 12:34 pm -
Hi, I am importing data from teradata to hive using Sqoop. I added --map-column-hive in sqoop import command --map-column-hive num_int=int, num_float=float ,num_dec=decimal,ts=timestamp, ...
Devender Yadav
Apr 13, 2016 at 10:43 am
Apr 13, 2016 at 10:43 am -
I'm using Sqoop to import tables from an Oracle database, but it's very slow, so I'm trying to use direct mode in order to speed up the import. Although standard sqoop with 1 mapper works fine, sqoop ...
Luca Piccolo
Apr 11, 2016 at 8:52 am
Apr 11, 2016 at 8:52 am -
Hi, I'm running Apache Sqoop (v 1.4.6 from the Cloudera distribution), I have a cluster on AWS with 1 Master, 1 Name, and 5 worker nodes (m4.4xlarge) and I'm trying to import a MySQL (v 5.6) table ...
Tobias Feldhaus
Apr 9, 2016 at 9:34 pm
Apr 11, 2016 at 8:46 pm -
Hi all, I got Invalid HTTP request error with message "Detected incorrect link config structure" when using REST API to create a MySQL link. My sqoop version is 1.99.6. I am using "POST /v1/link" API ...
Preston Cheung
Apr 8, 2016 at 9:54 am
Apr 8, 2016 at 9:54 am -
I’ve been looking through Sqoop to see if there are any dependencies on YARN. It seems that Sqoop 1 only submits through MapReduce and has no direct dependency on ResourceManager or NodeManagers. Is ...
Foster, Craig
Apr 5, 2016 at 5:52 pm
Apr 5, 2016 at 9:30 pm -
Hi, I'm running an oozie coordinator to periodically run an oozie workflow which runs an *incremental* sqoop import from Postgres to Parquet files. The coordinator was running OK, when no new data ...
Silva João
Apr 5, 2016 at 3:47 pm
Apr 6, 2016 at 9:08 am -
Can Sqoop perform conversion from flat / CSV data files into Parquet or Avro? I know Sqoop's primary purpose is RDBMS <- Hadoop, but it seems like it could be a good file format conversion utility as ...
Matt
Mar 28, 2016 at 6:08 pm
Mar 28, 2016 at 6:11 pm -
Hi, I try to import the table from oracle into hdfs by using SQOOP Commands. Command is: sqoop import --connect "jdbc:oracle:thin:@ipaddress:port:XE" --username root --password *** --table TabName ...
prabhu Mahendran
Mar 28, 2016 at 8:37 am
Mar 28, 2016 at 8:37 am -
Hi, Sqoop cannot import to Hive with incremental, but why? If consecutive imports are possible, why not an incremental? It seems to me that "--incremental append" only interferes with the origin of ...
Silva João
Mar 23, 2016 at 2:22 pm
Mar 23, 2016 at 2:22 pm -
Hi, I'm trying to use sqoop import with the where argument in which I have to use backticks when importing data from mysql. The original where clause is: `primary` 5 I have to escape the backticks ...
Ronald Green
Mar 22, 2016 at 3:04 pm
Mar 22, 2016 at 7:14 pm -
Hi, I am trying to import data from Netezza to HDFS. I am able to successfully import few tables. However I am getting following error message for few tables. Failing tables have primary key ...
Dadi, Aruna
Mar 22, 2016 at 2:50 pm
Mar 22, 2016 at 2:50 pm -
Hi, I'm trying to import data from external database to hive with Parquet option. But failed in the kerberos environment. It can success without kerberos. Since Parquet is new added on the 1.4.6, is ...
Ping PW Wang
Mar 22, 2016 at 10:13 am
Mar 25, 2016 at 3:11 pm -
Hi folks, I am trying to export data from HDFS to SQL Server. Theoriginal table has over 500 columns and every time I execute Sqoop export jobit gets stuck showing mapreduce completed at 100%. I ...
Rajkumar
Mar 17, 2016 at 4:29 am
Mar 20, 2016 at 12:13 am -
Is sqoop import using subpartition supported? I am using the following command. If I use partition name, then it works. If I only specify subpartition name then it does not work. I get an error "does ...
Suraj shrestha
Mar 15, 2016 at 9:18 pm
Mar 15, 2016 at 9:18 pm -
There will be Falcon content and also Sqoop content with user stories on using Falcon and Sqoop along with other components. Please see the following info == Hadoop Data Management and Data Movement ...
Venkat Ranganathan
Mar 14, 2016 at 10:43 pm
Mar 25, 2016 at 4:59 am -
Hi guys I`m trying to get a grip on sqoop2. I`m running hadoop2 cluster with 2 nodes, for yarn there is 28GB od memory, 24 cores and 4 disks available minimum allocation resources for yarn container ...
Michal Vince
Mar 14, 2016 at 2:43 pm
Mar 16, 2016 at 8:50 am -
Hi Sqoop experts - I'm exploring options to run Sqoop via a Java program. I could write this using the com.cloudera.sqoop.SqoopOptions and com.cloudera.sqoop.tool.ImportTool classes. However, these ...
Jagrut Sharma
Mar 11, 2016 at 10:14 pm
Mar 11, 2016 at 11:55 pm -
Hi, I'm trying to export from a directory of Parquet files to MySQL and I'm getting an error about a ".metadata" file not existing in HDFS. Reading https://issues.apache.org/jira/browse/SQOOP-1394 I ...
Rendon, Carlos (KBB - Irvine)
Mar 3, 2016 at 11:57 pm
Mar 3, 2016 at 11:57 pm -
The orphaned sqoop task issue described in SQOOP-1916 <https://issues.apache.org/jira/browse/SQOOP-1916 was observed in production on sqoop 1.99.5 and 1.99.6 (no other releases were tested). This ...
Scott Kuehn
Mar 2, 2016 at 5:14 pm
Mar 7, 2016 at 5:31 pm -
On behalf of Apache Sqoop PPMC, I am excited to welcome Abraham Fine as a new committer on Apache Sqoop. Please join me in gratulations! Abe has added several new features, most notably Parquet ...
Jarek Jarcec Cecho
Mar 1, 2016 at 8:08 pm
Mar 9, 2016 at 4:57 am -
Hi Everyone, I just have a question about using sqoop incremental operation with --as-avrodatafile. Let's say initially I did a sqoop import to pull the table 'xyz' from mysql db to HDFS with option ...
Ajay Chander
Mar 1, 2016 at 8:01 pm
Mar 2, 2016 at 4:08 pm -
Does sqoop2 provide a mechanism to configure jobs to run in ad-hoc scheduler pools? By ad-hoc, I mean a scheduler pool that is not necessarily the same as the pool configured in the sqoop2 server's ...
Scott Kuehn
Feb 29, 2016 at 6:33 pm
Mar 2, 2016 at 2:13 am -
Hi, I'm running a Sqoop job in a in-house hadoop cluster, and would like to output to a S3 location. I didn't find much talking about this type request. So, does Sqoop support this action? thanks, Wei
Wei Yan
Feb 25, 2016 at 2:18 am
Feb 29, 2016 at 6:45 pm -
Hi, I'm using Sqoop to extract some data that contains double-quotes which I want to escape. My data looks like: This is just some "test" data When I run sqoop it extracts the data as: This is just ...
Ted Dong
Feb 17, 2016 at 6:25 pm
Feb 17, 2016 at 6:25 pm -
Hi, Does Sqoop 1.4.6 provide REST APIs to create/manage/execute import/export jobs? The developer guide does not include any details around it. Thanks & Regards, Sudeep
Sudeep mishra
Feb 17, 2016 at 4:43 am
Feb 24, 2016 at 3:08 pm
Group Overview
group | user |
categories | sqoop, hadoop |
discussions | 1,030 |
posts | 3,698 |
users | 539 |
website | sqoop.apache.org |
Top users
Archives
- June 2016 (4)
- May 2016 (16)
- April 2016 (36)
- March 2016 (40)
- February 2016 (38)
- January 2016 (32)
- December 2015 (32)
- November 2015 (42)
- October 2015 (25)
- September 2015 (48)
- August 2015 (21)
- July 2015 (129)
- June 2015 (55)
- May 2015 (127)
- April 2015 (57)
- March 2015 (76)
- February 2015 (55)
- January 2015 (19)
- December 2014 (59)
- November 2014 (67)
- October 2014 (22)
- September 2014 (136)
- August 2014 (132)
- July 2014 (132)
- June 2014 (88)
- May 2014 (30)
- April 2014 (65)
- March 2014 (73)
- February 2014 (61)
- January 2014 (81)
- December 2013 (50)
- November 2013 (77)
- October 2013 (60)
- September 2013 (137)
- August 2013 (91)
- July 2013 (162)
- June 2013 (129)
- May 2013 (79)
- April 2013 (60)
- March 2013 (87)
- February 2013 (41)
- January 2013 (65)
- December 2012 (51)
- November 2012 (31)
- October 2012 (67)
- September 2012 (62)
- August 2012 (47)
- July 2012 (105)
- June 2012 (81)
- May 2012 (41)
- April 2012 (39)
- March 2012 (78)
- February 2012 (41)
- January 2012 (45)
- December 2011 (37)
- November 2011 (12)
- October 2011 (38)
- September 2011 (35)
- August 2011 (39)
- July 2011 (12)
- June 2011 (1)