FAQ
Hi,

I am unable to use sqoop and want to load data in hdfs for testing,
Is there any way by which i can load my csv or text file to hadoop
file system directly without writing code in java

Regards
Prabhjot

Search Discussions

  • Bejoy Ks at Jul 19, 2012 at 10:22 am
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Mohammad Tariq at Jul 19, 2012 at 10:25 am
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Iwannaplay games at Jul 19, 2012 at 10:33 am
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Mohammad Tariq at Jul 19, 2012 at 10:37 am
    Could you show me the structure of your csv, if possible??

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games
    wrote:
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Iwannaplay games at Jul 19, 2012 at 10:43 am
    PFA the csv file

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Could you show me the structure of your csv, if possible??

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games
    wrote:
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column
    families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks <bejoy.hadoop@gmail.com>
    wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Mohammad Tariq at Jul 19, 2012 at 11:15 am
    You have a few options. You can write a java program using Hbase API
    to do that. But you won't be able to exploit the parallelism to its
    fullest. Another option is to write a MapReduce program to do the
    same. You can also use Hive and Pig to serve the purpose. But if you
    are just starting with Hbase, then I would suggest to first get
    yourself familiar with Hbase API first and then go for other things.

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:12 PM, iwannaplay games
    wrote:
    PFA the csv file

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Could you show me the structure of your csv, if possible??

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games
    wrote:
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column
    families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks <bejoy.hadoop@gmail.com>
    wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Mohammad Tariq at Jul 19, 2012 at 11:23 am
    By looking at the csv, I would suggest to create an hbase table, say
    'IPINFO' with one column family, say 'cf' having 3 columns for
    'startip', 'endip' and 'countryname' respectively.

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 4:44 PM, Mohammad Tariq wrote:
    You have a few options. You can write a java program using Hbase API
    to do that. But you won't be able to exploit the parallelism to its
    fullest. Another option is to write a MapReduce program to do the
    same. You can also use Hive and Pig to serve the purpose. But if you
    are just starting with Hbase, then I would suggest to first get
    yourself familiar with Hbase API first and then go for other things.

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:12 PM, iwannaplay games
    wrote:
    PFA the csv file

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Could you show me the structure of your csv, if possible??

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games
    wrote:
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column
    families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks <bejoy.hadoop@gmail.com>
    wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Iwannaplay games at Jul 19, 2012 at 11:43 am
    Hi,

    I am sorry ,i am troubling you a lot.

    Thanks for helping me :)

    This file is there in my hdfs IPData ( both in csv and tsv format).
    Can i transfer the records from this file and populate this data using
    importtsv?
    If yes then what source and destination should i write.

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    By looking at the csv, I would suggest to create an hbase table, say
    'IPINFO' with one column family, say 'cf' having 3 columns for
    'startip', 'endip' and 'countryname' respectively.

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 4:44 PM, Mohammad Tariq wrote:
    You have a few options. You can write a java program using Hbase API
    to do that. But you won't be able to exploit the parallelism to its
    fullest. Another option is to write a MapReduce program to do the
    same. You can also use Hive and Pig to serve the purpose. But if you
    are just starting with Hbase, then I would suggest to first get
    yourself familiar with Hbase API first and then go for other things.

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:12 PM, iwannaplay games
    wrote:
    PFA the csv file

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Could you show me the structure of your csv, if possible??

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games
    wrote:
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column
    families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks <bejoy.hadoop@gmail.com>
    wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Mohammad Tariq at Jul 19, 2012 at 12:21 pm
    There is absolutely no need to be sorry. And once you have the data
    inside your Hdfs you can use importtsv to imort the data like this :
    $ bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
    -Dimporttsv.columns=a,b,c <tablename> <hdfs-inputdir>


    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 5:12 PM, iwannaplay games
    wrote:
    Hi,

    I am sorry ,i am troubling you a lot.

    Thanks for helping me :)

    This file is there in my hdfs IPData ( both in csv and tsv format).
    Can i transfer the records from this file and populate this data using
    importtsv?
    If yes then what source and destination should i write.

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    By looking at the csv, I would suggest to create an hbase table, say
    'IPINFO' with one column family, say 'cf' having 3 columns for
    'startip', 'endip' and 'countryname' respectively.

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 4:44 PM, Mohammad Tariq wrote:
    You have a few options. You can write a java program using Hbase API
    to do that. But you won't be able to exploit the parallelism to its
    fullest. Another option is to write a MapReduce program to do the
    same. You can also use Hive and Pig to serve the purpose. But if you
    are just starting with Hbase, then I would suggest to first get
    yourself familiar with Hbase API first and then go for other things.

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:12 PM, iwannaplay games
    wrote:
    PFA the csv file

    Thanks
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Could you show me the structure of your csv, if possible??

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games
    wrote:
    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column
    families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq


    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks <bejoy.hadoop@gmail.com>
    wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • Harsh J at Jul 19, 2012 at 12:42 pm
    Hi,

    This is a question suited for the HBase user lists (user@hbase.apache.org).
    In any case, you want to use the importtsv tool:
    http://hbase.apache.org/book.html#importtsv. You do not have to write any
    custom code to do it.
    On Thu, Jul 19, 2012 at 4:03 PM, iwannaplay games wrote:

    Thanks Tariq

    Now i want to convert this csv file to table.(HBase table with column
    families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot


    --
    Harsh J
  • Lars hofhansl at Jul 20, 2012 at 6:56 am
    Check out HBase's importtsv. (http://hbase.apache.org/book/ops_mgt.html#importtsv)


    -- Lars


    ----- Original Message -----
    From: iwannaplay games <funnlearnforkids@gmail.com>
    To: hdfs-user@hadoop.apache.org
    Cc:
    Sent: Thursday, July 19, 2012 3:33 AM
    Subject: Re: Loading data in hdfs

    Thanks Tariq

    Now i want to convert  this csv file to table.(HBase table with column families)
    How can i do that

    Regards
    Prabhjot

    On 7/19/12, Mohammad Tariq wrote:
    Hi Prabhjot

    You can also use :
    hadoop fs -put <src fs path> <destn hdfs path>

    Regards,
    Mohammad Tariq

    On Thu, Jul 19, 2012 at 3:52 PM, Bejoy Ks wrote:
    Hi Prabhjot

    Yes, Just use the filesystem commands
    hadoop fs -copyFromLocal <src fs path> <destn hdfs path>

    Regards
    Bejoy KS

    On Thu, Jul 19, 2012 at 3:49 PM, iwannaplay games
    wrote:
    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text  file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot
  • VIGNESH PRAJAPATI at Jul 19, 2012 at 10:23 am
    Hello,

    You can also do this by using Hive, But after that you have to deal with
    database and table for further processing.


    On Thu, Jul 19, 2012 at 6:19 AM, iwannaplay games wrote:

    Hi,

    I am unable to use sqoop and want to load data in hdfs for testing,
    Is there any way by which i can load my csv or text file to hadoop
    file system directly without writing code in java

    Regards
    Prabhjot


    --

    *Vignesh Prajapati*
    Tel: 9427415949 |
    vignesh2066@gmail.com | www.vipras.com.co.in
    MYTK [image: Facebook] <https://www.facebook.com/vigs143> [image:
    Twitter]<http://twitter.com/#%21/vigs143> [image:
    LinkedIn] <http://www.linkedin.com/pub/vignesh-prajapati/37/756/46a> [image:
    about.me] <http://www.way4fun.tk><http://r1.wisestamp.com/r/landing?promo=7&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_7>

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-user @
categorieshadoop
postedJul 19, '12 at 10:20a
activeJul 20, '12 at 6:56a
posts13
users6
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2021 Grokbase