FAQ
Hello!
I would like to know how to obtain for a file (via Java API) the following information
- how many blocks occupies in the HDFS
-which are the machines where the blocks were replicated
I look forward for your answers.
Thank you,
Florin

Search Discussions

  • Harsh J at Jul 28, 2011 at 2:11 pm
    Hello,

    What version of Hadoop are you using?
    On Thu, Jul 28, 2011 at 6:03 PM, Florin P wrote:
    Hello!
    I would like to know how to obtain for a file (via Java API) the following information
    - how many blocks occupies in the HDFS
    Is this request per file? or overall? [If overall, I think metrics are
    your best bet]
    -which are the machines where the blocks were replicated
    If this request is per-file, then FileSystem#getBlockLocations(…) API
    can help you. Just request it over the entire size of the file
    (FileStatus information can be got via FileSystem#getFileStatus(…))

    --
    Harsh J
  • Florin P at Jul 29, 2011 at 6:26 am
    Hello!
    Thank you for your answers. I'm using 0.20 API.
    My request are per file, and I would like to have the same results that are displayed in the when you are using the web interface for browsing the file system (brosweBlock.jsp page).
    Regards,
    Florin

    P.S. My results should look like this


    filename filesize blocks machine_block_replication

    --- On Thu, 7/28/11, Harsh J wrote:
    From: Harsh J <harsh@cloudera.com>
    Subject: Re: How to obtain block information of a HDFS file?
    To: hdfs-user@hadoop.apache.org
    Date: Thursday, July 28, 2011, 10:10 AM
    Hello,

    What version of Hadoop are you using?
    On Thu, Jul 28, 2011 at 6:03 PM, Florin P wrote:
    Hello!
    I would like to know how to obtain for a file (via
    Java API) the following information
    - how many blocks occupies in the HDFS
    Is this request per file? or overall? [If overall, I think
    metrics are
    your best bet]
    -which are the machines where the blocks were
    replicated

    If this request is per-file, then
    FileSystem#getBlockLocations(…) API
    can help you. Just request it over the entire size of the
    file
    (FileStatus information can be got via
    FileSystem#getFileStatus(…))

    --
    Harsh J
  • Harsh J at Jul 29, 2011 at 6:30 am
    Florin,

    Right, all that info is gettable via the FileStatus objects for a given Path.
    On Fri, Jul 29, 2011 at 11:55 AM, Florin P wrote:
    Hello!
    Thank you for your answers. I'm using 0.20 API.
    My request are per file, and I would like to have the same results that are displayed in the when you are using the web interface for browsing the file system (brosweBlock.jsp page).
    Regards,
    Florin

    P.S. My results should look like this


    filename  filesize blocks machine_block_replication

    --- On Thu, 7/28/11, Harsh J wrote:
    From: Harsh J <harsh@cloudera.com>
    Subject: Re: How to obtain block information of a HDFS file?
    To: hdfs-user@hadoop.apache.org
    Date: Thursday, July 28, 2011, 10:10 AM
    Hello,

    What version of Hadoop are you using?

    On Thu, Jul 28, 2011 at 6:03 PM, Florin P <florinpico@yahoo.com>
    wrote:
    Hello!
    I would like to know how to obtain for a file (via
    Java API) the following information
    - how many blocks occupies in the HDFS
    Is this request per file? or overall? [If overall, I think
    metrics are
    your best bet]
    -which are the machines where the blocks were
    replicated

    If this request is per-file, then
    FileSystem#getBlockLocations(…) API
    can help you. Just request it over the entire size of the
    file
    (FileStatus information can be got via
    FileSystem#getFileStatus(…))

    --
    Harsh J


    --
    Harsh J

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-user @
categorieshadoop
postedJul 28, '11 at 12:33p
activeJul 29, '11 at 6:30a
posts4
users2
websitehadoop.apache.org...
irc#hadoop

2 users in discussion

Harsh J: 2 posts Florin P: 2 posts

People

Translate

site design / logo © 2022 Grokbase