FAQ

Question related to Number of Mapper

Ramasubramanian Narayanan
Nov 7, 2012 at 4:22 pm
Hi,

If a Custer’s HDFS block size is 64MB. You have a directory containing 100
plain text files, each of which Is 100MB in size. The InputFormat for your
job is TextInputFormat. How many Mappers will run?

regards,
Rams
reply

Search Discussions

3 responses

  • Michael Segel at Nov 7, 2012 at 4:32 pm
    0 Custer didn't run. He got surrounded and then massacred. :-P (See Custer's last stand at Little Big Horn)

    Ok... plain text files 100 files 2 blocks each would by default attempt to schedule 200 mappers.

    Is this one of those online Cert questions?
    On Nov 7, 2012, at 10:20 AM, Ramasubramanian Narayanan wrote:

    Hi,

    If a Custer’s HDFS block size is 64MB. You have a directory containing 100 plain text files, each of which Is 100MB in size. The InputFormat for your job is TextInputFormat. How many Mappers will run?

    regards,
    Rams

  • Ramasubramanian Narayanan at Nov 7, 2012 at 4:47 pm
    Hi,

    Thanks!

    But it is given as 100 Mappers... I think we can also use 'n' number of
    Mappers as the same as the number of input files... (not for this
    question)... If you know more detail on that please share..

    Note : I forgot from where this question I taken :)

    regards,
    Rams.
    On Wed, Nov 7, 2012 at 10:01 PM, Michael Segel wrote:

    0 Custer didn't run. He got surrounded and then massacred. :-P (See
    Custer's last stand at Little Big Horn)

    Ok... plain text files 100 files 2 blocks each would by default attempt to
    schedule 200 mappers.

    Is this one of those online Cert questions?

    On Nov 7, 2012, at 10:20 AM, Ramasubramanian Narayanan <
    ram...@...com> wrote:
    Hi,

    If a Custer’s HDFS block size is 64MB. You have a directory containing
    100 plain text files, each of which Is 100MB in size. The InputFormat for
    your job is TextInputFormat. How many Mappers will run?
    regards,
    Rams

  • Michael Segel at Nov 7, 2012 at 5:31 pm
    The larger question is how many blocks are required to store a 100MB file if the HDFS block size is 64MB.

    If it takes 2 blocks then when you run your job, you will have 1 mapper per block, unless the file is not splittable. (But from your example its a simple text file which is splittable.)

    Again, where are you getting these questions?

    On Nov 7, 2012, at 10:46 AM, Ramasubramanian Narayanan wrote:

    Hi,

    Thanks!

    But it is given as 100 Mappers... I think we can also use 'n' number of Mappers as the same as the number of input files... (not for this question)... If you know more detail on that please share..

    Note : I forgot from where this question I taken :)

    regards,
    Rams.

    On Wed, Nov 7, 2012 at 10:01 PM, Michael Segel wrote:
    0 Custer didn't run. He got surrounded and then massacred. :-P (See Custer's last stand at Little Big Horn)

    Ok... plain text files 100 files 2 blocks each would by default attempt to schedule 200 mappers.

    Is this one of those online Cert questions?
    On Nov 7, 2012, at 10:20 AM, Ramasubramanian Narayanan wrote:

    Hi,

    If a Custer’s HDFS block size is 64MB. You have a directory containing 100 plain text files, each of which Is 100MB in size. The InputFormat for your job is TextInputFormat. How many Mappers will run?

    regards,
    Rams

Related Discussions

Discussion Navigation
viewthread | post