I successfully installed two data nodes and a manger node. Worked out a few
minor kinks and all is working fine with small jobs however when I tried to
run TeraGen it crashed. After some investigation I believe I found the root

I did a fresh minimal install of CentOS 6.4 on all machines. The two data
nodes both have 6ea 300G disk drives. CentOS installs lvm2 by default. Now
I only get Cluster Configured Capacity of ~89G when I was expecting ~1.3T
on each machine.

[root@IBM ~]# sudo -u hdfs hdfs dfsadmin -report
Configured Capacity: 95120437248 (88.59 GB)
Present Capacity: 94225088847 (87.75 GB)
DFS Remaining: 94098546688 (87.64 GB)
DFS Used: 126542159 (120.68 MB)
DFS Used%: 0.13%
Under replicated blocks: 98
Blocks with corrupt replicas: 0
Missing blocks: 0

Datanodes available: 2 (2 total, 0 dead)

Live datanodes:
Name: (Dell.test.net)
Hostname: Dell.test.net
Rack: /default
Decommission Status : Normal
Configured Capacity: 47560218624 (44.29 GB)
DFS Used: 63279439 (60.35 MB)
Non DFS Used: 328152753 (312.95 MB)
DFS Remaining: 47168786432 (43.93 GB)
DFS Used%: 0.13%
DFS Remaining%: 99.18%
Last contact: Wed Apr 03 14:33:48 PDT 2013

Name: (IBM.test.net)
Hostname: IBM.test.net
Rack: /default
Decommission Status : Normal
Configured Capacity: 47560218624 (44.29 GB)
DFS Used: 63262720 (60.33 MB)
Non DFS Used: 567195648 (540.92 MB)
DFS Remaining: 46929760256 (43.71 GB)
DFS Used%: 0.13%
DFS Remaining%: 98.67%
Last contact: Wed Apr 03 14:33:49 PDT 2013

So I read a lot of threads about issues with creating multiple hdfs
directories on partitions and looked for something about lvm2 and have
found one thread from some time back that indicated lvm2 was a bad idea for
MapR. Obviously I have a problem and I do now understand that mapr will
under perform with this structure.

I can break 5 disks out of the lvm on each machine if needed but is this
the best (only) solution? Then the question is do I have to do other clean
up of hdfs to get it to play nice since some of it may get left on the lvm
drive. Any hints on getting the 5 disks hooked back into hdfs would be
helpful but I have seen some threads on add new drives so I have a (very)
rough idea of what's required.



Search Discussions

Discussion Posts

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 1 of 4 | next ›
Discussion Overview
groupscm-users @
postedApr 3, '13 at 9:52p
activeApr 4, '13 at 1:57a

2 users in discussion

Mike Mc: 3 posts Philip Zeyliger: 1 post



site design / logo © 2022 Grokbase