FAQ
Improve FsShell's heap consumption by switching to listStatus that returns an iterator
--------------------------------------------------------------------------------------

Key: HADOOP-6732
URL: https://issues.apache.org/jira/browse/HADOOP-6732
Project: Hadoop Common
Issue Type: Improvement
Reporter: Hairong Kuang
Fix For: 0.22.0


When listing a large directory from the command line using the default heap configuration, FsShell often runs out of memory. This is because all stats of the entries under the directory need to be in memory before printing them. The new API listStatus that returns an iterator of FileStatus, which implemented in HDFS-1091, no longer requires that all entries are fetched first. Thus switching to this new API will greatly improve the use of heap space.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
categorieshadoop
postedApr 29, '10 at 12:38a
activeApr 29, '10 at 12:38a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Hairong Kuang (JIRA): 1 post

People

Translate

site design / logo © 2022 Grokbase