FAQ
Improve the speed of FileSystem object creation
-----------------------------------------------

Key: HDFS-1127
URL: https://issues.apache.org/jira/browse/HDFS-1127
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 0.20.2, 0.20.1
Reporter: Zheng Shao
Assignee: Zheng Shao


It takes a long time to create a lot of {{DistributedFileSystem}} object via {{FileSystem.create}} or {{FileSystem.createNewInstance}} function.
In our experience with scribe-hdfs, it can take up to 2 minutes for 100 such calls from each node of a cluster of 40 nodes.

The reason for the delay is mainly because of the lock on the CACHE object. We should release the lock while creating new FileSystem objects.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
grouphdfs-dev @
categorieshadoop
postedMay 4, '10 at 4:09a
activeMay 4, '10 at 4:09a
posts1
users1
websitehadoop.apache.org...
irc#hadoop

1 user in discussion

Zheng Shao (JIRA): 1 post

People

Translate

site design / logo © 2022 Grokbase