DFS should detect slow links(nodes) and avoid them

Key: HADOOP-3589
URL: https://issues.apache.org/jira/browse/HADOOP-3589
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Reporter: Runping Qi

The current DFS does not detect slow links (nodes).
Thus, when a node or its network link is slow, it may affect the overall system performance significantly.
Specifically, when a map job needs to read data from such a node, it may progress 10X slower.
And when a DFS data node pipeline consists of such a node, the write performance degrades significantly.
This may lead to some long tails for map/reduce jobs. We have experienced such behaviors quite often.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupcommon-dev @
postedJun 18, '08 at 1:21a
activeJun 18, '08 at 1:21a

1 user in discussion

Runping Qi (JIRA): 1 post



site design / logo © 2022 Grokbase