My cluster nodes have 2 dual-core processors, so, in general, I want
to configure my nodes with a maximum of 3 task processes executed per
node at a time.
But, for some jobs, my tasks load large amounts of memory, and I
cannot fit 3 such tasks on a single node. For these jobs, I'd like to
enforce running a maximum of 1 task process per node at a time.
I've tried to enforce this by setting
mapred.tasktracker.map.tasks.maximum at runtime, but I see it has no
effect, because this is a configuration for the TaskTracker, which is
of course already running before my job starts.
Is there no way to configure a maximum # of map tasks per node on a
per-job basis?
Thanks!