Hi Alan,
I have tested the following code , but sometimes exception will been throw
out:
/
*********************************************************************************************************
ExecutorService executor = new ScheduledThreadPoolExecutor(10);
Callable<Long> totalVisitorTaks = new Callable<Long>() {
@Override
public Long call() throws Exception {
PigServer pig=new PigServer(ExecType.MAPREDUCE);
pig.register(..........);
}
};
Callable<Map<String, Long>> visitorBySourceTask = new
Callable<Map<String, Long>>() {
@Override
public Map<String, Long> call() throws Exception {
PigServer pig=new PigServer(ExecType.MAPREDUCE);
pig.register(..........);
}
};
Future<Long> totalVisitor = executor.submit(totalVisitorTaks);
Future<Map<String, Long>> visitBySource =
executor.submit(visitBySourceTask);
/
*********************************************************************************************************
It seems I can not execute two pig scripts in different threads at the same
time. I think it would be better if Pig can support this.
Because one job one time can not use the resource efficiently. What do you
think ?
Jeff zhang.
On Fri, Jun 19, 2009 at 11:16 PM, Alan Gates wrote:On Jun 19, 2009, at 2:06 AM, zhang jianfeng wrote:
First question: I use the one single PigServer to run serveral pig scripts
in multiple threads, but some exceptions will be throw, so Pig do not
support multi thread, Is it right ? I just want to make sure.
Correct, Pig is not multi-threaded at this point. We have tried to design
it such that changing that in the future would not be too much work. But as
we don't test multi-threaded, I'm sure there are some shared static
variables, etc.
Alan.
