Thanks for your help. However, when my code try to get "diagnosticInfo",
this value is empty, no logs inside, is there anything i need to set in
order to get the log from job? Below is the C# code that i developed:
*var socket = new TSocket("172.16.104.151",
* var transport = new TBufferedTransport(socket);*
* var protocol = new TBinaryProtocol(transport);*
* var client = new Jobtracker.Client(protocol);*
* RequestContext ctx = new RequestContext();*
* ctx.ConfOptions = new Dictionary<string, string>();*
* ctx.ConfOptions.Add("effective_user", "root");*
* ThriftJobID thriftJobID = new ThriftJobID();*
* thriftJobID.AsString = "job_201308051202_0057";*
* thriftJobID.JobID = 57;*
* thriftJobID.JobTrackerID = "201308051202";*
* var job = client.getJob(ctx, thriftJobID);*
* ThriftTaskInProgressList thriftTaskInProgressList =
* List <ThriftTaskInProgress> taskList =
* ThriftTaskStatus thriftTaskStatus = new ThriftTaskStatus();
* taskList.ForEach(l =>*
* Dictionary<string, ThriftTaskStatus>
thriftTaskStatusDic = l.TaskStatuses;*
* foreach (KeyValuePair<string, ThriftTaskStatus>
status in thriftTaskStatusDic)*
* thriftTaskStatus = status.Value;*
thriftTaskStatus.DiagnosticInfo); *//No logs shown here*
Any idea why the "diagnosticInfo" is empty?? Besides that, i have another
question. When i getAllJob, it will always return only 5 latest job, can we
set it to keep more than 5 jobs? You may refer to attached picture when i
try to retrieve the job, it will hit the error due to it only keep the
latest 5 jobs.
Thanks a lot !!
On Wednesday, August 7, 2013 12:26:05 AM UTC+8, abe wrote:
You should be able to use "getTask", which returns an object of type
"ThriftTaskStatus". You're looking for the member "diagnosticInfo" in that
I am able to generate c# client stubs and connect to job tracker thrift
server. However, when i get the job or task, I am not able to find a
function that i can get the log. Can anyone let me know which function from
thrift interface to query the logs for each task? Please find attached
picture (circles in red) for the logs that i want to retrieve.
Thanks a lot !!
On Monday, August 5, 2013 5:20:57 PM UTC+8, [email protected] wrote:
Thanks for your clear explanation. I will try it again using the
reference provided. Appreciate your help.
On Monday, August 5, 2013 12:25:11 PM UTC+8, abe wrote:
With thrift, there are servers and clients. Hue comes with the server
code in the form of a jar. The "org.apache.ahdoop.thriftfs.**ThriftJobTrackerPlugin"
property can use this jar to start a Thrift service (server).
The server and client communicate using a language agnostic protocol.
Thus, all you need to do is generate client "stubs". Then, in what ever
program you are writing, call these stub methods. Take a look athttp://diwakergupta.github.**io/thrift-missing-guide/<http://diwakergupta.github.io/thrift-missing-guide/
Here is what your C# code might look like (ripped directly from link
static void Main(string args)
var socket = new TSocket("localhost", 9888);
var transport = new TBufferedTransport(socket);
var protocol = new TBinaryProtocol(transport);
var client = new CalculatorService.Client(**protocol);
catch (Exception ex)
Notice how a socket, transport, protocol, and Client are instantiated.
The Client should be generated for you.
On Sun, Aug 4, 2013 at 8:40 PM, wrote:
I go through apache thrift and get to know a little about thrift (The
Apache Thrift software framework, for scalable cross-language services
development, combines a software stack with a code generation engine to
build services that work efficiently and seamlessly between C++, Java,
Smalltalk, OCaml and Delphi and other languages). So is this means that I
am able to generate C# code from the thrift file and then put the generated
code in hadoop environment. Then i will need to modify the below to point
to the source file that i created?
<description>Comma-separated list of jobtracker plug-ins to be activated.</description>
Appreciate your help. Thanks.
On Sunday, August 4, 2013 10:33:32 PM UTC+8, [email protected] wrote:
Thanks for your reply. However, is it I can use the same plugin and
call the thrift interface? Actually i still don't have any idea to call the
thrift interface from the program, could you shed me some light if
possible? Thanks a lot.
On Saturday, August 3, 2013 12:20:44 AM UTC+8, Romain Rigaux wrote:
Hue is using a plugin on the JobTracker to access the MapReduce
logs. It has a Thrift interface:https://github.com/cloudera/**hu**e/blob/master/desktop/libs/**had**
On Fri, Aug 2, 2013 at 3:02 AM, wrote:
As I know when we submit a Oozie job, we are able to retrieve the
job log through Oozie web services API (eg. GET
/oozie/v1/job/job-3?show=log).**** However, the logs that I got
are Oozie job logs. From Hue, I can click view logs to drill down and check
on the syslog which contains all the map reduce logs.
Is there any way for me to get the log of the task tracker showing
all the map reduce logs? Anyone got any idea on this? I am planning to
query real time map reduce log.
Appreciate your help. Thanks.
You received this message because you are subscribed to the Google
Groups "CDH Users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
For more options, visit https://groups.google.com/a/**cl**