[Hadoop-common-user] Having multiple values in Value field

Jason Venner

at Oct 6, 2009 at 6:03 am

You can always pass them as comma delimited strings, which is what you are
already doing with your python streaming code, and then use Text as your
value.

On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
wrote:

I am having issues having multiple values in my value field.My desired
result is
<key>,<float,int> or even <key>,<float,float>.

It seems easy in Python where I can pass a tuple as value.What is the best
way to do this in Java.
I have tried ArrayWritable but it looks like I need to write my own class
like IntArrayWritable.

Thanks,
Akshaya

--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

reply | permalink

Amogh Vasekar

at Oct 6, 2009 at 7:55 am

⇧

You can always pass them as comma delimited strings

Which would be pretty expensive per <k,v> right? Would avro be looking into solving such problems?

Amogh

-----Original Message-----
From: Jason Venner
Sent: Tuesday, October 06, 2009 11:33 AM
To: common-user@hadoop.apache.org
Subject: Re: Having multiple values in Value field

You can always pass them as comma delimited strings, which is what you are
already doing with your python streaming code, and then use Text as your
value.

On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
wrote:

I am having issues having multiple values in my value field.My desired
result is
<key>,<float,int> or even <key>,<float,float>.

It seems easy in Python where I can pass a tuple as value.What is the best
way to do this in Java.
I have tried ArrayWritable but it looks like I need to write my own class
like IntArrayWritable.

Thanks,
Akshaya

--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

reply | permalink

Tom Chen

at Oct 6, 2009 at 4:28 pm

⇧

Hi Akshaya,

Take a look at the yahoo hadoop tutorial for custom data types.

http://developer.yahoo.com/hadoop/tutorial/module5.html#types

It's actually quite easy to create your own types and stream them. You can
use the

void readFields(DataInput in);
void write(DataOutput out);

methods to initialize your objects.

Tom

On Tue, Oct 6, 2009 at 12:52 AM, Amogh Vasekar wrote:
You can always pass them as comma delimited strings

Which would be pretty expensive per <k,v> right? Would avro be looking
into solving such problems?
Amogh

-----Original Message-----
From: Jason Venner
Sent: Tuesday, October 06, 2009 11:33 AM
To: common-user@hadoop.apache.org
Subject: Re: Having multiple values in Value field

You can always pass them as comma delimited strings, which is what you are
already doing with your python streaming code, and then use Text as your
value.

On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
wrote:

I am having issues having multiple values in my value field.My desired
result is
<key>,<float,int> or even <key>,<float,float>.

It seems easy in Python where I can pass a tuple as value.What is the

best

way to do this in Java.
I have tried ArrayWritable but it looks like I need to write my own class
like IntArrayWritable.

Thanks,
Akshaya

--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

reply | permalink

Akshaya iyengar

at Oct 7, 2009 at 1:00 am

⇧

Thanks Tom. The link really was helpful. the CSVs were getting nasty to
handle.

On Tue, Oct 6, 2009 at 12:27 PM, Tom Chen wrote:

Hi Akshaya,

Take a look at the yahoo hadoop tutorial for custom data types.

http://developer.yahoo.com/hadoop/tutorial/module5.html#types

It's actually quite easy to create your own types and stream them. You can
use the

void readFields(DataInput in);
void write(DataOutput out);

methods to initialize your objects.

Tom

On Tue, Oct 6, 2009 at 12:52 AM, Amogh Vasekar wrote:
You can always pass them as comma delimited strings

Which would be pretty expensive per <k,v> right? Would avro be looking
into solving such problems?
Amogh

-----Original Message-----
From: Jason Venner
Sent: Tuesday, October 06, 2009 11:33 AM
To: common-user@hadoop.apache.org
Subject: Re: Having multiple values in Value field

You can always pass them as comma delimited strings, which is what you are
already doing with your python streaming code, and then use Text as your
value.

On Mon, Oct 5, 2009 at 10:54 PM, akshaya iyengar
wrote:

I am having issues having multiple values in my value field.My desired
result is
<key>,<float,int> or even <key>,<float,float>.

It seems easy in Python where I can pass a tuple as value.What is the

best

way to do this in Java.
I have tried ArrayWritable but it looks like I need to write my own

class

like IntArrayWritable.

Thanks,
Akshaya

--
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

reply | permalink

group	common-user @
categories	hadoop
posted	Oct 6, '09 at 5:55a
active	Oct 7, '09 at 1:00a
posts	5
users	4
website	hadoop.apache.org...
irc	#hadoop

responses	expanded	Hotkey: s
font	variable	Hotkey: f
user style	avatars	Hotkey: a

[Hadoop-common-user] Having multiple values in Value field

Search Discussions

Related Discussions

4 users in discussion