FAQ
Good afternoon,

I am using pig on server logs to make statistics on visited pages.

For now I am able to do such matches:

- one user has visited a given page matching a given aim.
- one user has visited a given page belonging to one of the page's aim.
*
aim.id file:*

1@1@add_to_kart --> aim 1 can be add_to_cart
1@1@browse --> or aim 1 can be browse
1@2@paid --> aim 2 is only paid

*site.log file:*

user1,1,www.site.com/browse
user1,1,www.site.com/browse
user2,1,www.site.com/add_to_kart
user2,1,www.site.com/add_to_kart
user2,1,www.site.com/paid
user2,1,www.site.com/browse
user3,1,www.site.com/browse

*Pig script:*

register 'piggybank.jar';

-- load aim id database:
aim_ids = LOAD 'aim.id' USING PigStorage('@') AS (aim_site_id : int, aim_id
: int , aim_url:chararray);

DUMP aim_ids;

-- load site log:
site = LOAD 'site.log' USING PigStorage(',') AS (user_id : chararray ,
site_id : int, url : chararray);

site_all_aims = JOIN site BY site_id, aim_ids BY aim_site_id;

site_match = FOREACH site_all_aims GENERATE user_id, site_id, aim_id,
org.apache.pig.piggybank.evaluation.string.INDEXOF(url, aim_url) AS match;

site_aims = FILTER site_match BY (match != -1) AND (match IS NOT null);

DUMP site_aims;

*results:*

(user1,1,1,13) --> user 1 achieved aim 1
(user1,1,1,13) --> user 1 achieved aim 1
(user2,1,1,13) --> user 2 achieved aim 1
(user2,1,1,13) --> user 2 achieved aim 1
(user2,1,2,13) --> user 2 achieved aim 2
(user2,1,1,13) --> user 2 achieved aim 1
(user3,1,1,13) --> user 3 achieved aim 1


Now I would like to check that a user has visited several pages to achieve
one aim. Like for a user to achieve aim 3, he needs to visit "browse" AND
"add_to_kart" AND "paid".

My idea was to load tuples of aim:

1@3@{(browse),(add_to_kart),(paid)}

And to write an UDF to compare aim URL tuple, with user's visited URL bag
for the site.

But I am not able to load tuples with an undefined number of elements. As
aims might be:

1@3@{(browse),(add_to_kart),(paid)}
1@4
@{(browse_something_else),(add_to_kart_something_else),(paid_something_else),(another_page)}

So finally, I am stuck with this problem right now, still searching for
another way to write this script and aim.id file.

If any of you as any idea, mail me.

Thanks

Vincent HERVIEUX

Search Discussions

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedSep 6, '10 at 2:06p
activeSep 6, '10 at 2:06p
posts1
users1
websitepig.apache.org

1 user in discussion

Vincent: 1 post

People

Translate

site design / logo © 2021 Grokbase