FAQ
Hi,

I have a situation where I need to do urldecode on one particular column. Is there any hive built in function available?

Thank you,
Chalcy

Search Discussions

  • Loren Siebert at Sep 15, 2011 at 4:28 pm
    You need to write a UDF, like this person did:

    http://search-hadoop.com/m/HFWE32CYs6x/v=plain
    On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:

    Hi,

    I have a situation where I need to do urldecode on one particular column. Is there any hive built in function available?

    Thank you,
    Chalcy
  • Chalcy Raja at Sep 15, 2011 at 4:38 pm
    Thank you! Loren. So far I have only using hive as is. Looks like I have to start extending :)

    --Chalcy

    From: Loren Siebert
    Sent: Thursday, September 15, 2011 12:28 PM
    To: user@hive.apache.org
    Subject: Re: urldecode hive column

    You need to write a UDF, like this person did:

    http://search-hadoop.com/m/HFWE32CYs6x/v=plain

    On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:


    Hi,

    I have a situation where I need to do urldecode on one particular column. Is there any hive built in function available?

    Thank you,
    Chalcy
  • Carl Steinbach at Sep 15, 2011 at 6:37 pm
    It's possible to call Java static methods directly using the built-in
    'reflect' UDF. The performance will not be as good as writing your
    own UDF, but you'll get results a lot faster.

    On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja
    wrote:
    Thank you! Loren. So far I have only using hive as is. Looks like I
    have to start extending J****

    ** **

    --Chalcy****

    ** **

    *From:* Loren Siebert
    *Sent:* Thursday, September 15, 2011 12:28 PM
    *To:* user@hive.apache.org
    *Subject:* Re: urldecode hive column****

    ** **

    You need to write a UDF, like this person did:****

    ** **

    http://search-hadoop.com/m/HFWE32CYs6x/v=plain****

    ** **

    On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:****



    ****

    Hi,****

    ****

    I have a situation where I need to do urldecode on one particular column.
    Is there any hive built in function available?****

    ****

    Thank you,****

    Chalcy****

    ** **
  • Loren Siebert at Sep 15, 2011 at 6:52 pm
    That is cool! I didn’t know this feature existed at all.

    I’ve been using https://cwiki.apache.org/confluence/display/Hive/LanguageManual as the online manual for Hive, and I don’t see ‘reflect’ mentioned there. I see it here, though: http://archive.cloudera.com/cdh/3/hive/udf/reflect.html. This archive site also contains some good info on variable substitution in Hive scripts, which I didn’t know was available either.

    Thanks for the tip on ‘reflect’ !

    On Sep 15, 2011, at 11:36 AM, Carl Steinbach wrote:

    It's possible to call Java static methods directly using the built-in
    'reflect' UDF. The performance will not be as good as writing your
    own UDF, but you'll get results a lot faster.

    On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja wrote:
    Thank you! Loren. So far I have only using hive as is. Looks like I have to start extending J



    --Chalcy



    From: Loren Siebert
    Sent: Thursday, September 15, 2011 12:28 PM
    To: user@hive.apache.org
    Subject: Re: urldecode hive column



    You need to write a UDF, like this person did:



    http://search-hadoop.com/m/HFWE32CYs6x/v=plain



    On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:




    Hi,



    I have a situation where I need to do urldecode on one particular column. Is there any hive built in function available?



    Thank you,

    Chalcy


  • Chalcy Raja at Sep 16, 2011 at 2:23 pm
    I used reflect to urldecode like below and it works great.

    SELECT reflect("java.net.URLDecoder", "decode","to be decoded value) from footable limit 1;

    I have to start building UDF at somepoint. I am deferring that for now.

    Thank you Loren and Carl for the ideas and your time.

    Thank you,
    Chalcy

    From: Loren Siebert
    Sent: Thursday, September 15, 2011 2:52 PM
    To: user@hive.apache.org
    Subject: Re: urldecode hive column

    That is cool! I didn't know this feature existed at all.

    I've been using https://cwiki.apache.org/confluence/display/Hive/LanguageManual as the online manual for Hive, and I don't see 'reflect' mentioned there. I see it here, though: http://archive.cloudera.com/cdh/3/hive/udf/reflect.html. This archive site also contains some good info on variable substitution in Hive scripts, which I didn't know was available either.

    Thanks for the tip on 'reflect' !


    On Sep 15, 2011, at 11:36 AM, Carl Steinbach wrote:


    It's possible to call Java static methods directly using the built-in
    'reflect' UDF. The performance will not be as good as writing your
    own UDF, but you'll get results a lot faster.

    On Thu, Sep 15, 2011 at 9:38 AM, Chalcy Raja wrote:
    Thank you! Loren. So far I have only using hive as is. Looks like I have to start extending :)

    --Chalcy

    From: Loren Siebert
    Sent: Thursday, September 15, 2011 12:28 PM
    To: user@hive.apache.org
    Subject: Re: urldecode hive column

    You need to write a UDF, like this person did:

    http://search-hadoop.com/m/HFWE32CYs6x/v=plain

    On Sep 15, 2011, at 9:03 AM, Chalcy Raja wrote:

    Hi,

    I have a situation where I need to do urldecode on one particular column. Is there any hive built in function available?

    Thank you,
    Chalcy
  • Longmans163 at Sep 16, 2011 at 6:05 am
    It's easy to write a udf by yourself.
    such as:


    import org.apache.hadoop.hive.ql.exec.UDF;
    import org.apache.hadoop.io.Text;
    import java.net.URLDecoder;
    import java.io.UnsupportedEncodingException;


    public class urlDecoder extends UDF{
    public urlDecoder(){
    }
    private Text text = new Text();
    public Text evaluate(final Text column) throws UnsupportedEncodingException{
    text.clear();
    String columnString = column.toString();
    if(column != null){
    text = new Text(URLDecoder.decode(columnString, "UTF-8"));
    }
    return text;
    }
    }



    At 2011-09-16 00:03:59,"Chalcy Raja" wrote:


    Hi,



    I have a situation where I need to do urldecode on one particular column. Is there any hive built in function available?



    Thank you,

    Chalcy

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedSep 15, '11 at 4:04p
activeSep 16, '11 at 2:23p
posts7
users4
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase