Grokbase Groups Hive user August 2011
FAQ
Hi,
I have UDF jar that I need to install so that the Hive web clients can execute that function. Can someone point me to the doc that describes it?

-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.

Search Discussions

  • Chinna at Aug 3, 2011 at 4:23 am
    Hi,



    In the below link this section " creating custom UDFs" will help u ...



    https://cwiki.apache.org/confluence/display/Hive/HivePlugins



    - Chinna Rao Lalam

    _____

    From: Ayon Sinha
    Sent: Wednesday, August 03, 2011 9:09 AM
    To: Hive Mailinglist
    Subject: How to install UDF jar into Hive server



    Hi,

    I have UDF jar that I need to install so that the Hive web clients can
    execute that function. Can someone point me to the doc that describes it?



    -Ayon
    See My Photos <http://www.flickr.com/photos/ayonsinha/> on Flickr
    Also check out my <http://dailyadvisor.blogspot.com> Blog for answers to
    commonly asked questions.
  • Ayon Sinha at Aug 3, 2011 at 5:21 am
    Thanks Chinna. Unfortunately this doesn't say how to install it such that the HiveServer is able to use this jar. Examples in this wiki link does it all in the Hive CLI.

    -Ayon
    See My Photos on Flickr
    Also check out my Blog for answers to commonly asked questions.



    ________________________________
    From: Chinna <chinnarao@huawei.com>
    To: user@hive.apache.org; 'Ayon Sinha' <ayonsinha@yahoo.com>
    Sent: Tuesday, August 2, 2011 9:22 PM
    Subject: RE: How to install UDF jar into Hive server



    Hi,

    In the below link this section "creating custom
    UDFs"  will help u ...

    https://cwiki.apache.org/confluence/display/Hive/HivePlugins

    - Chinna Rao Lalam

    ________________________________

    From:Ayon Sinha

    Sent: Wednesday, August 03, 2011
    9:09 AM
    To: Hive Mailinglist
    Subject: How to install UDF jar
    into Hive server

    Hi,
    I have
    UDF jar that I need to install so that the Hive web clients can execute that
    function. Can someone point me to the doc that describes it?

    -Ayon
    See My Photos
    on Flickr
    Also check out my
    Blog for answers to commonly asked questions.
  • Chinna at Aug 3, 2011 at 6:41 am
    Hi,



    For example u have an UDF jar like lower.jar (Here u have the
    implementation for the lower function) in /home/myjars location.



    if it is Hive CLI or Hive Server or web client... what mode u r using there
    u need to execute the following queries



    1. You need to add the jar to Hive class path for this u can execute this
    query



    add jar '/home/myjars/lower.jar';



    2. U need to create the function using this query



    create temporary fucntion my_lower as 'com.example.hive.udf.Lower';



    then u r function is ready now u can use the function like



    select my_lower(title) from titles



    -Chinna Rao Lalam

    _____

    From: Ayon Sinha
    Sent: Wednesday, August 03, 2011 10:50 AM
    To: user@hive.apache.org
    Subject: Re: How to install UDF jar into Hive server



    Thanks Chinna. Unfortunately this doesn't say how to install it such that
    the HiveServer is able to use this jar. Examples in this wiki link does it
    all in the Hive CLI.



    -Ayon
    See My Photos <http://www.flickr.com/photos/ayonsinha/> on Flickr
    Also check out my <http://dailyadvisor.blogspot.com> Blog for answers to
    commonly asked questions.

    _____

    From: Chinna <chinnarao@huawei.com>
    To: user@hive.apache.org; 'Ayon Sinha' <ayonsinha@yahoo.com>
    Sent: Tuesday, August 2, 2011 9:22 PM
    Subject: RE: How to install UDF jar into Hive server

    Hi,



    In the below link this section " creating custom UDFs" will help u ...



    https://cwiki.apache.org/confluence/display/Hive/HivePlugins



    - Chinna Rao Lalam

    _____

    From: Ayon Sinha
    Sent: Wednesday, August 03, 2011 9:09 AM
    To: Hive Mailinglist
    Subject: How to install UDF jar into Hive server



    Hi,

    I have UDF jar that I need to install so that the Hive web clients can
    execute that function. Can someone point me to the doc that describes it?



    -Ayon
    See My Photos <http://www.flickr.com/photos/ayonsinha/> on Flickr
    Also check out my Blog <http://dailyadvisor.blogspot.com> for answers to
    commonly asked questions.
  • Ayon Sinha at Aug 3, 2011 at 6:55 am
    So if I have a web-client calling the Hive server, will step 1 & 2 need to be executed for every call as a sequence (that would be inconvenient). I really want this UDF to be available to every user of our web interface. They shouldn't need to know where the jar is.

    -Ayon
    See My Photos on Flickr
    Also check out my Blog for answers to commonly asked questions.



    ________________________________
    From: Chinna <chinnarao@huawei.com>
    To: user@hive.apache.org; 'Ayon Sinha' <ayonsinha@yahoo.com>
    Sent: Tuesday, August 2, 2011 11:38 PM
    Subject: RE: How to install UDF jar into Hive server



    Hi,

    For example u have an UDF jar
    like   lower.jar (Here u have the implementation for the lower
    function) in /home/myjars location.

    if it is Hive CLI or Hive Server or web
    client... what mode u r using there u need to execute the following queries

    1. You need to add the jar  to Hive
    class path for this u can execute this query

    add jar '/home/myjars/lower.jar';

    2. U need to create the function using
    this query

    create temporary fucntion
    my_lower as 'com.example.hive.udf.Lower';

    then u r function is ready now u can use the
    function like

    select my_lower(title) from titles

    -Chinna Rao Lalam

    ________________________________

    From:Ayon Sinha

    Sent: Wednesday, August 03, 2011
    10:50 AM
    To: user@hive.apache.org
    Subject: Re: How to install UDF
    jar into Hive server

    Thanks
    Chinna. Unfortunately this doesn't say how to install it such that the
    HiveServer is able to use this jar. Examples in this wiki link does it all in
    the Hive CLI.

    -Ayon
    See My Photos
    on Flickr
    Also check out my
    Blog for answers to commonly asked questions.

    ________________________________

    From:Chinna
    <chinnarao@huawei.com>
    To: user@hive.apache.org; 'Ayon
    Sinha' <ayonsinha@yahoo.com>
    Sent: Tuesday, August 2, 2011 9:22
    PM
    Subject: RE: How to install UDF
    jar into Hive server
    Hi,

    In the below link
    this section "creating custom UDFs"  will help u ...


    https://cwiki.apache.org/confluence/display/Hive/HivePlugins

    - Chinna Rao
    Lalam

    ________________________________

    From:Ayon Sinha

    Sent: Wednesday, August 03, 2011
    9:09 AM
    To: Hive Mailinglist
    Subject: How to install UDF jar
    into Hive server

    Hi,
    I have
    UDF jar that I need to install so that the Hive web clients can execute that
    function. Can someone point me to the doc that describes it?

    -Ayon
    See My Photos
    on Flickr
    Also check out my Blog
    for answers to commonly asked questions.
  • Edward Capriolo at Aug 3, 2011 at 2:31 pm

    On Wed, Aug 3, 2011 at 2:55 AM, Ayon Sinha wrote:

    So if I have a web-client calling the Hive server, will step 1 & 2 need to
    be executed for every call as a sequence (that would be inconvenient). I
    really want this UDF to be available to every user of our web interface.
    They shouldn't need to know where the jar is.

    -Ayon
    See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
    Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>

    ------------------------------
    *From:* Chinna <chinnarao@huawei.com>
    *To:* user@hive.apache.org; 'Ayon Sinha' <ayonsinha@yahoo.com>
    *Sent:* Tuesday, August 2, 2011 11:38 PM
    *Subject:* RE: How to install UDF jar into Hive server

    Hi,

    For example u have an UDF jar like lower.jar (Here u have the
    implementation for the lower function) in /home/myjars location.

    if it is Hive CLI or Hive Server or web client... what mode u r using there
    u need to execute the following queries

    1. You need to add the jar to Hive class path for this u can execute this
    query

    add jar '/home/myjars/lower.jar';

    2. U need to create the function using this query

    create temporary fucntion my_lower as 'com.example.hive.udf.Lower';

    then u r function is ready now u can use the function like

    select my_lower(title) from titles

    -Chinna Rao Lalam
    ------------------------------
    *From:* Ayon Sinha
    *Sent:* Wednesday, August 03, 2011 10:50 AM
    *To:* user@hive.apache.org
    *Subject:* Re: How to install UDF jar into Hive server

    Thanks Chinna. Unfortunately this doesn't say how to install it such that
    the HiveServer is able to use this jar. Examples in this wiki link does it
    all in the Hive CLI.

    -Ayon
    See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
    Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>
    ------------------------------
    *From:* Chinna <chinnarao@huawei.com>
    *To:* user@hive.apache.org; 'Ayon Sinha' <ayonsinha@yahoo.com>
    *Sent:* Tuesday, August 2, 2011 9:22 PM
    *Subject:* RE: How to install UDF jar into Hive server
    Hi,

    In the below link this section "* **creating custom UDFs*" will help u
    ...

    https://cwiki.apache.org/confluence/display/Hive/HivePlugins

    - Chinna Rao Lalam
    ------------------------------
    *From:* Ayon Sinha
    *Sent:* Wednesday, August 03, 2011 9:09 AM
    *To:* Hive Mailinglist
    *Subject:* How to install UDF jar into Hive server

    Hi,
    I have UDF jar that I need to install so that the Hive web clients can
    execute that function. Can someone point me to the doc that describes it?

    -Ayon
    See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
    Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>


    Users have no way to send the hive-service jars but you can work around it.

    1) Option 1: hive now has a hive.rc file. You can put this on the hive
    service server.
    2) Option 2: There was a ticket open to add jars that are present on hdfs. I
    believe it was completed.
    3) Option 3: (I think this works)
    (from client)
    hadoo dfs -copyFromLocal myjar.jar /tmp;
    (inside hive-service session)
    dfs -copyToLocal /tmp/myjar.jar /tmp ;
    add jar /tmp/my.jar;

    Edward
  • Ayon Sinha at Aug 3, 2011 at 4:36 pm
    Hi Edward,
    I think option 1 with hive.rc is the path we want to take.
    I am really not looking for web-client users ton install UDF jars. Our use-case is:
    We as the Infrastructure team have written a UDF that we want our internal web-client customers to use without them having to install any jars. They wouldn't know the difference in usage between
    to_date(string) vs our_custom_fn(string).

    Let me know if you have more detailed steps for option 1.

    -Ayon
    See My Photos on Flickr
    Also check out my Blog for answers to commonly asked questions.



    ________________________________
    From: Edward Capriolo <edlinuxguru@gmail.com>
    To: user@hive.apache.org; Ayon Sinha <ayonsinha@yahoo.com>
    Cc: "chinnarao@huawei.com" <chinnarao@huawei.com>
    Sent: Wednesday, August 3, 2011 7:31 AM
    Subject: Re: How to install UDF jar into Hive server





    On Wed, Aug 3, 2011 at 2:55 AM, Ayon Sinha wrote:

    So if I have a web-client calling the Hive server, will step 1 & 2 need to be executed for every call as a sequence (that would be inconvenient). I really want this UDF to be available to every user of our web interface. They shouldn't need to know where the jar is.
    -Ayon
    See My Photos on Flickr
    Also check out my Blog for answers to commonly asked questions.



    ________________________________
    From: Chinna <chinnarao@huawei.com>
    To: user@hive.apache.org; 'Ayon Sinha' <ayonsinha@yahoo.com>
    Sent: Tuesday, August 2, 2011 11:38 PM
    Subject: RE: How to install UDF jar into Hive server


    Hi,

    For example u have an UDF jar
    like   lower.jar (Here u have the implementation for the lower
    function) in /home/myjars location.
    if it is Hive CLI or Hive Server or web
    client... what mode u r using there u need to execute the following queries
    1. You need to add the jar  to Hive
    class path for this u can execute this query
    add jar '/home/myjars/lower.jar';

    2. U need to create the function using
    this query
    create temporary fucntion
    my_lower as 'com.example.hive.udf.Lower';
    then u r function is ready now u can use the
    function like
    select my_lower(title) from titles

    -Chinna Rao Lalam

    ________________________________

    From:Ayon Sinha
    Sent: Wednesday, August 03, 2011 10:50 AM
    To: user@hive.apache.org
    Subject: Re: How to install UDF
    jar into Hive server
    Thanks
    Chinna. Unfortunately this doesn't say how to install it such that the
    HiveServer is able to use this jar. Examples in this wiki link does it all in
    the Hive CLI.
    -Ayon
    See My Photos on Flickr
    Also check out my
    Blog for answers to commonly asked questions.
    ________________________________

    From:Chinna
    <chinnarao@huawei.com>
    To: user@hive.apache.org; 'Ayon
    Sinha' <ayonsinha@yahoo.com>
    Sent: Tuesday, August 2, 2011 9:22 PM
    Subject: RE: How to install UDF
    jar into Hive server
    Hi,

    In the below link
    this section "creating custom UDFs"  will help u ...
    https://cwiki.apache.org/confluence/display/Hive/HivePlugins

    - Chinna Rao Lalam
    ________________________________

    From:Ayon Sinha
    Sent: Wednesday, August 03, 2011 9:09 AM
    To: Hive Mailinglist
    Subject: How to install UDF jar
    into Hive server
    Hi,
    I have
    UDF jar that I need to install so that the Hive web clients can execute that
    function. Can someone point me to the doc that describes it?
    -Ayon
    See My Photos on Flickr
    Also check out my Blog
    for answers to commonly asked questions.
    Users have no way to send the hive-service jars but you can work around it.

    1) Option 1: hive now has a hive.rc file. You can put this on the hive service server.
    2) Option 2: There was a ticket open to add jars that are present on hdfs. I believe it was completed.
    3) Option 3: (I think this works)
    (from client)
    hadoo dfs -copyFromLocal myjar.jar /tmp;
    (inside hive-service session)
    dfs -copyToLocal /tmp/myjar.jar /tmp ;
    add jar /tmp/my.jar;

    Edward

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedAug 3, '11 at 3:40a
activeAug 3, '11 at 4:36p
posts7
users3
websitehive.apache.org

People

Translate

site design / logo © 2021 Grokbase