FAQ
Hello all,

Is this possible? If I add a file to distributed cache via ADD FILE,
could I access it in my Java-based user defined function using the
Hadoop APIs or is this not a good idea?

Thanks,
Ryan

Search Discussions

  • Eric Arenas at Nov 2, 2009 at 5:27 pm
    Hi Ryan,

    You can access your file as it were in the local working directory

    For instance you need to access it as

    Stringmyfile = "./mydatafile.txt"; // local file

    Then you open it with a normal FileReader as follows

    try
    {
    FileReader fr = new FileReader(blacklistUAfile); //reading the file and loading that into memory
    BufferedReader br = new BufferedReader(fr);

    }
    catch(Exception e)
    {
    System.err.println("invalid data file ; error - " + e.getMessage());
    return false;
    }


    I have not tried using the hadoop APIs to access this file, but you can code anything you want in Java to work with your file.

    Hope this helps,

    Eric Arenas



    ----- Original Message ----
    From: Ryan LeCompte <lecompte@gmail.com>
    To: hive-user@hadoop.apache.org
    Sent: Fri, October 30, 2009 2:35:26 PM
    Subject: Accessing distributed cache from UDF


    Hello all,

    Is this possible? If I add a file to distributed cache via ADD FILE, could I access it in my Java-based user defined function using the Hadoop APIs or is this not a good idea?

    Thanks,
    Ryan

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categorieshive, hadoop
postedOct 30, '09 at 9:36p
activeNov 2, '09 at 5:27p
posts2
users2
websitehive.apache.org

2 users in discussion

Eric Arenas: 1 post Ryan LeCompte: 1 post

People

Translate

site design / logo © 2022 Grokbase