Grokbase Groups Pig user July 2010
FAQ
Hi all,

I was wondering if it would be possible to process images on a
low level using PIG. I want to be able to write a pig script
that can differentiate between two images.

Search Discussions

  • Alan Gates at Jul 26, 2010 at 8:28 pm
    Pig itself does not contain image processing primitives. But if you
    write your image processing in a UDF, then Pig can be a great
    framework for dealing with the parallelism, running it on Hadoop, etc.

    Alan.
    On Jul 26, 2010, at 11:56 AM, Ifeanyichukwu Osuji wrote:



    Hi all,

    I was wondering if it would be possible to process images
    on a
    low level using PIG. I want to be able to write a pig script
    that can differentiate between two images.
  • Ashutosh Chauhan at Jul 26, 2010 at 8:40 pm
    To add in bit more details: You basically load images as bytearray
    type. Pig wont interpret it. Then you can write a UDF which takes
    these bytearrays as input. You can choose to work directly on
    bytearray or create image objects out of them, do your processing and
    return bytearray back to Pig. You can further do other sql-y
    operations on your images (like filter, group etc) using Pig
    primitives which will let you leverage Hadoop's strength.
    So, in short answer to your question is Yes.

    Ashutosh
    On Mon, Jul 26, 2010 at 13:27, Alan Gates wrote:
    Pig itself does not contain image processing primitives.  But if you write
    your image processing in a UDF, then Pig can be a great framework for
    dealing with the parallelism, running it on Hadoop, etc.

    Alan.
    On Jul 26, 2010, at 11:56 AM, Ifeanyichukwu Osuji wrote:



    Hi all,

    I was wondering if it would be possible to process images on a
    low level using PIG. I want to be able to write a pig script
    that can differentiate between two images.
  • Mridul Muralidharan at Jul 27, 2010 at 2:33 am
    Hi,

    We have a few projects which do this on hadoop, but I dont see any
    reason why it cant have been done in pig.
    As Alan and Ashutosh mentioned, the image itself will be just bytearray
    (and so you need your own loader, or in our case use a sequence file
    loader) : but you can extract and populate meta-data about the image
    through udf primitives which can then be used in the pig workflow to
    control how it is processed in a scaleout fashion on top of hadoop.


    Regards,
    Mridul
    On Tuesday 27 July 2010 12:26 AM, Ifeanyichukwu Osuji wrote:


    Hi all,

    I was wondering if it would be possible to process images on a
    low level using PIG. I want to be able to write a pig script
    that can differentiate between two images.
  • Russell Jurney at Jul 27, 2010 at 2:44 am
    There was a slide desk sitting around google that had a 'facial recognition'
    UDF a while back.

    On Mon, Jul 26, 2010 at 7:32 PM, Mridul Muralidharan
    wrote:
    Hi,

    We have a few projects which do this on hadoop, but I dont see any reason
    why it cant have been done in pig.
    As Alan and Ashutosh mentioned, the image itself will be just bytearray
    (and so you need your own loader, or in our case use a sequence file loader)
    : but you can extract and populate meta-data about the image through udf
    primitives which can then be used in the pig workflow to control how it is
    processed in a scaleout fashion on top of hadoop.


    Regards,
    Mridul

    On Tuesday 27 July 2010 12:26 AM, Ifeanyichukwu Osuji wrote:



    Hi all,

    I was wondering if it would be possible to process images on a
    low level using PIG. I want to be able to write a pig script
    that can differentiate between two images.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupuser @
categoriespig, hadoop
postedJul 26, '10 at 8:20p
activeJul 27, '10 at 2:44a
posts5
users5
websitepig.apache.org

People

Translate

site design / logo © 2021 Grokbase