[Hadoop-common-user] Invalid link http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar during ivy download whiling mumak build.

So set a proxy?

http://ant.apache.org/manual/proxy.html

Hi all !
I have added the following code to build.xml and tried to build : $ant
package.
I have also tried to remove removed the entire ivy2 (~/.ivy2/* ) directory
and rebuild but couldn't succeed.
<setproxy proxyhost="192.168.0.90" proxyport="8080"
proxyuser="ranam" proxypassword="passwd" nonproxyhosts="xyz.svn.com
"/>
I get the error UNRESOLVED DEPENDENCIES.
I have attached the log file.
Any idea ?

Thanks,
Arun K

Arun,

Looks like 0.21.0 pieces are still not present in the maven
repositories online. There was a fix for 0.21.1 but that didn't get
cut yet (perhaps abandoned).

If you're looking to get into development, I'd suggest either using
the stable 0.20 branch or using the trunk, depending on what you're
gonna be working on.

If you still want to persist with the 0.21, try to get it directly off
of the apache svn 0.21 branch of each component (common, mapreduce,
hdfs).

The artifact is there, so it's a proxy problem

export $ANT_OPTS = "-Dhttp.proxyHost=proxy -Dhttp.proxyPort=8080
-Dhttps.proxyHost=proxy -Dhttps.proxyPort=8080"

These don't set ant properties, they set JVM options, and do work for
Hadoop builds

Hello,
I have a setup where a bunch of clients store 'events' in an Hbase table . Also, periodically(once a day), I run a mapreduce job that goes over the table and computes some reports.

Now my issue is that the next time I don't want mapreduce job to process the 'events' that it has already processed previously. I know that I can mark processed event in the hbase table and the mapper can filter them them out during the next run. But what I would really like/want is that previously processed events don't even hit the mapper.

One solution I can think of is to backup the hbase table after running the job and then clear the table. But this has lot of problems..
1) Clients may have inserted events while the job was running.
2) I could disable and drop the table and then create it again...but then the clients would complain about this short window of unavailability.


What do people using Hbase (live) + mapreduce typically do. ?

Thanks!
Chinmay

I believe HBase has some kind of TTL (timeout-based expiry) for
records and it can clean them up on its own.

On Sat, Sep 10, 2011 at 1:54 AM, Dhodapkar, Chinmay
wrote:

Chinmay, how are you configuring your job? Have you checked using setScan
and selecting the keys you care to run MR over? See

http://ofps.oreilly.com/titles/9781449396107/mapreduce.html

As a shameless plug - For your reports, see if you want to leverage Crux:
https://github.com/sonalgoyal/crux

Best Regards,
Sonal
Crux: Reporting for HBase <https://github.com/sonalgoyal/crux>
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>

If you already have your ivy2 cache with all the artifacts
then you can manually download/copy the ivy.jar and place that in the ivy
folder and do an ant build with -Doffline=true.

-Giri