FAQ
DBOutputFormat will very likely put significantly more load on your MySQL
server vs. LOAD DATA INFILE. DBOutputFormat will trounce your MySQL server
with at least one connection per reducer. This may be OK if you have a
small number of reducers and a small amount of output data. LOAD DATA
INFILE is a much safer approach, in that it won't potentially axe your MySQL
database.

Alex
On Tue, May 19, 2009 at 2:58 PM, dealmaker wrote:


Does DBOutputFormat have similar performance as Load Data Infile?
Thanks.


TimRobertson100 wrote:
So you are using a java program to execute a "load data infile"
command on mysql through JDBC?
If so I *think* you would have to copy it onto the mysql machine from
HDFS first, or the machine running the command and then try a 'load
data local infile'.

Or pehaps use the
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/lib/db/DBOutputFormat.html
Cheers

Tim
On Tue, May 19, 2009 at 9:54 PM, dealmaker wrote:

Hi,
I am want to load data in mysql using a hadoop file similar to
following:
LOAD DATA INFILE 'test.txt' INTO TABLE test
FIELDS TERMINATED BY ',' LINES STARTING BY 'xxx';

But how do I load the hdfs file into the mysql comand above? Do I start
the
file name with hdfd://test.txt? I am using java.
Thanks.

--
View this message in context:
http://www.nabble.com/Mysql-Load-Data-Infile-with-Hadoop--tp23623256p23623256.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
--
View this message in context:
http://www.nabble.com/Mysql-Load-Data-Infile-with-Hadoop--tp23623256p23625140.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Search Discussions

Discussion Posts

Previous

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 6 of 6 | next ›
Discussion Overview
groupcommon-user @
categorieshadoop
postedMay 19, '09 at 9:44p
activeMay 20, '09 at 12:20a
posts6
users5
websitehadoop.apache.org...
irc#hadoop

People

Translate

site design / logo © 2022 Grokbase