I already posted this as "COPY FROM encoding error", but I
have been
doing some more tests since then.

I'm trying to export data from MS SQL Server to PostgreSQL.
The tables are quite big (>20M rows), so a CSV export and a "COPY
FROM3 import seems to be the only reasonable solution.
Or go via MS-Access/Perl and ODBC/DBI perhaps?
Yes, I think it would work. The problem is that the DB is too
big for this king of export. Using DTS from MSSQL to export
directly to PostgreSQL using psqlODBC Unicode Driver, I
exported ~1000 rows per second in a 2-columns table with ~20M
rows. That means several days just for this table, and I have
bigger ones !
Interesting. What did you set the "Inser batch size" to? (I think that's
available for all transformatino tasks). And did you remember to check
the box for "use transactions"?

While it's never as fast as a COPY, it should be possible to make it
faster than that, Ithink.


Another option is to just BCP the file out, and then COPY it into
postgresql. No nice GUI, but you can tune almost everything with BCP.

//Magnus

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 12 of 32 | next ›
Discussion Overview
grouppgsql-general @
categoriespostgresql
postedNov 21, '06 at 9:50p
activeNov 24, '06 at 8:08a
posts32
users11
websitepostgresql.org
irc#postgresql

People

Translate

site design / logo © 2022 Grokbase