Wednesday, March 28, 2012

Poor performance import text file

Hi,

I'm very new at SSIS. I've got a package to import a 1,000,000 row tab delimited text file into my SQL2005 database. Using SSIS with a Text File source and OLE DB PROVIDER target it crawls. I do have it skipping error rows but that is the only logic I am using. No indexes on the table either. This is pretty basic and it takes SSIS 1.5 hours, DTS blows this away... Assuming this is not something outside of SSIS(like the network, or a user locking issue), what I am I doing wrong?

BTW -- I'm searching for some batch commit sizes, like in DTS, but I cannot find them.

Any ideas.

From the information that you've given, it's hard to know what the problem is, but here are a few things you can check.

Try to isolate the problem. Change the destination to a rowcount or trash destination from SQLIS.com, see if it improves the throughput.

Is the flatfile on the same machine as the executing package?

Are you minimally logged on the destination?

Is the destination DB on the same machine?

Have you tried the SQL Server Destination?

How many errors are you getting from the flat file?

Rows per batch is on the OLEDB Destination Adapter. Did you try changing that?

K

No comments:

Post a Comment