Access and a Large Text File: over 8GB

Hello All,

I was hoping to get suggestions on the best way to handle this situation.   We have a large text file with many fields and many records.  The size of the file is over 8GB.   We need to be able to look at the data to ensure fields contain correct data, and to be able to massage some of the data.   We will be using MS Access 2010 as the front end.

I know there is a 2GB limit for an Access file, so I was thinking that we could import all the data to a SQL Server table.  We could then use Access to view and massage the data using vba code.    

I want to install SQL Server on my desktop and create a database with that.  Then I could link to the table that houses all the imported information.

My question is, can you install SQL Server and create a database all on your desktop and not on a server?
sgtoverlordAsked:
Who is Participating?
 
Jim Dettman (Microsoft MVP/ EE MVE)Connect With a Mentor President / OwnerCommented:
0
 
Jim Dettman (Microsoft MVP/ EE MVE)President / OwnerCommented:
<<My question is, can you install SQL Server and create a database all on your desktop and not on a server? >>

 Yes, the Express edition is free and you'll have no problems doing that.

 What you've outlined is doable.

Jim.
0
 
sgtoverlordAuthor Commented:
Awesome.  

Now I am fairly astute at Access and VBA code, but I haven't created a SQL Server Database before.   Is this something that I should be able to fumble my way thru, or is it completely different?  If so, do you know of a good website or web tutorial that would assist me with this endeavor (and gives correct information)?

One other question:  Will this method be more efficient (faster) than using access to link to the text file as a table itself?   I am hoping that this is the case.  Our process currently takes hours & hours.   I was hoping that using Access with a local SQL Server db would increase the efficiency.
0
Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

 
jerryb30Commented:
I believe the express edition has a 4gb limit.
0
 
sgtoverlordAuthor Commented:
poo
0
 
sgtoverlordAuthor Commented:
Well, I do see that there is a 4GB limit with SQL Server 2005 Express Edition.   And A 10GB limit with SQL Server 2008 R2.  

My company has licenses for SQL 2005 Business Intelligence Development Studio.  Does anyone know the size limit for this?

But is this file size what I am supposed to be worried about.   Let me explain:   Currently, we are using Access 2003 to link to a text file that is 8GB+ in size.   We are able to do some crunching with this arrangement, it just takes forever (7 hours+).    

I guess my main goal is to start using Sql Server to increase the speed at which this stuff is processed.  

Any thoughts on this???
0
 
Jim Dettman (Microsoft MVP/ EE MVE)President / OwnerCommented:
<<But is this file size what I am supposed to be worried about.   Let me explain:   Currently, we are using Access 2003 to link to a text file that is 8GB+ in size.   >>

 Just because it's 8GB in the file doesn't mean it will be 8GB in the database.   Some may be duplicated for example.  

 When brought into a DB and normalized, it's possible that size would go down.

<<We are able to do some crunching with this arrangement, it just takes forever (7 hours+).  >>

 Depends on what your doing there may be ways to speed that up as it stands.

<<I guess my main goal is to start using Sql Server to increase the speed at which this stuff is processed.  

Any thoughts on this??? >>

  Too hard to say without a lot more detail (sample of the file, what your looking to get out of it, etc).

Jim.
0
 
sgtoverlordAuthor Commented:
Thank you for the information.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.