handling/parsing LARGE text files in c#
Hi,
What is the best way to handle/parse large text files in c# (i.e. 500+ MB). Here's what the process would look like; there are large text files comming in, and going out, 24 hours a day 7 days a week. These files (i.e. EDI files, format is X12) have to be parsed properly and passed on to next process. File sizes vary but they are large. So, I was thinking, which way should I go...
I thought using unsafe technique in c#, also thought about using asyncronious StremReader/StreamWriter, thought about using both of these, and I still cannot deside. Any advise is highly appreciated.
Thanks!
What is the best way to handle/parse large text files in c# (i.e. 500+ MB). Here's what the process would look like; there are large text files comming in, and going out, 24 hours a day 7 days a week. These files (i.e. EDI files, format is X12) have to be parsed properly and passed on to next process. File sizes vary but they are large. So, I was thinking, which way should I go...
I thought using unsafe technique in c#, also thought about using asyncronious StremReader/StreamWriter, thought about using both of these, and I still cannot deside. Any advise is highly appreciated.
Thanks!
ASKER
article on this website says that changing strings using unsafe code is a bad aidea
http://msdn.microsoft.com/msdnmag/issues/04/04/NETMatters/
[article]
In the .NET Framework, strings are immutable, meaning that once created, they cannot be changed (some counter this assertion stating that strings can be changed using unsafe code and direct memory access, but doing so is a very bad idea). As such, instead of a String, the first parameter in the P/Invoke declaration is a mutable StringBuilder, marshaled as an unmanaged LPWSTR. You also need to ensure that StringBuilder is large enough to hold the path of the folder, which on a Win32 system is at most 260 characters. The resulting C# method looks like the following:
http://msdn.microsoft.com/msdnmag/issues/04/04/NETMatters/
[article]
In the .NET Framework, strings are immutable, meaning that once created, they cannot be changed (some counter this assertion stating that strings can be changed using unsafe code and direct memory access, but doing so is a very bad idea). As such, instead of a String, the first parameter in the P/Invoke declaration is a mutable StringBuilder, marshaled as an unmanaged LPWSTR. You also need to ensure that StringBuilder is large enough to hold the path of the folder, which on a Win32 system is at most 260 characters. The resulting C# method looks like the following:
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
thanks!
ASKER