Two programs accessing the same CSV file

How about using a single program accessing the data...(just pick the speedier if the data come too fast) and store them in a database ...then everyone will what they want and the data will only get written once.

As long as they are opening the file in read mode (not locking it for update) I wouldn't think that would interfere with your code.

Could there be something else in your code that might be failing with the change to a shorter cycle time (1 second vs 5 second)?

---

»bp

As John Tsioumpris says, store it in a database rather than a CSV, then multiple programs can read and write to it. SQL express will do and that's free.

We have done this type of data file creation many times at 1 second intervals and never had a problem. So my guess is my client's program is interfering with something.  It is difficult to tell a client that they are wrong so I need something I can say to them to force them to re-evaluate their code or stop using a second program to access our file.

When your program stops, is there a specific error message?  Does it sound like a problem accessing the shared file?

---

»bp

The error message appears to be coming from Windows. It just says that our program has stopped working.

In my experience, using correct locks/sharing is confusing for .NET programmers who haven't done it before. More often than not, the developer will be looking to AVOID a lock and instead actually CREATES a lock - it's just a bit confusing if you haven't done it before.

So I wouldn't be surprised if they were trying to lock the file while you were writing to it and you hit a conflict.

On a side note, I fully agree with the previous comments about using a database for this. Flat files are great when the I/O is sequential and controlled like "Write, Write, Write, Write, Stop, Read, Stop, Write, Write, Write, etc..."

But they are terrible for multi-process access. Even if you got the locking corrected (if that's what it is), my guess is that you have no protection against someone coming in and writing to the same file at the same time (e.g. multiple processes get spawned), which would corrupt the data quickly. Plus, there's no usability in terms of accessing the data (reading, analyzing, etc).

The whole point of databases is to give you a fast, reliable way of storing data like this, allowing multiple processes to read and write as necessary without any fear of corrupting the data. On top of all that, you can enforce access control, so your database credentials can write to the table, but your 3rd party might have credentials that only allows them to read. There are tons of benefits to using a DB for this, even if you've successfully used CSV files in the past for this.

>>I'm wondering if my client's program could be causing the crash of our program by attempting to access the same CSV file that we are appending to.

Without knowing the client program code the answer is: it could be.
Which isn't much help to you.

The error message that your app has stopped working however is worrying.  Not quite what I would have expected if it was caused by the clients app.  However it depends on just how you handle errors when writing to the csv file.

I would start by asking the client to stop there program for a couple of days. If the error doesn't show then you will know that there program has something to do with the error. If not, your app has some kind of issues.

At the same time, I would add error handling to your application and error logging so that you would not be completely blind trying to find the error.

@charlieb01,

Are you all set with this now, or do you need more help?  If all set, could you please close it out now.  If you need help with the question close process take a look at:

• How do I close my question?

---

»bp