• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 411
  • Last Modified:

Updating a text file in java

Hi Experts...

I have a txt file delimited by '|' as follows:

name|id|age|address|rank
naruto|2|24
vegeta|3|25
gaara|1|22


In the above mentioned text file the data are not complete for each line.

Later i have the full data for say the third record
i.e
Name: Gaara
Id:1
age:22
address:sand
rank:100

Now how do i go and update the original data in the file as
name|id|age|address|rank
naruto|2|24
vegeta|3|25
gaara|1|22|sand|100


Please help...
0
gaugeta
Asked:
gaugeta
  • 12
  • 11
  • 3
  • +1
1 Solution
 
for_yanCommented:
You cannot update text file in place - the only way to od it is to read the file line by line , readadditional data (from second file?) and then write out the new file with the addutions in each line - when you re-wrote the whjole file you can delete the original file and rename the updated file to gthe oriinal name
0
 
for_yanCommented:
Don't foget to close both files before you start deleting the original file using
File.delete() and renaing the new file to old name
using reanmeTo() methgod of the File class
0
 
gaugetaAuthor Commented:
@for_yan:Thanks for the reply.

Could this be done if the above mentioned data were in a csv file.

Please help...
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
gaugetaAuthor Commented:
@for_yan and @gudii9:Basically why i wanted this to happen was that i had another question asked in the link:
Previous Question
I was thinking i would extract the initiallly available data into a text or a csv file and update the file after the entire data for every record is available.
Could you suggest a better way to do this.
0
 
CEHJCommented:
Build a Map for the new record and write it out with a csv writer to a new file
0
 
gaugetaAuthor Commented:
@CEHJ:Thanks for the reply.
How can a java map have data which are equivalent to 67 columns of the data base or is it even practical to do so.

I was thinking as suggested by you to extact the data in a text or a csv file and update the record or a line when the complete data is present.

I was thinking to do this on the database side but was apprehensive as i have more than 5 million records to work with.

Please help...
0
 
for_yanCommented:
I think to do it on database side would probably be a better solution
0
 
CEHJCommented:
How can a java map have data which are equivalent to 67 columns of the data base or is it even practical to do so.

That's tiny for a Map, which is the new record for update
0
 
gaugetaAuthor Commented:
@CEHJ and @for_yan:Thanks for the reply.

>>That's tiny for a Map, which is the new record for update
Sorry, I did not get you.

I need to decide as if i take either to do it on the database side or to do it externally via files, later i would not be able to turn back.

Cant this process be speeded up by threads.
Please help...
0
 
for_yanCommented:
What kind of database are you using?
If this is Oracle - it is designed for big tables and 5 million rows is not that dramatic.
When it comes to updating rows, as you saw,  database is definitely much better place  to deal with
0
 
for_yanCommented:
You can do it just row by row raeding rthe rows - calculation the remaning rcolumns and filling them in
Actually database and JDBC talke care of the memory issues for you, so
it is not that bad as you may think.
You can devise some way to select  and update grouped in certain chunks.
With database in such situation you are much more flexible than with files.
0
 
gaugetaAuthor Commented:
@for_yan:Thanks for the reply.
Could i create say 4 different temp tables which would contain all the records to be updated.

Could i use threads on these temp tables so that the records could get updated simultaneously and could complete fast.
0
 
for_yanCommented:
I don't see any reason why you can't have 4 different temp tables

I would first start doing it without threads - it is additional complication - look at your times real life situations, if this is some application which you need to run once to update or you'll be using it all the time see how it goes first in termas of time especially if it ios mostly one time opperation- if you need it then think about threads
0
 
CEHJCommented:
>>That's tiny for a Map, which is the new record for update
Sorry, I did not get you.

The Map would represent i row in the db. Are you reading and writing ONLY to and from the same row?

Multi-threading is not likely to help you
0
 
gaugetaAuthor Commented:
@for_yan and @CEHJ:Thanks for the reply.

This would be a one time operation or load that i would be performing once in a few months.

What I thought was that I'll create 4 temp tables and distribute the total load in these 4 tables and simultaneously kick off 4 threads which would run simultaneously and load the data.

If threading would not be used then there is no reason for me to divide the load in 4 different tables.

What is the issue in using threads in the scenario explained.

Please help...
0
 
for_yanCommented:
Can you explain what is the essence of calculation which you'll be doing to update the rest of the fields in the table ?
0
 
gaugetaAuthor Commented:
@for_yan: Basically I'll be taking about three fields which are the unique fields from each record and based on these three field values I'll be filling up the 52 columns which were empty before from 2 different tables.

Its basically plain mathematical calculation which is done here.
0
 
for_yanCommented:
so you take three vlaues and look up some values from two more tables based on these three and then use them to calculate values for 52 columns and fill them in ?

So why can't you do it not going to java at all ?
0
 
gaugetaAuthor Commented:
@for_yan:Basically the calculations done for a single value in those 52 columns were many where to get the final value i have to perform 5 to 6 calculations and then only i get the final resultant value and the same goes for the rest of the columns too.

And these are very specific mathematical calculations where i need java.lang.math and date and simpledateformat classes etc.

I could not have these many calculations in a stored proc.

What else did you have in mind and what is not suitable with java using threads.
0
 
for_yanCommented:
I'm pretty sure all that stuff in math and dates is possible to do within oracle itself
within SQL.


In general, if the question is about speed, my way is first to try in straightforward simplest way and then assess what kind of issues you are facing in the sense of timing

the thing is that let's assume threads will be successful and with a lot of additional work may end up say in saving time let's say even two, or assuming a lot, three times but most probably would not end up in orders of magnitudes, whereas moving to PL/SQL from Java in many cases produces order of magnitude saving in time.

And there is a good probablility that the issue would be really in the orders of magintudes or there would be no issue at all - it would all be really fast.
Probability that running it twice as fast will solve your practical problem is much smaller.

So instead of thinking is it worth using threads or not, I'd rather sit down and do it without any threads and see how it performs and then see what is the issue I'm facing and if I face any serious issue at all.
0
 
gaugetaAuthor Commented:
@for_yan:Thanks for the reply.
I have had similar loads which ran for about 4 to 5 hours.
The database is sybase in my case.
I will try this thing first without threads.

But if it executes two or even three times fast with threads isn't it worth as i have had very similar loads done previously which takes more than 4 hours.
What is your opinion on this.
0
 
for_yanCommented:
yes, if it turns out a matter of say two times longer and you have really strict requirement of 4 hours, then playing with threads or say just with splitting the table   (as i understand you process each row independently of  others) may indeed help.
And your work to do it in the straightforward way would not be in vain anyway - I'm sure you'll be able to re-use big parts of the code
0
 
gaugetaAuthor Commented:
@for_yan:So in this case of huge data if i have a dual-core processor, what is the maximum threads that i can run simultaneously.
0
 
for_yanCommented:
I don't think there is any strict limit. I think a few of them may lead to some time saving; further increase will not end up in time saving. I don't think anyone can tell you the exact number. It is trial and error which can give you some number.
0
 
for_yanCommented:
>This would be a one time operation or load that i would be performing once in a few months.

Given that statement it is hard to understand why 4 hours is OK and, say,  7 hours is not?

I think your time spent on figuring out how to reduce from 7 hours to 4 hours will worth more.

Taht's wahy I'm saying difference in the order of magnitude in such situation can make soem didfference.  To get from 7 hours to 4 hours once in several months at the expense of say a day of debugging does not make sense to me. I would not use any threads.
0
 
gaugetaAuthor Commented:
@for_yan:Thanks for the reply.
I'll try this without threads and get back if i face any issues.
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 12
  • 11
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now