Solved

Updating a text file in java

Posted on 2012-03-14
27
247 Views
Last Modified: 2012-06-27
Hi Experts...

I have a txt file delimited by '|' as follows:

name|id|age|address|rank
naruto|2|24
vegeta|3|25
gaara|1|22


In the above mentioned text file the data are not complete for each line.

Later i have the full data for say the third record
i.e
Name: Gaara
Id:1
age:22
address:sand
rank:100

Now how do i go and update the original data in the file as
name|id|age|address|rank
naruto|2|24
vegeta|3|25
gaara|1|22|sand|100


Please help...
0
Comment
Question by:gaugeta
  • 12
  • 11
  • 3
  • +1
27 Comments
 
LVL 47

Expert Comment

by:for_yan
ID: 37720634
You cannot update text file in place - the only way to od it is to read the file line by line , readadditional data (from second file?) and then write out the new file with the addutions in each line - when you re-wrote the whjole file you can delete the original file and rename the updated file to gthe oriinal name
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37720660
Don't foget to close both files before you start deleting the original file using
File.delete() and renaing the new file to old name
using reanmeTo() methgod of the File class
0
 

Author Comment

by:gaugeta
ID: 37720666
@for_yan:Thanks for the reply.

Could this be done if the above mentioned data were in a csv file.

Please help...
0
 
LVL 7

Expert Comment

by:gudii9
ID: 37720682
0
 

Author Comment

by:gaugeta
ID: 37720728
@for_yan and @gudii9:Basically why i wanted this to happen was that i had another question asked in the link:
Previous Question
I was thinking i would extract the initiallly available data into a text or a csv file and update the file after the entire data for every record is available.
Could you suggest a better way to do this.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 37720748
Build a Map for the new record and write it out with a csv writer to a new file
0
 

Author Comment

by:gaugeta
ID: 37720780
@CEHJ:Thanks for the reply.
How can a java map have data which are equivalent to 67 columns of the data base or is it even practical to do so.

I was thinking as suggested by you to extact the data in a text or a csv file and update the record or a line when the complete data is present.

I was thinking to do this on the database side but was apprehensive as i have more than 5 million records to work with.

Please help...
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37720789
I think to do it on database side would probably be a better solution
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 37720868
How can a java map have data which are equivalent to 67 columns of the data base or is it even practical to do so.

That's tiny for a Map, which is the new record for update
0
 

Author Comment

by:gaugeta
ID: 37720911
@CEHJ and @for_yan:Thanks for the reply.

>>That's tiny for a Map, which is the new record for update
Sorry, I did not get you.

I need to decide as if i take either to do it on the database side or to do it externally via files, later i would not be able to turn back.

Cant this process be speeded up by threads.
Please help...
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37720955
What kind of database are you using?
If this is Oracle - it is designed for big tables and 5 million rows is not that dramatic.
When it comes to updating rows, as you saw,  database is definitely much better place  to deal with
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37720978
You can do it just row by row raeding rthe rows - calculation the remaning rcolumns and filling them in
Actually database and JDBC talke care of the memory issues for you, so
it is not that bad as you may think.
You can devise some way to select  and update grouped in certain chunks.
With database in such situation you are much more flexible than with files.
0
 

Author Comment

by:gaugeta
ID: 37721012
@for_yan:Thanks for the reply.
Could i create say 4 different temp tables which would contain all the records to be updated.

Could i use threads on these temp tables so that the records could get updated simultaneously and could complete fast.
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 47

Expert Comment

by:for_yan
ID: 37721060
I don't see any reason why you can't have 4 different temp tables

I would first start doing it without threads - it is additional complication - look at your times real life situations, if this is some application which you need to run once to update or you'll be using it all the time see how it goes first in termas of time especially if it ios mostly one time opperation- if you need it then think about threads
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 37721463
>>That's tiny for a Map, which is the new record for update
Sorry, I did not get you.

The Map would represent i row in the db. Are you reading and writing ONLY to and from the same row?

Multi-threading is not likely to help you
0
 

Author Comment

by:gaugeta
ID: 37723313
@for_yan and @CEHJ:Thanks for the reply.

This would be a one time operation or load that i would be performing once in a few months.

What I thought was that I'll create 4 temp tables and distribute the total load in these 4 tables and simultaneously kick off 4 threads which would run simultaneously and load the data.

If threading would not be used then there is no reason for me to divide the load in 4 different tables.

What is the issue in using threads in the scenario explained.

Please help...
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37723321
Can you explain what is the essence of calculation which you'll be doing to update the rest of the fields in the table ?
0
 

Author Comment

by:gaugeta
ID: 37723347
@for_yan: Basically I'll be taking about three fields which are the unique fields from each record and based on these three field values I'll be filling up the 52 columns which were empty before from 2 different tables.

Its basically plain mathematical calculation which is done here.
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37723350
so you take three vlaues and look up some values from two more tables based on these three and then use them to calculate values for 52 columns and fill them in ?

So why can't you do it not going to java at all ?
0
 

Author Comment

by:gaugeta
ID: 37723358
@for_yan:Basically the calculations done for a single value in those 52 columns were many where to get the final value i have to perform 5 to 6 calculations and then only i get the final resultant value and the same goes for the rest of the columns too.

And these are very specific mathematical calculations where i need java.lang.math and date and simpledateformat classes etc.

I could not have these many calculations in a stored proc.

What else did you have in mind and what is not suitable with java using threads.
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37723369
I'm pretty sure all that stuff in math and dates is possible to do within oracle itself
within SQL.


In general, if the question is about speed, my way is first to try in straightforward simplest way and then assess what kind of issues you are facing in the sense of timing

the thing is that let's assume threads will be successful and with a lot of additional work may end up say in saving time let's say even two, or assuming a lot, three times but most probably would not end up in orders of magnitudes, whereas moving to PL/SQL from Java in many cases produces order of magnitude saving in time.

And there is a good probablility that the issue would be really in the orders of magintudes or there would be no issue at all - it would all be really fast.
Probability that running it twice as fast will solve your practical problem is much smaller.

So instead of thinking is it worth using threads or not, I'd rather sit down and do it without any threads and see how it performs and then see what is the issue I'm facing and if I face any serious issue at all.
0
 

Author Comment

by:gaugeta
ID: 37723378
@for_yan:Thanks for the reply.
I have had similar loads which ran for about 4 to 5 hours.
The database is sybase in my case.
I will try this thing first without threads.

But if it executes two or even three times fast with threads isn't it worth as i have had very similar loads done previously which takes more than 4 hours.
What is your opinion on this.
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37723400
yes, if it turns out a matter of say two times longer and you have really strict requirement of 4 hours, then playing with threads or say just with splitting the table   (as i understand you process each row independently of  others) may indeed help.
And your work to do it in the straightforward way would not be in vain anyway - I'm sure you'll be able to re-use big parts of the code
0
 

Author Comment

by:gaugeta
ID: 37723632
@for_yan:So in this case of huge data if i have a dual-core processor, what is the maximum threads that i can run simultaneously.
0
 
LVL 47

Expert Comment

by:for_yan
ID: 37723638
I don't think there is any strict limit. I think a few of them may lead to some time saving; further increase will not end up in time saving. I don't think anyone can tell you the exact number. It is trial and error which can give you some number.
0
 
LVL 47

Accepted Solution

by:
for_yan earned 500 total points
ID: 37723645
>This would be a one time operation or load that i would be performing once in a few months.

Given that statement it is hard to understand why 4 hours is OK and, say,  7 hours is not?

I think your time spent on figuring out how to reduce from 7 hours to 4 hours will worth more.

Taht's wahy I'm saying difference in the order of magnitude in such situation can make soem didfference.  To get from 7 hours to 4 hours once in several months at the expense of say a day of debugging does not make sense to me. I would not use any threads.
0
 

Author Comment

by:gaugeta
ID: 37734961
@for_yan:Thanks for the reply.
I'll try this without threads and get back if i face any issues.
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

An old method to applying the Singleton pattern in your Java code is to check if a static instance, defined in the same class that needs to be instantiated once and only once, is null and then create a new instance; otherwise, the pre-existing insta…
Java had always been an easily readable and understandable language.  Some relatively recent changes in the language seem to be changing this pretty fast, and anyone that had not seen any Java code for the last 5 years will possibly have issues unde…
Viewers learn about the scanner class in this video and are introduced to receiving user input for their programs. Additionally, objects, conditional statements, and loops are used to help reinforce the concepts. Introduce Scanner class: Importing…
The viewer will learn how to implement Singleton Design Pattern in Java.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now