?
Solved

wait for data to be written completely?

Posted on 2006-04-14
22
Medium Priority
?
406 Views
Last Modified: 2010-03-31
My program uses a DataOutputStream to write a load of data from a file (a bunch of 1024 element byte arrays) to a web server (via HTTP).

No matter how big the file is, it does this very very quickly - and then updates a progress bar for each packet of data that it sends. However, it then pauses for quite some time [while the progress bar says 100%] - while it waits for the server to respond to the file that's been uploaded... (As if it's waiting for the data to make it's way through the network cables before responding).

This is not exactly what I'm after...

It's updating the progress bar immediately after writing the data -- and not once the data has been recieved by the server... how can I solve this?

My output stream is defined as:

out = new DataOutputStream( new BufferedOutputStream( os ) ) ;


Thanks  =)
0
Comment
Question by:InteractiveMind
  • 8
  • 7
  • 2
  • +2
19 Comments
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16455045
(And yes, I am flushing the stream after writing each chunk)
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16455319
Try it thus:

out = new DataOutputStream( os ) ;
0
 
LVL 14

Expert Comment

by:hoomanv
ID: 16455329
send the data in several chunks and make server send you a delivery message after recieving each chunk
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 86

Expert Comment

by:CEHJ
ID: 16455339
Sorry - that really should have gone in your other question

As far as this one is concerned, the 'real' upload time is going to be difficult to get without a specialized protocol. The most practicable solution is to introduce a bit of 'fake' timing
0
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16455426
>> out = new DataOutputStream( os ) ;
Same problem.


>> send the data in several chunks and make server send you a delivery message after recieving each chunk
I can't, the protocol needs to be based on HTTP ..


>> The most practicable solution is to introduce a bit of 'fake' timing
I'm hoping to leave this as a last resort.
But to do this, would I just add a Thread.sleep() after each flush() ? ... Because I just tried this (to see how it works out), and it seems that the data stops moving while it's paused - so, the delay at the end is still there, but it all just takes a little longer.. I guess shoving it in a timer would solve this. ?


Is there no way of saying 'wait for all of the data to be sent' ?  :\
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16455463
>>Same problem.

You mean same memory problem?

>>would I just add a Thread.sleep() after each flush()

No, i'd do something like scale the percentage in some way

>>Is there no way of saying 'wait for all of the data to be sent' ?

Until when?
0
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16455493
> You mean same memory problem?
Well, I actually meant about it writing something like 10 MB of data within 2 seconds...
But yes, if I try and write a huge file (+ ~30 MB) then I still get the out of mem problem.

> Until when?
I mean, after sending each packet - pause until the buffer has definitely been cleared, etc.. because there's hardly _any_ delay in writing each packet... something's wrong.  :\
0
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16455510
Would I be an idiot to consider that this problem is occuring because I'm writing via a URLConnection rather than a Socket (which I usually do, and have never had a problem with) ?
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16455517
>>pause until the buffer has definitely been cleared

But you don't *want* to pause. You want to get rid of it as quickly as possible ;-)
0
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16455543
lol Yer; but 10 MB in under 2 seconds?? (And this is no T1 connection!!)  ;)
0
 
LVL 86

Accepted Solution

by:
CEHJ earned 1000 total points
ID: 16455643
At this point i'm inclined to offer a 'Gordian knot' suggestion. Don't do your own multipart coding, but use a ready-made library such as HttpClient

http://jakarta.apache.org/commons/httpclient/methods/multipartpost.html
0
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16455730
I suppose that seems reasonable.

I'll give it a go, cheers.
0
 
LVL 92

Assisted Solution

by:objects
objects earned 1000 total points
ID: 16459785
> I suppose that seems reasonable.

As you'll be using the same protocol, you'll get the same issue. In fact it may be wore as you have less control over the process.
Have you tried reducing the number of bytes written in each packet. What ever you do they will likely slow down the overall upload time so I'd question its value. In the past we've setujp the progress bar to onl;y go to say 90% once all data is written, and then the final 10% once connection closed.
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16459821
Incidentally, in case of there being any doubt, my last suggestion was made to facilitate the whole process, not to cure the progress issue. If you want a realistic progress report, you'd be better off using something like RMI
0
 
LVL 25

Author Comment

by:InteractiveMind
ID: 16460247
Hm.

Writing smaller chunks is making no difference either.


I am currently recoding my entire applet -- including this file upload thing.

Can anyone suggest what I might try differently in order to get this to work how I want it to? Should I try and use Socket instead of URLConnection perhaps ?


Thanks
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16460308
>>Should I try and use Socket instead of URLConnection perhaps ?

Don't really see why that would help.

>>Can anyone suggest what I might try differently in order to get this to work how I want it to?

Well, i think i already made them:

a. use RMI to get the responses you need
b. if using HTTP, use an HttpClient library to simply your implementation
0
 
LVL 30

Expert Comment

by:Mayank S
ID: 16464160
>> It's updating the progress bar immediately after writing the data -- and not once the data has been recieved by the server

Can you post the code fragment which updates the progress bar?
0
 
LVL 30

Expert Comment

by:Mayank S
ID: 16464177
>> (And yes, I am flushing the stream after writing each chunk)

BTW, in your code (in the old Q), I saw that you are writing flush () only at the end of the file, not after every chunk that you write using write ( buf, 0, .... ) ;
0
 
LVL 86

Expert Comment

by:CEHJ
ID: 16464323
Can you tell us what happens when all the progress bar functionality is commented out?
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

After being asked a question last year, I went into one of my moods where I did some research and code just for the fun and learning of it all.  Subsequently, from this journey, I put together this article on "Range Searching Using Visual Basic.NET …
Java functions are among the best things for programmers to work with as Java sites can be very easy to read and prepare. Java especially simplifies many processes in the coding industry as it helps integrate many forms of technology and different d…
Video by: Michael
Viewers learn about how to reduce the potential repetitiveness of coding in main by developing methods to perform specific tasks for their program. Additionally, objects are introduced for the purpose of learning how to call methods in Java. Define …
Viewers will learn about arithmetic and Boolean expressions in Java and the logical operators used to create Boolean expressions. We will cover the symbols used for arithmetic expressions and define each logical operator and how to use them in Boole…
Suggested Courses
Course of the Month16 days, 7 hours left to enroll

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question