wait for data to be written completely?

My program uses a DataOutputStream to write a load of data from a file (a bunch of 1024 element byte arrays) to a web server (via HTTP).

No matter how big the file is, it does this very very quickly - and then updates a progress bar for each packet of data that it sends. However, it then pauses for quite some time [while the progress bar says 100%] - while it waits for the server to respond to the file that's been uploaded... (As if it's waiting for the data to make it's way through the network cables before responding).

This is not exactly what I'm after...

It's updating the progress bar immediately after writing the data -- and not once the data has been recieved by the server... how can I solve this?

My output stream is defined as:

out = new DataOutputStream( new BufferedOutputStream( os ) ) ;


Thanks  =)
LVL 25
InteractiveMindAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

InteractiveMindAuthor Commented:
(And yes, I am flushing the stream after writing each chunk)
0
CEHJCommented:
Try it thus:

out = new DataOutputStream( os ) ;
0
hoomanvCommented:
send the data in several chunks and make server send you a delivery message after recieving each chunk
0
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

CEHJCommented:
Sorry - that really should have gone in your other question

As far as this one is concerned, the 'real' upload time is going to be difficult to get without a specialized protocol. The most practicable solution is to introduce a bit of 'fake' timing
0
InteractiveMindAuthor Commented:
>> out = new DataOutputStream( os ) ;
Same problem.


>> send the data in several chunks and make server send you a delivery message after recieving each chunk
I can't, the protocol needs to be based on HTTP ..


>> The most practicable solution is to introduce a bit of 'fake' timing
I'm hoping to leave this as a last resort.
But to do this, would I just add a Thread.sleep() after each flush() ? ... Because I just tried this (to see how it works out), and it seems that the data stops moving while it's paused - so, the delay at the end is still there, but it all just takes a little longer.. I guess shoving it in a timer would solve this. ?


Is there no way of saying 'wait for all of the data to be sent' ?  :\
0
CEHJCommented:
>>Same problem.

You mean same memory problem?

>>would I just add a Thread.sleep() after each flush()

No, i'd do something like scale the percentage in some way

>>Is there no way of saying 'wait for all of the data to be sent' ?

Until when?
0
InteractiveMindAuthor Commented:
> You mean same memory problem?
Well, I actually meant about it writing something like 10 MB of data within 2 seconds...
But yes, if I try and write a huge file (+ ~30 MB) then I still get the out of mem problem.

> Until when?
I mean, after sending each packet - pause until the buffer has definitely been cleared, etc.. because there's hardly _any_ delay in writing each packet... something's wrong.  :\
0
InteractiveMindAuthor Commented:
Would I be an idiot to consider that this problem is occuring because I'm writing via a URLConnection rather than a Socket (which I usually do, and have never had a problem with) ?
0
CEHJCommented:
>>pause until the buffer has definitely been cleared

But you don't *want* to pause. You want to get rid of it as quickly as possible ;-)
0
InteractiveMindAuthor Commented:
lol Yer; but 10 MB in under 2 seconds?? (And this is no T1 connection!!)  ;)
0
CEHJCommented:
At this point i'm inclined to offer a 'Gordian knot' suggestion. Don't do your own multipart coding, but use a ready-made library such as HttpClient

http://jakarta.apache.org/commons/httpclient/methods/multipartpost.html
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
InteractiveMindAuthor Commented:
I suppose that seems reasonable.

I'll give it a go, cheers.
0
objectsCommented:
> I suppose that seems reasonable.

As you'll be using the same protocol, you'll get the same issue. In fact it may be wore as you have less control over the process.
Have you tried reducing the number of bytes written in each packet. What ever you do they will likely slow down the overall upload time so I'd question its value. In the past we've setujp the progress bar to onl;y go to say 90% once all data is written, and then the final 10% once connection closed.
0
CEHJCommented:
Incidentally, in case of there being any doubt, my last suggestion was made to facilitate the whole process, not to cure the progress issue. If you want a realistic progress report, you'd be better off using something like RMI
0
InteractiveMindAuthor Commented:
Hm.

Writing smaller chunks is making no difference either.


I am currently recoding my entire applet -- including this file upload thing.

Can anyone suggest what I might try differently in order to get this to work how I want it to? Should I try and use Socket instead of URLConnection perhaps ?


Thanks
0
CEHJCommented:
>>Should I try and use Socket instead of URLConnection perhaps ?

Don't really see why that would help.

>>Can anyone suggest what I might try differently in order to get this to work how I want it to?

Well, i think i already made them:

a. use RMI to get the responses you need
b. if using HTTP, use an HttpClient library to simply your implementation
0
Mayank SAssociate Director - Product EngineeringCommented:
>> It's updating the progress bar immediately after writing the data -- and not once the data has been recieved by the server

Can you post the code fragment which updates the progress bar?
0
Mayank SAssociate Director - Product EngineeringCommented:
>> (And yes, I am flushing the stream after writing each chunk)

BTW, in your code (in the old Q), I saw that you are writing flush () only at the end of the file, not after every chunk that you write using write ( buf, 0, .... ) ;
0
CEHJCommented:
Can you tell us what happens when all the progress bar functionality is commented out?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.