Solved

Writing a HTTPClient

Posted on 2000-03-13
5
189 Views
Last Modified: 2008-03-17
Hello ,

I try to write a bot which fetches a Website by a given URL using the URL and HTTPCon. classes.
My Problem is, i am using s.th. like << while ( (byteArr = f.read()) != null) >> to check if the read packet is null or not for the while loop condition.
But with this approach I dont get the full file cause sometimes the connection is stalled and I only get half of the site. Is there a possibility which is compactible with HTTP 1.0 and 1.1 to check wether the file is full read or not

Thanx a lot for any advices and hints ...

Binh  
0
Comment
Question by:kaber
5 Comments
 

Expert Comment

by:kbeier
Comment Utility
Hi kaber,

What about using a BufferedReader:

BufferedReader in = new BufferedReader(new InputStreamReader(MyHTTPUrlConnection.getInputStream()));

and then using:
String inputLine;
while ((inputLine = in.readLine())
{
    //do your stuff here
}

Just a suggestion.  Hope it helps.

:-)
0
 
LVL 4

Expert Comment

by:jerch
Comment Utility
Yes, you can use the content-length header (use getContentLength() of HttpURLConnection).  But the problem not all web sites provide this information.  If the content-length is available, you can compare the size of the actual file to the content-length header.  So you would be able to determine if you were able to download the complete file.

Hope this helped...

Jerson
0
 

Expert Comment

by:zevious
Comment Utility
Use Socket.setSoTimeout();

This might solve it...
0
 

Author Comment

by:kaber
Comment Utility
Hi ,

kbeier your comment helps me out to solve the prob. Thanx a lot , I would give you the points, but how can I do that ?
0
 

Accepted Solution

by:
kbeier earned 100 total points
Comment Utility
Hi kaber,

I will post my comment as an answer for you (hopefully this will let you allocate the points ;-)).  My comment was:

What about using a BufferedReader:

BufferedReader in = new BufferedReader(new InputStreamReader(MyHTTPUrlConnection.getInputStream()));

and then using:
String inputLine;
while ((inputLine = in.readLine())
{
    //do your stuff here
}

0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
Introduction This article is the last of three articles that explain why and how the Experts Exchange QA Team does test automation for our web site. This article covers our test design approach and then goes through a simple test case example, how …
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
This tutorial explains how to use the VisualVM tool for the Java platform application. This video goes into detail on the Threads, Sampler, and Profiler tabs.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now