Solved

Writing a HTTPClient

Posted on 2000-03-13
5
194 Views
Last Modified: 2008-03-17
Hello ,

I try to write a bot which fetches a Website by a given URL using the URL and HTTPCon. classes.
My Problem is, i am using s.th. like << while ( (byteArr = f.read()) != null) >> to check if the read packet is null or not for the while loop condition.
But with this approach I dont get the full file cause sometimes the connection is stalled and I only get half of the site. Is there a possibility which is compactible with HTTP 1.0 and 1.1 to check wether the file is full read or not

Thanx a lot for any advices and hints ...

Binh  
0
Comment
Question by:kaber
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 

Expert Comment

by:kbeier
ID: 2614465
Hi kaber,

What about using a BufferedReader:

BufferedReader in = new BufferedReader(new InputStreamReader(MyHTTPUrlConnection.getInputStream()));

and then using:
String inputLine;
while ((inputLine = in.readLine())
{
    //do your stuff here
}

Just a suggestion.  Hope it helps.

:-)
0
 
LVL 4

Expert Comment

by:jerch
ID: 2614483
Yes, you can use the content-length header (use getContentLength() of HttpURLConnection).  But the problem not all web sites provide this information.  If the content-length is available, you can compare the size of the actual file to the content-length header.  So you would be able to determine if you were able to download the complete file.

Hope this helped...

Jerson
0
 

Expert Comment

by:zevious
ID: 2614851
Use Socket.setSoTimeout();

This might solve it...
0
 

Author Comment

by:kaber
ID: 2615639
Hi ,

kbeier your comment helps me out to solve the prob. Thanx a lot , I would give you the points, but how can I do that ?
0
 

Accepted Solution

by:
kbeier earned 100 total points
ID: 2616012
Hi kaber,

I will post my comment as an answer for you (hopefully this will let you allocate the points ;-)).  My comment was:

What about using a BufferedReader:

BufferedReader in = new BufferedReader(new InputStreamReader(MyHTTPUrlConnection.getInputStream()));

and then using:
String inputLine;
while ((inputLine = in.readLine())
{
    //do your stuff here
}

0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
configure dependency in POM for new database 3 44
throw exception 21 67
how to debug htl and js pages 8 54
Is Applet the way to go for my drag and drop system? 8 35
INTRODUCTION Working with files is a moderately common task in Java.  For most projects hard coding the file names, using parameters in configuration files, or using command-line arguments is sufficient.   However, when your application has vi…
After being asked a question last year, I went into one of my moods where I did some research and code just for the fun and learning of it all.  Subsequently, from this journey, I put together this article on "Range Searching Using Visual Basic.NET …
Viewers will learn one way to get user input in Java. Introduce the Scanner object: Declare the variable that stores the user input: An example prompting the user for input: Methods you need to invoke in order to properly get  user input:
Viewers will learn about basic arrays, how to declare them, and how to use them. Introduction and definition: Declare an array and cover the syntax of declaring them: Initialize every index in the created array: Example/Features of a basic arr…

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question