get file size before downloading via http

Posted on 2009-07-15
Medium Priority
Last Modified: 2012-06-27
In the previous question I was shown how to specify the byte range in an http request (using HTTP::Request and LWP::UserAgent). This allows one to download a specific chunk of a while (e.g., bytes 10000 to 20000).

What I forgot to include in that question is: Can I get the size of the file before specifying the bytes range. Since I will always be retrieving the last x bytes in the file, I would like to get the total size so that I can calculate the range so that the request range is, in effect, total_size minus x to total_size.

Question by:drunnels
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2

Expert Comment

ID: 24862577
Check the Content-Length response header from code like the following:
use LWP;
my $agent = LWP::UserAgent->new(); 
my $request = HTTP::Request->new(HEAD => "http://members.iglou.com/fairlite/cod4_mapgrabber_setup.exe"); 
my $response = $agent->request($request); 
Response portion pertinent:
Content-Length: 3470512

Open in new window


Author Comment

ID: 24862613
I don't think this will work for my purposes. There is a very large file and I need to see the size of it before I request it. I believe that by the time you see the content length in the response, you've already gotten the file.

Expert Comment

ID: 24862627
Negative.  Look carefully at the HTTP::Request->new() call.  We're specifying HEAD, not GET or POST.  That gets -only- the headers, nothing else.

Accepted Solution

Fairlight2cx earned 2000 total points
ID: 24862654
Here's the one-liner version, so forgive the spacing, but I'll illustrate the entire response:

 perl -e 'use LWP;my $agent = LWP::UserAgent->new(); my $r
equest = HTTP::Request->new(HEAD => "http://members.iglou.com/fairlite/cod4_mapg
rabber_setup.exe"); my $response = $agent->request($request); print($response->a
HTTP/1.1 200 OK
Connection: close
Date: Wed, 15 Jul 2009 18:41:20 GMT
Accept-Ranges: bytes
ETag: "f7a5db-34f4b0-4a2e3ff4"
Server: Apache/1.3.41 (Unix)
Content-Length: 3470512
Content-Type: application/octet-stream
Last-Modified: Tue, 09 Jun 2009 10:56:52 GMT
Client-Date: Wed, 15 Jul 2009 18:41:20 GMT
Client-Response-Num: 1
#####  That's -all- it obtains.  Just headers, since we specify HEAD.

Open in new window


Author Closing Comment

ID: 31603886
Sorry - missed the HEAD obviously.


Featured Post

Enroll in August's Course of the Month

August's CompTIA IT Fundamentals course includes 19 hours of basic computer principle modules and prepares you for the certification exam. It's free for Premium Members, Team Accounts, and Qualified Experts!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction This warning has to be one of the most commonly issued warnings in the history of PHP.  The article explains why this warning arises and what to do to mitigate the problem. How this Happens HTTP headers include many different kinds…
A few customers have recently asked my thoughts on Password Managers.  As Security is a big part of our industry I was initially very hesitant and sceptical about giving a program all of my secret passwords.  But as I was getting asked about them mo…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question