get file size before downloading via http

Posted on 2009-07-15
Last Modified: 2012-06-27
In the previous question I was shown how to specify the byte range in an http request (using HTTP::Request and LWP::UserAgent). This allows one to download a specific chunk of a while (e.g., bytes 10000 to 20000).

What I forgot to include in that question is: Can I get the size of the file before specifying the bytes range. Since I will always be retrieving the last x bytes in the file, I would like to get the total size so that I can calculate the range so that the request range is, in effect, total_size minus x to total_size.

Question by:drunnels
  • 3
  • 2

Expert Comment

ID: 24862577
Check the Content-Length response header from code like the following:
use LWP;
my $agent = LWP::UserAgent->new(); 
my $request = HTTP::Request->new(HEAD => ""); 
my $response = $agent->request($request); 
Response portion pertinent:
Content-Length: 3470512

Open in new window


Author Comment

ID: 24862613
I don't think this will work for my purposes. There is a very large file and I need to see the size of it before I request it. I believe that by the time you see the content length in the response, you've already gotten the file.

Expert Comment

ID: 24862627
Negative.  Look carefully at the HTTP::Request->new() call.  We're specifying HEAD, not GET or POST.  That gets -only- the headers, nothing else.

Accepted Solution

Fairlight2cx earned 500 total points
ID: 24862654
Here's the one-liner version, so forgive the spacing, but I'll illustrate the entire response:

 perl -e 'use LWP;my $agent = LWP::UserAgent->new(); my $r
equest = HTTP::Request->new(HEAD => "
rabber_setup.exe"); my $response = $agent->request($request); print($response->a
HTTP/1.1 200 OK
Connection: close
Date: Wed, 15 Jul 2009 18:41:20 GMT
Accept-Ranges: bytes
ETag: "f7a5db-34f4b0-4a2e3ff4"
Server: Apache/1.3.41 (Unix)
Content-Length: 3470512
Content-Type: application/octet-stream
Last-Modified: Tue, 09 Jun 2009 10:56:52 GMT
Client-Date: Wed, 15 Jul 2009 18:41:20 GMT
Client-Response-Num: 1
#####  That's -all- it obtains.  Just headers, since we specify HEAD.

Open in new window


Author Closing Comment

ID: 31603886
Sorry - missed the HEAD obviously.


Featured Post

Courses: Start Training Online With Pros, Today

Brush up on the basics or master the advanced techniques required to earn essential industry certifications, with Courses. Enroll in a course and start learning today. Training topics range from Android App Dev to the Xen Virtualization Platform.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (…
Since pre-biblical times, humans have sought ways to keep secrets, and share the secrets selectively.  This article explores the ways PHP can be used to hide and encrypt information.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This video shows how to quickly and easily add an email signature for all users on Exchange 2016. The resulting signature is applied on a server level by Exchange Online. The email signature template has been downloaded from: www.mail-signatures…

786 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question