get file size before downloading via http

Posted on 2009-07-15
Last Modified: 2012-06-27
In the previous question I was shown how to specify the byte range in an http request (using HTTP::Request and LWP::UserAgent). This allows one to download a specific chunk of a while (e.g., bytes 10000 to 20000).

What I forgot to include in that question is: Can I get the size of the file before specifying the bytes range. Since I will always be retrieving the last x bytes in the file, I would like to get the total size so that I can calculate the range so that the request range is, in effect, total_size minus x to total_size.

Question by:drunnels
  • 3
  • 2

Expert Comment

ID: 24862577
Check the Content-Length response header from code like the following:
use LWP;

my $agent = LWP::UserAgent->new(); 

my $request = HTTP::Request->new(HEAD => ""); 

my $response = $agent->request($request); 



Response portion pertinent:

Content-Length: 3470512

Open in new window


Author Comment

ID: 24862613
I don't think this will work for my purposes. There is a very large file and I need to see the size of it before I request it. I believe that by the time you see the content length in the response, you've already gotten the file.

Expert Comment

ID: 24862627
Negative.  Look carefully at the HTTP::Request->new() call.  We're specifying HEAD, not GET or POST.  That gets -only- the headers, nothing else.

Accepted Solution

Fairlight2cx earned 500 total points
ID: 24862654
Here's the one-liner version, so forgive the spacing, but I'll illustrate the entire response:

 perl -e 'use LWP;my $agent = LWP::UserAgent->new(); my $r

equest = HTTP::Request->new(HEAD => "

rabber_setup.exe"); my $response = $agent->request($request); print($response->a


HTTP/1.1 200 OK

Connection: close

Date: Wed, 15 Jul 2009 18:41:20 GMT

Accept-Ranges: bytes

ETag: "f7a5db-34f4b0-4a2e3ff4"

Server: Apache/1.3.41 (Unix)

Content-Length: 3470512

Content-Type: application/octet-stream

Last-Modified: Tue, 09 Jun 2009 10:56:52 GMT

Client-Date: Wed, 15 Jul 2009 18:41:20 GMT

Client-Response-Num: 1

#####  That's -all- it obtains.  Just headers, since we specify HEAD.

Open in new window


Author Closing Comment

ID: 31603886
Sorry - missed the HEAD obviously.


Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction A frequent question goes something like this, "How can I show an introductory page to my clients on the first site visit, but not show it again on every visit?"  The answer is by using a cookie.  This article shows the design pattern f…
Introduction One of the frequent application design questions goes something like this: "How can I confirm when a client registers on my web site?" The registration might be for general use of a self-administered site like a forum, or for attend…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Migrating to Microsoft Office 365 is becoming increasingly popular for organizations both large and small. If you have made the leap to Microsoft’s cloud platform, you know that you will need to create a corporate email signature for your Office 365…

912 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now