get file size before downloading via http

Posted on 2009-07-15
Last Modified: 2012-06-27
In the previous question I was shown how to specify the byte range in an http request (using HTTP::Request and LWP::UserAgent). This allows one to download a specific chunk of a while (e.g., bytes 10000 to 20000).

What I forgot to include in that question is: Can I get the size of the file before specifying the bytes range. Since I will always be retrieving the last x bytes in the file, I would like to get the total size so that I can calculate the range so that the request range is, in effect, total_size minus x to total_size.

Question by:drunnels
  • 3
  • 2

Expert Comment

ID: 24862577
Check the Content-Length response header from code like the following:
use LWP;

my $agent = LWP::UserAgent->new(); 

my $request = HTTP::Request->new(HEAD => ""); 

my $response = $agent->request($request); 



Response portion pertinent:

Content-Length: 3470512

Open in new window


Author Comment

ID: 24862613
I don't think this will work for my purposes. There is a very large file and I need to see the size of it before I request it. I believe that by the time you see the content length in the response, you've already gotten the file.

Expert Comment

ID: 24862627
Negative.  Look carefully at the HTTP::Request->new() call.  We're specifying HEAD, not GET or POST.  That gets -only- the headers, nothing else.

Accepted Solution

Fairlight2cx earned 500 total points
ID: 24862654
Here's the one-liner version, so forgive the spacing, but I'll illustrate the entire response:

 perl -e 'use LWP;my $agent = LWP::UserAgent->new(); my $r

equest = HTTP::Request->new(HEAD => "

rabber_setup.exe"); my $response = $agent->request($request); print($response->a


HTTP/1.1 200 OK

Connection: close

Date: Wed, 15 Jul 2009 18:41:20 GMT

Accept-Ranges: bytes

ETag: "f7a5db-34f4b0-4a2e3ff4"

Server: Apache/1.3.41 (Unix)

Content-Length: 3470512

Content-Type: application/octet-stream

Last-Modified: Tue, 09 Jun 2009 10:56:52 GMT

Client-Date: Wed, 15 Jul 2009 18:41:20 GMT

Client-Response-Num: 1

#####  That's -all- it obtains.  Just headers, since we specify HEAD.

Open in new window


Author Closing Comment

ID: 31603886
Sorry - missed the HEAD obviously.


Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

Introduction A frequent question goes something like this, "How can I show an introductory page to my clients on the first site visit, but not show it again on every visit?"  The answer is by using a cookie.  This article shows the design pattern f…
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now