How to force one thread per download

style-sheets
style-sheets used Ask the Experts™
on
Hi,

I have a php script that serve files dynamically.

Is there a reliable way to force anyone who call my script (ie. download file served) to use only 1 thread (ie. especially those using flashget / download managers that create multiple threads per download)

Thanks!
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Most Valuable Expert 2011
Top Expert 2016

Commented:
You might try putting a cookie on their browser.  Use an expiration of zero, so the cookie goes away when they close the browser.  If you find the cookie in the request, you could issue a message or die or something like that.

Of course for this to work, the downloading application would need to act like a well-behaved browser.  Hackers probably will not do that.

You could also require some kind of login before permitting a download.  That would enable you to tighten up the cookie environment so the other part of this would work more reliably.

Author

Commented:
Thanks, yes file access is indeed restricted & does require a login.

With that said, I don't want to block flashget / download accelerators. What I want is to block the option that those softwares open 5-10 connection to the same file (which does break my script logic).
Darude1234IT Employee (Developer & Helpdesk)

Commented:
I think you have to limit the connection serverside and not in your script.
I don't know if you are using shared hosting or a private server. In case of shared hosting this will not going to work, but when using a private server you can limit the connections using iptables. See here for an example.

Another way when you are using apache, you can install a script for apache like this: http://dominia.org/djao/limitipconn.html
Acronis in Gartner 2019 MQ for datacenter backup

It is an honor to be featured in Gartner 2019 Magic Quadrant for Datacenter Backup and Recovery Solutions. Gartner’s MQ sets a high standard and earning a place on their grid is a great affirmation that Acronis is delivering on our mission to protect all data, apps, and systems.

Aaron TomoskyDirector of Solutions Consulting

Commented:
You can do this manually... Use a session or cookie or whatever to uniquely identify the user. When they start downloading a file, put a record in a table with their sessionid and the fileid an a timestamp. When the file is done clear the record.

Kinda a pain, can slow down your site if not done carefully, would need to be very careful with additional timeouts and error checking so as not to block access with orphaned entries

Author

Commented:
@Darude1234

Thanks for the suggestion. I have a dedicated server. I'm going to take a look @iptables and will get back to you.

@aarontomosky Thank you, but the cookie has already been suggested by Ray_Paseur (please take a look at my reply)
Most Valuable Expert 2011
Top Expert 2016

Commented:
Just curious - how big are the things that people download from your site?

Author

Commented:
@Ray_Paseur

File size greatly vary, it could be several GBs :(

Commented:
Messing with IPtables is also going to limit web browser connections for the regular web site.  When a user loads a page many threads can be opened to improve load time.  You don't want to block this.

To limit downloads to one thread per IP, use Apache with mod_limitipconn

mod_limitipconn: http://dominia.org/djao/limitipconn2.html

Then in your httpd.conf, for example, you can set this:
<IfModule mod_limitipconn.c>
   <Location /mydownloadfiles>
   MaxConnPerIP 1
   </Location>
</IfModule>

Open in new window

Author

Commented:
Thank you burnsj2, does this allows two different downloads at the same time (one different download = one connection)?

I'm asking because since it's dynamic, and since served files can be large (streamed in batch), I'm worried that users can only initiate one download at a time, which is a fairly serious limitation for them.
Aaron TomoskyDirector of Solutions Consulting

Commented:
If you only want one download per user per file I don't think there is an easy iptables solution. I could be wrong...
Commented:
MaxConnPerIP is going to put a hard limit on the connection from that IP to the specified directory.

These download accelerators would need to make partial download requests, so just block those HTTP header requests in httpd.conf


RewriteCond %{HTTP:Range} !^$
RewriteRule ^mydownloadfiles/.+$ - [F,L]

Open in new window

Author

Commented:
@burnsj2 So, if I understood you correctly, with MaxConnPerIP set to 1, only *one* download is possible at a time, is that correct?
Aaron TomoskyDirector of Solutions Consulting

Commented:
That partial download blocker is slick! It will however stop the user from resuming partial downloads rIght?

Commented:
style-sheets: correct, mod_limitipconn is not going to be able to block multiple downloads of the same file while still allowing multiple downloads of different files, but you may want to use it anyway to establish some reasonable limit of simultaneous downloads, say "MaxConnPerIP 5".

aarontomosky is right, the RewriteRule is going to block all partial downloads.

Author

Commented:
@burnsj2 I'm not sure we're talking about the same thing, so please let me re-explain the situation and hopefully clarify what I meant earlier:

Let's say I have this directive in http.conf:

<IfModule mod_limitipconn.c>
   <Location /download-files.php>
   MaxConnPerIP 1
   </Location>
</IfModule>

Open in new window


Please keep in mind that files are user-specific, they are not shared among users.

If I understood you correctly, according to the directive above:

1.    User A can download file_1.zip using FlashGet, and while downloading this file, s/he

    a.  Still can browse website normally (because of the "Location" condition)
    b.  Cannot initiate another download of file_2.zip because s/he already reached the limit
    c.  FlashGet / download managers can only initiate 1 connection because of the RewriteCond %{HTTP:Range} !^$ directive.

2.    IF I set MaxConnPerIP to 5, user A can download file_1.zip using FlashGet, and while downloading this file, s/he

    a.  Still can browse website normally (because of the "Location" condition)
    b.  Can initiate up to 4 other downloads (of the same or different files)
    c.  FlashGet / download managers can only initiate 1 connection because of the RewriteCond %{HTTP:Range} !^$ directive.

is that correct?

Not being able to resume partial downloads is a real shame, but it's reasonable trade off for now.

Thanks!

Commented:
That's exactly correct.  The only thing I see wrong is "Location /download-files.php".  This wouldn't work because "location needs to be a directory of files rather than a script.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial