Fastest way to retrieve a file from multiple servers using perl

I need to write a function that tries to retrieve a file from a list of webservers. The data I know is:
- filename
- list of servers (array of IP's)

The main point here is that this filename is not on ALL these webservers, it's on only one or maybe two servers and my application doesn't know which one.

A very simple example of function that would do what I need is attached....
This is only for demonstration - my server list contains about 30 servers and it is very time-consuming if the file is on the last server since this function checks these servers one-by-one and if one is maybe not responding it waits a long time for a timeout...

Now I want to tweak this function so it would do the following:
- make more than one request at once (let's say that it would check 5 servers at once) and stop all the requests when it finds this file on one of the servers and after that it would download this file from the server where it's located
- implement shorter timeouts. Since all the webservers are on the same LAN timeouts should be measured in miliseconds - one second is maybe too much (minimum timeout for LWP calls is 1 second)

Thank you for any help!
Start Free Trial

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:

use LWP::Simple;
 
my @servers = ('192.168.2.1','192.168.2.2','192.168.2.3');
my $filename = "somefile.txt";
 
foreach my $server (@servers) {
  my $url = "http://".$server."/".$filename;
  my $destination_file = "/home/temp/".$filename;
 
  if (getstore($url, $destination_file)) {
     print "Stored file in /home/temp\n";
     last;
  } else {
     # we got 404 error / timeout / any other error, proceed to next server in list
     next;
  }
}