peeldog
asked on
samba load/queue query - unix
I have a application that sends out jobs to around 100 machines, and those jobs running on each machine will request files from the server. The files are anything from bytes to Gb. When too many machines request data, the server hits the limit of what it can output and the jobs end up running slowly as they are waiting on data and this is causing problems.
So I need to measure the load on my samba server so can I stop handing out jobs when the server is at the limit of what it can handle.
I need unix command that returns a single number that represents the load of the server _above_ the limit of what it can handle - i.e. the number of requests in the queue. I want the server to run at capacity, but not queue up too many requests. The throttle must be done by my serving app as denying clients files is not an option. If this number is high, no jobs will be handed out.
Server is running Fedora Core 4
Thanks.
So I need to measure the load on my samba server so can I stop handing out jobs when the server is at the limit of what it can handle.
I need unix command that returns a single number that represents the load of the server _above_ the limit of what it can handle - i.e. the number of requests in the queue. I want the server to run at capacity, but not queue up too many requests. The throttle must be done by my serving app as denying clients files is not an option. If this number is high, no jobs will be handed out.
Server is running Fedora Core 4
Thanks.
ravenpl
is pasing output from 'smbstatus' command applicable?
ASKER
Anything that will run on my Fedora core samba server is fine. Shell and perl scripts are fine too. Even a small c app that I can compile would do. Really I just need a way to measure the load above the limit, and if it is high, I will not hand out any more jobs.
I don't know that smbstatus will give the information I need though. The jobs that are running could have lots of files open for the duration they are running, but I need to know if they are sitting idle waiting on data from the server.
I don't know that smbstatus will give the information I need though. The jobs that are running could have lots of files open for the duration they are running, but I need to know if they are sitting idle waiting on data from the server.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I graphed the i/o wait value last night and it doesn't seem to be too bad. I also graphed the total Send-Q output from netstat and it was pretty bad. I'm throttling based on that and it seems to be helping lots. Is there a way to get statistics based on the Send-Q? If I run netstat it returns what it is at any given moment.
Also is there a way to get the sendq based on an indvidual ip?
This is what I have hacked together for now with php, which seems to be working...
function netstat()
{
$st = `netstat -t -n`;
$lines = split("\n", $st);
$total = 0;
for($i=0; $i<sizeof($lines); $i++)
{
$sp= preg_split("/\s+/", $lines[$i]);
if(sizeof($sp)>2 && is_numeric($sp[2]))
$total+= $sp[2];
}
return $total;
}
function netstat2()
{
error_reporting(E_ALL ^ E_NOTICE);
$st = `netstat -t -n`;
$lines = split("\n", $st);
$iplist = array();
$idlist = array();
$total = 0;
for($i=0; $i<sizeof($lines); $i++)
{
$sp= preg_split("/\s+/", $lines[$i]);
if(sizeof($sp)>2 && is_numeric($sp[2]) && $sp[2] > 0)
{
$value = $sp[2];
$ips = split(":", $sp[4]);
if(sizeof($ips)>1)
{
$ip = $ips[0];
$iplist[$ip]+=$value;
}
$total+= $sp[2];
}
}
}
Does that look right maybe?
I can use the ip address of the destination to see what jobs are causing the traffic and throttle them only, which seems to work great as it will let small jobs that don't need many files through and limit large jobs from hammering the server.
Also is there a way to get the sendq based on an indvidual ip?
This is what I have hacked together for now with php, which seems to be working...
function netstat()
{
$st = `netstat -t -n`;
$lines = split("\n", $st);
$total = 0;
for($i=0; $i<sizeof($lines); $i++)
{
$sp= preg_split("/\s+/", $lines[$i]);
if(sizeof($sp)>2 && is_numeric($sp[2]))
$total+= $sp[2];
}
return $total;
}
function netstat2()
{
error_reporting(E_ALL ^ E_NOTICE);
$st = `netstat -t -n`;
$lines = split("\n", $st);
$iplist = array();
$idlist = array();
$total = 0;
for($i=0; $i<sizeof($lines); $i++)
{
$sp= preg_split("/\s+/", $lines[$i]);
if(sizeof($sp)>2 && is_numeric($sp[2]) && $sp[2] > 0)
{
$value = $sp[2];
$ips = split(":", $sp[4]);
if(sizeof($ips)>1)
{
$ip = $ips[0];
$iplist[$ip]+=$value;
}
$total+= $sp[2];
}
}
}
Does that look right maybe?
I can use the ip address of the destination to see what jobs are causing the traffic and throttle them only, which seems to work great as it will let small jobs that don't need many files through and limit large jobs from hammering the server.