BlakeEM
asked on
Perl socket/select with threads help
I have a server and client right now use IO::Socket, IO::Select, and I was trying to use threads to split off the connections so I can have more than one at a time.
Currently the server_process function processes anything that client sends until they disconnect. Mostly it's the client sending same text and the server sends some reply data/text. I want to be able to connect any number of clients to the server and to take turns requesting data over a persistent connection and I figured IO select used to sort all that stuff out (so I assume.)
I'm wondering if having more then one client connected on the same socket and sending data is even possible in the first place?
When I start the 2nd client it just hangs at the first command until the server is done with the first connection, after this it then processes whatever the 2nd client sent.
The server also never even gets back to the top of the loop after I start the first thread, I'm not sure why this part isn't even working (even if my socket stuff is wrong.) I think I may need to use fork instead, but I don't see why thread doesn't seem to be working.
Any help would be much appreciated.
Currently the server_process function processes anything that client sends until they disconnect. Mostly it's the client sending same text and the server sends some reply data/text. I want to be able to connect any number of clients to the server and to take turns requesting data over a persistent connection and I figured IO select used to sort all that stuff out (so I assume.)
I'm wondering if having more then one client connected on the same socket and sending data is even possible in the first place?
When I start the 2nd client it just hangs at the first command until the server is done with the first connection, after this it then processes whatever the 2nd client sent.
The server also never even gets back to the top of the loop after I start the first thread, I'm not sure why this part isn't even working (even if my socket stuff is wrong.) I think I may need to use fork instead, but I don't see why thread doesn't seem to be working.
Any help would be much appreciated.
#### SERVER CODE
use strict;
use IO::Socket;
use Crypt::CBC;
use threads;
use IO::Select;
##################################
# Socket Connection Server Setup #
##################################
our $server = IO::Socket::INET->new(
Listen => 16,
LocalAddr => '192.168.10.105',
LocalPort => 5050,
Proto => 'tcp'
) or die "Can't create server socket: $!";
$server->autoflush(1);
my $read_set = new IO::Select();
$read_set->add($server);
while(1)
{
print "top of WHILE\n";
my ($rh_set) = IO::Select->select($read_set);
foreach $rh (@$rh_set)
{
if ($rh == $server)
{
print "Connection Made!\n";
my $ns = $rh->accept();
$read_set->add($ns);
}
else
{
print "START!\n";
threads->create(\&server_process($rh))->detach();
print "DONE!\n";
$read_set->remove($rh);
close($rh);
}
}
}
sub server_process($)
{
my $client = shift;
while (<$client>)
{
# process all input from this client
}
}
The IO::Select will tell you if a file handle is ready for reading. Using this, you don't need to have multiple threads. You can check to see if any file handle is ready for reading. If so, only read from those ready - and because there is data, it won't block. If you use multiple threads, you could have each thread read with blocking from one socket. As soon as there is data, it would get returned. You don't need to care if a child threads blocks.
ASKER
I don't fully understand what you're saying Adam but I got rid of the threads and added a timeout and I was able to get it working close to what I wanted. I think to get it fully working as I wanted I'd need to use a range of ports instead so all can have constant access to multiple clients real time. I have yet to find something to do this.
I do have one issue however that should be easy.
In this code where I check if the socket is valid. When I check that it's valid I have read the socket and already lost my data so when it gets to my code it just hangs there because the data from the server has already been read. Without this part in my code none of my sockets properly close and I get errors.
What's a good way to validate the nonseekable file handle without incrementing it? Every example I found passed to a $buf variable however the code still hangs when doing this.
I do have one issue however that should be easy.
In this code where I check if the socket is valid. When I check that it's valid I have read the socket and already lost my data so when it gets to my code it just hangs there because the data from the server has already been read. Without this part in my code none of my sockets properly close and I get errors.
What's a good way to validate the nonseekable file handle without incrementing it? Every example I found passed to a $buf variable however the code still hangs when doing this.
foreach $rh (@$rh_set)
{
if ($rh == $server)
{
print "Primary Connection!\n";
my $ns = $rh->accept();
$read_set->add($ns);
}
else
{
# my problem starts here
$buf = <$rh>;
if ($buf)
{
print "START!\n";
server_process($buf);
}
else
{
print "DONE!\n";
$read_set->remove($rh);
close($rh);
}
}
}
sub server_process($)
{
my $client = shift;
while(<$client>)
{
#do stuff
}
}
This line:
$buf = <$rh>;
will wait until there is a line to read on $rh. If $rh isn't ready to be read, your program will hang here until it is.
The IO::Select will let you get the set of file handles that are ready for reading, so you only read those that are ready. Something like this:
$buf = <$rh>;
will wait until there is a line to read on $rh. If $rh isn't ready to be read, your program will hang here until it is.
The IO::Select will let you get the set of file handles that are ready for reading, so you only read those that are ready. Something like this:
@ready = $read_set->can_read();
foreach $rh (@ready) {
if ($rh == $server)
{
print "Primary Connection!\n";
my $ns = $rh->accept();
$read_set->add($ns);
}
else
{
#now it should block, because you are only looping through
#those that have something to read
$buf = <$rh>;ASKER
I am doing it that way now but the problem is I still can't get past $buf = <$rh> unless I send another line from the server but that seems like a bad way to get around the issue.
When the client sends a message to the server the client hangs at that line waiting for the reply and I can never push that first reply into my function as long as the line $buf = <$rh>; is there... if I don't have that line then I can never turn off closed connections and I get a ton of errors.
I also don't want to simply close the connection every message, I only want to close it when the client disconnects. I can't check if they are disconnected without checking the file pointer and that increments it so I miss the reply.
When the client sends a message to the server the client hangs at that line waiting for the reply and I can never push that first reply into my function as long as the line $buf = <$rh>; is there... if I don't have that line then I can never turn off closed connections and I get a ton of errors.
I also don't want to simply close the connection every message, I only want to close it when the client disconnects. I can't check if they are disconnected without checking the file pointer and that increments it so I miss the reply.
The code you posted is very different from the code I posted. In what I posted, the @ready array is from the IO::Select module. It is an array of filehandles that are ready for reading. The foreach then only loops through those that are ready. In what you have posted, the code loops through all filehandles.
If using something similar to what I posted, it will always hang at the <$rh>, then you might have the input record separator set to something other than what the client is using to end their messages.
If using something similar to what I posted, it will always hang at the <$rh>, then you might have the input record separator set to something other than what the client is using to end their messages.
ASKER
well when I switched to using the can_read() I didn't notice anything different in that array as opposed to the one coming off the select? The select was already returning only the ones that are readable so doing can_read seems redundant to me.
So I guess there is just no way to see if a file handle if readable without actually incrementing it?
"you might have the input record separator set to something other than what the client is using to end their messages."
I don't fully understand what you're saying here. I thought all messages had to end in /n otherwise the line is never seen.
So I guess there is just no way to see if a file handle if readable without actually incrementing it?
"you might have the input record separator set to something other than what the client is using to end their messages."
I don't fully understand what you're saying here. I thought all messages had to end in /n otherwise the line is never seen.
I thought in your second post, the $rh_set was all of the filehandles, not just the ones from the select. If it is the ones from the select, then it will be the same as the can_read, and you can ignore that part of my previous post.
For the input record separator, the client can use whatever they want to end a message. The "\n" is common, but it could be anything. Then, when reading using <$rh>, this will wait until an input record separator is found. If it is waiting for "\n" (the default), but the client doesn't send one - but sends some data, this will block.
What OS are you on? The documentation states that on some OSes, the select will report a socket as ready for reading even when it isn't - causing the next read to block. The work-around is to use the O_NONBLOCK flag on the socket.
For the input record separator, the client can use whatever they want to end a message. The "\n" is common, but it could be anything. Then, when reading using <$rh>, this will wait until an input record separator is found. If it is waiting for "\n" (the default), but the client doesn't send one - but sends some data, this will block.
What OS are you on? The documentation states that on some OSes, the select will report a socket as ready for reading even when it isn't - causing the next read to block. The work-around is to use the O_NONBLOCK flag on the socket.
ASKER
I'm doing this on windows now but it's going to need to run on linux as well.
Going by what you say the best way to maybe fix this would be put a \n at the start of my commands that way they will get past the first <$rh> and when it gets into my function to do the processing it will have data.
Going by what you say the best way to maybe fix this would be put a \n at the start of my commands that way they will get past the first <$rh> and when it gets into my function to do the processing it will have data.
That will affect how you do your processing. If you add a \n to the start of your commands, then this:
<$rh>
will get an empty line.
The best way to fix this would be to find the cause of it hanging. If it is hanging because the select returns the file handle as being ready for reading when it isn't, then adding a \n to the start of the command will not solve the problem.
<$rh>
will get an empty line.
The best way to fix this would be to find the cause of it hanging. If it is hanging because the select returns the file handle as being ready for reading when it isn't, then adding a \n to the start of the command will not solve the problem.
ASKER
I have known why it's hanging. It's reading the first line before my function can read the first line.
Normally that first line goes directly into my server_process function and is read by a while loop like so...
while(<$client>)
That line is ready to be read, only by checking if it's valid to be read I am reading it so it causes my while loop to hang because after that line my client doesn't send anything else so it hangs waiting for another line or an end of file.
Only logical way I can see around this is to view that line without automatically incrementing the pointer (something I don't think is possible,) or to have the client send an initialization line first then send the real command so it can push that into my server_process function without stopping after the first command.
I was just hoping there was a more clean way of doing this without having to send more data before the command line that I need.
Normally that first line goes directly into my server_process function and is read by a while loop like so...
while(<$client>)
That line is ready to be read, only by checking if it's valid to be read I am reading it so it causes my while loop to hang because after that line my client doesn't send anything else so it hangs waiting for another line or an end of file.
Only logical way I can see around this is to view that line without automatically incrementing the pointer (something I don't think is possible,) or to have the client send an initialization line first then send the real command so it can push that into my server_process function without stopping after the first command.
I was just hoping there was a more clean way of doing this without having to send more data before the command line that I need.
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
There should be a next after the close, so you don't try to process data on the closed handle... so between line 17 and 18, add this:
next;ASKER
Ok I got it working the best way I could. Before a typed command from the client I first send another "ok\n" line first and this fixed the issue like I figured it would.
This should work for now unless there is a better way.
This should work for now unless there is a better way.
ASKER
The reason I did it the way I did is because the server_process currently does file send/receive over the same socket depending on the command sent so I'm not simply just doing commands back and forth.
On the client side I just do something like...
$msg = <STDIN>;
print $server send_prepare("OK\n");
print $server send_prepare($msg);
send_prepare just handles encryption and such, but you get the idea of what I'm doing.
That way the first check picks up the OK\n and then pushes the command and anything else goes into my server_process function.
On the client side I just do something like...
$msg = <STDIN>;
print $server send_prepare("OK\n");
print $server send_prepare($msg);
send_prepare just handles encryption and such, but you get the idea of what I'm doing.
That way the first check picks up the OK\n and then pushes the command and anything else goes into my server_process function.
while(1)
{
print "top of WHILE\n";
my ($rh_set) = IO::Select->select($read_set, undef, undef, 1);
foreach $rh (@$rh_set)
{
if ($rh == $server)
{
print "Primary Connection!\n";
my $ns = $rh->accept();
$read_set->add($ns);
}
else
{
$buf = <$rh>;
if ($buf)
{
print "START!\n";
server_process($rh);
}
else
{
print "DONE!\n";
$read_set->remove($rh);
close($rh);
}
}
}
}
As long as it is working... but with this method, a single client that gets hung up will hang your entire server, which could in turn hang all other clients. That might not be a concern though.
ASKER
I tried your code with the unless and it does the exact same thing as the if and I still need to send it the extra line first otherwise it hangs.
This way seems to be working good. Any time I kill the client it drops whatever it was doing and continues and if I send both those lines at the same time it shouldn't ever get stuck hanging.
This way seems to be working good. Any time I kill the client it drops whatever it was doing and continues and if I send both those lines at the same time it shouldn't ever get stuck hanging.
okay. post again if you have any other troubles...