• C

force select() to unblock, from another thread

I have a thread that blocks on select(), waiting for something to happen on one of many sockets that are connected to it.

Meanwhile, in another thread, I need to make a new socket connection, and add that socket to the list that select is monitoring (the fd_set).

The problem is that select() is blocked until something happens on one of its existing set of sockets.  I'd like to do something to trigger it to return (i.e.unblock), so it can start again with a modified fd_set that includes the new socket.

What I *could* do is create a new socket connection to itself, then immdiately disconnect.  Alternatively I could leave a socket connected, and send a couple bytes across it every time I want select to return.

I don't like these for two reasons...one, they are messy and ugly, and two, I otherwise don't need a listener socket and would prefer not have one.  I want to have this program be a client in the sense that it will initiate all connections, and having it listen on a port simply so it can "talk to itself" to wake up select() seems to be a really bad ugly brute-force way to do things.

Any ideas as to a better way?
LVL 1
catbuttAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Tommy BraasCommented:
What is your system supposed to do? Knowing the whole, might enable me to help you solve this particular issue.

\t
0
BlueTrinCommented:
This is a general IO problem (well I think that you didn't setup the IO to NONBLOCK with fcntl)

int main()
{
    printf ("initializing...\n");

    // initialize socket
    if ((s = socket (AF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0)
    {
        perror ("socket");
        return 0;
    }

    unsigned flags = fcntl (s, F_GETFL);
    if (fcntl (s, F_SETFL, flags | O_NONBLOCK))
    {
        perror ("fcntl");
        return 0;
    }

... etc


A program does nonblocking I/O by setting the O_NONBLOCK flag associated with the file descriptor. The fcntl system call modifies the flag

0
jcaldwelCommented:
If you are multithreaded already, it may be easier to block on a server socket, and spin connections off into their own thread. I just refactored a project to do this, and it simplifies it greatly.
0
Hey MSSPs! What's your total cost of ownership?

WEBINAR: Managed security service providers often deploy & manage products from a variety of solution vendors. But is this really the best approach when it comes to saving time AND money? Join us on Aug. 15th to learn how you can improve your total cost of ownership today!

catbuttAuthor Commented:
"If you are multithreaded already, it may be easier to block on a server socket, and spin connections off into their own thread. I just refactored a project to do this, and it simplifies it greatly."

Thanks, but no.  The program works great as is (aside from this one messy detail), I don't want 200+ processes running, which is the number of typical connections, and various other reasons the non-blocking select based model works great.  Even if I wanted to go with a non-blocking one-thread-per-connection design, I don't want to rewrite something with a lot of work and debugging behind it simply to avoid one ugly workaround.
0
catbuttAuthor Commented:
"This is a general IO problem (well I think that you didn't setup the IO to NONBLOCK with fcntl)"

All the sockets are correctly set up as non-blocking.  Select() blocks of course, as it is supposed to.  All I want to do is to interrupt select() long enough to add one more socket to its list.

Thanks anyway.
0
catbuttAuthor Commented:
"What is your system supposed to do?"

Well, I can try to explain. But I can tell you (as I mentioned above) I am looking to solve one problem, not re-architect the whole application.  I can live with the ugly workaround that I suggested, but I am hoping for a more elegant way to address it.

The program is a "hub" which connects over 200 nodes (applications, typically running on separate machines from the hub).  There is an open connection (i.e. a long term socket connection) between each application and the hub.  Small messages can be sent from either side of the connection.  This allows me to monitor the applications, it allows the apps to message each other through the hub, etc. For each message sent, there isn't the overhead of making a socket connection, which would make it too slow for many of the purposes I am using it for.

In its current design (which is working well, is solid and is ultra fast), the nodes initiate the connection to the hub, making each node a "client" in the traditional sense.  However, once the connection is made, messages can be sent either way, so there isn't really a client server relationship anymore, it's very a very symmetrical relationship, making them kind of "peers".

One thread on the hub listens for incoming messages (or new connections) and possibly send messages in response, and a second thread would send messages in response to external input.

Now I want to change the design a bit to make it so that instead of the *nodes* initiating the connection to the hub, the hub connects to a list of known nodes (I provide a list of ip addresses and ports, which changes rarely).  It will no longer listen for incoming connections from the nodes, as all connections would be initiated on the hub side. This way it is the responsibility of the hub to try to reconnect when a connection gets shut down, to alert me when one of the nodes doesn't respond for a while, etc.  It cleans up the whole thing considerably to make it work this way (aside from this one detail).

Since the connections are made rarely (when the hub starts up, when one of the applications restarts, or when one of the connections dies for some reason), I'm not overly concerned about ultra efficiency for the process of connecting.  That's why, if necessary, I can do things like have it "ping" itself just to unblock select() temporarily.  I don't like it, but if I have to, that's what I will do.

BTW, I added all the rest of my points in case someone knows a good solution to this.
0
jcaldwelCommented:
You could always set a low timeout on select.

Polling with a NONBLOCK is expensive if you don't sleep. I have taken a RISC to its knees trying that.

Maybe you could add kind of a "control" fd to the select list, and send a message up the control fd to stop the block, notifying it to add the new descriptors.

Are you fork()'ing or pthread_creating new processes. If it is a fork, perhaps you could use signals.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
catbuttAuthor Commented:
"You could always set a low timeout on select. "

Well, I guess that might work, say if I have it time out after a second.  Its not perfect though, because I'd prefer be able to open a connection and have it *immediately* get any incoming messages from that connection, without having to sleep for a second.

"Polling with a NONBLOCK is expensive if you don't sleep. I have taken a RISC to its knees trying that."

Yeah I don't want to do that.

"Maybe you could add kind of a "control" fd to the select list, and send a message up the control fd to stop the block, notifying it to add the new descriptors."

This I am interested in.  How can I do that?  All I can think of is open a socket connection (to itself) that is used simply to trigger select().  

"Are you fork()'ing or pthread_creating new processes. If it is a fork, perhaps you could use signals. "

Pthreads.  Actually, this runs on Win32 as well as linux, (it runs on linux in production, but I like to have a win32 version for development and such), so in that case, windows threads.  I'd prefer any solution to work on both.
0
jcaldwelCommented:
A loopback socket might work, but may be a little more expensive than you might like.

I haven't tested this, but when you are setting up the fd_set to pass into the select function, I am certain that you could pass another fd type into the select call. You could create an unnamed pipe through the pipe() call, and use it instead. (man (2) pipe).
0
BlueTrinCommented:
That s why you put some usleep() inside the polling loop, so it is not continously checking if something was received.
0
jcaldwelCommented:
If you sleep, you have gained nothing from setting a low timeout.

I'm not sure about Linux, but USleep doesn't work on some platforms. I just got bitten by an AIX 4.3 box... usleep returned immediately.
0
catbuttAuthor Commented:
I use nanosleep with no problem.

Let me try select() with a one second timeout, that might do the job.  I guess sleeping isn't necessary -- although I'm not talking about sleeping in the thread that calls select, I'm talking about the thread that is making a new connection and then sending messages on that connection....as long as select() will still receive messages on sockets that happened *before* it was restarted to watch that socket, I'm ok.  (sorry if that's confusing :) )

I'm not sure how to do a loopback socket...that's a single socket that is "connected to itself", correct?  Can I do that without setting up a "listener" socket? If so that would be the best idea yet.

0
jcaldwelCommented:
Ah... sorry... Forgot you were a client socket only. Loopback wouldn't make a lot of sense (in my opinion) then.

If you can live with a little lag, a timeout might be the best.
0
catbuttAuthor Commented:
A one second timeout on select() seems to do what I need and its very simple and clean.

Thanks for your help!
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.