Indy Client Server freezing

I would highly recommend moving back to Indy 9 for all around stability. When using Indy 10 I have come across some problems that switching back never had.

Hi Johnjces.

Can server and client use two different ports for sending and receiving? Didn't know that, I thought each connection was binded to one specific port. Could you please send me an example? Thanks.

Hey ThievingSix, thanks for your comment.

Yes, in fact I have thought about going back to Indy 9 times, but I 'd rather prefer to stay in indy 10 as long as possible. I may want to ensure unicode compliance to my project later on or even move it to Delphi 2009 (now I am using Delphi 2006). In my opinion, going back to indy 9 (which I think is being barely maintained right now) could throw me into more problems in the short term.

Try it! You have separate client and server components and it can work.

Hi Johnjces, I talked about using the same TIdTCPServer and TIdTCPClient, of course I know they are separate components ;).

The point is that I do not want to use more than one TIdTCPClient and TIdTCPServer in each module, that would be the last option. I need to keep the project as simple as possible, experience has showed me that debugging multithreaded projects can be absolutely discouraging and time consuming. Two different connections on both sides means lots of extra code: paired connection and disconnection events, two different timers to reconnect sockets in case of failure, two synchronized server threads for each client connection besides the main VCL thread... There's too much code attached to one single connection to think in handle two of them per client, at least at the moment. If there would be a way of using two ports in one connection that would be fine, but I understand that this is not possible, by definition. Thanks anyway for your suggestion.

What I really need to know is why does a blocking socket get data in the client side which does not come out in the server side.

I am working on a simplified project which will show the problem. Tomorrow I will have it ready. If you need the code, just ask me.

Hi experts. following the principle: no news = bad news, I have just finsihed a simple client-server version where the freezing arises. The problem is that EE won't allow me to upload the zip archive with the project files, and even though the project is small, the code is too long to fit as a comment.

How can I publish the code in a civilized manner? Didn't find the answer in EE help or FAQs. Thanks

use http://www.ee-stuff.com
you can put EE attachments there, and it will make a link that you can paste here
I do it all the time
If you are unable to do that, then email it to me (my address is in my profile) and I will put it on EE stuff for you, for all to see. This should still be within the EE rules.

As for your problem, I have various demos that do what you are asking, but we will modify your code if that is what you prefer.
FWIW, sending the "length of the stream", followed by the stream, is the way to go (imho)

Even though indy uses blocking sockets, TCP is still a slightly random beast
hardware and the like can change things...
not like UDP, but for example, you may send 1 "packet", but it may arrive as several packets, and even have large delays between packets.
Even though Indy 10 is a big change from Indy 9, I can only re-iterate what Chad Z Hower (author of Indy) told me.
I was having problems with Indy 10, and he suggested I download the latest "development snapshot", not the official release. This was a while ago now, but it certainly contained many fixes to the official release.
Give that a go.
Looking forward to seeing your code...

oh, and please tell me what version of Delphi you are using. That will make a big difference to us.

Hi experts. I just uploaded the project files so you can test them and see the problem. I have tried to make it easy to test: connection can be done on any port (it defaults to 9997), and to any host name, without having to retype the code. I have translated screen messages and made an effort to remove unimportant things. You should try to send a big file over the lan (I use a 300 Mb file), because it may not freeze with small ones.

http://www.ee-stuff.com/Expert/Upload/getFile.php?fid=7612

Thanks

Hi TheRealLoki, thanks for your tips. I received an administrative comment telling me how to upload files to www.ee-stuff.com, so I have just uploaded them. Thanks for your offer, anyway.

As you say, I would rather prefer you having a look at my code. It could be that the origin of my problem was in a particular way I am using threads or indy components, so I prefer to get solutions which will apply straight to my project.

I do use the "length of stream followed by the stream", approach, to send the chinks. In fact I have tried several different ways, but this is the one that remains.

About having the latest snapshot available, I did install tortoise svn client to download latest snapshot of indy 10 available on the trunk folder (following instructions found in indy project's page). This was three weeks ago, more or less. If you can run the project without freezing, I assure you that I will download the latest snapshot straight away.

I am using Delphi 2006 Professional, BTW.

Hi there, TheRealLoki. While you were looking at the code I downloaded the latest snapshot from Indy 10 and installed it (after uninstalling current version completely). The problem remains the same, it still freezes. If you cannot find the time to look at my code, please send me one of the working examples you mentioned above of sending files in chunks using indy10, so I can compare them with my project. I will add differences one by one to see what is that makes it crash. I am really stucked now with this issue. Thanks.

sorry, I've been off work, so havent checked EE.
It's probably a thread issue, but I''ll see if I can reproduce it on my home dev pc
I'm concerned about the way you use the TIdContext.
Give me a few hours to try and get my dev pc up :-)

Hi  Loki,

I downloaded Indy 10 client and server and I'm trying to compile them. In both server and client I get some strange "Incompatbile types integer and Int64" compiler errors, in the onConnect event, where you assign OnWork, and OnWorkBegin connection events. I am looking at it at the moment.

When I get to compile your projects I will try to apply differences with my project, one by one, so to see which one will lock things up. First things I see different:

- You create the TIdTCPclient at runtime
- You send the file in a single write instruction.
- TCP dialog is started by a client's command
- You use Synchronize in the client, to call VCL functions from the thread (I use windows messaging, like you do in the server).

Maybe I left someone out, it's just a first inspection.

I will tell you how I get on with this. If you discover the origin of the freezing problem meanwhile, please tell me.

Thanks.

Hi Loki. First of all, my apologies, i could not work on your demos until this morning. We had a 4 day weekend here.

It looks like I am close to isolating the problem, now.
Using your freezing-free client/server demo as a basis, I began to add changes pasting pieces of my code into your project.

1) Made server send the file to the client in the OnConnect event: Worked OK
2) Sent the file in chunks: OK
3) Added a button in the server to send the file at any time, using a custom created thread: FREEZED!!!

So, the problem arises when I try to use my own custom thread to send the file from the server, instead of using the thread spawned by the TIdTCPServer. The problem is that, as sending the file to the client will be the user's decission, I must be able to send the file at any time and I cannot do it without spawning a thread outside indy's. Also, I do not want to freeze users interface with lengthy file operations, so I cannot use the main VCL thread either.

The custom thread is the most simple thread I can imagine, I just create it, resume, and it does not even have a execution loop (I wanted to make it as low CPU time consuming as possible), just sends the file and terminates. The code to send the file is the same thread-safe procedure I use from the onconnect event.

I have uploaded the project to ee-stuff. You will see a checkbox and a button. With the checkbox enabled, it sends the file as soon as the client connects using indy thread (works OK). If you use the button, it creates a new thread and sends the file (Freezes).

If you see where is the problem, tell me. I am working in it right now.

Thanks.

I have been trying around things and this is what I have seen:

1) If I use threads spawned by TIdTCPServer in the OnConnect or the OnExecute events, the file is sent OK. If I use OnExecute, I can activate a boolean flag in the main form when I click a button and test it inside the event. This is a partial solution because I can respond to a user action to send the file.
2) Whenever I use a custom thread it might interfere with indy's own listener threads and at some point my thread gets blocked.

Using the onexecute thread implies that the onexecute event won't be triggered again for any other client while the file is being sent? This would be an important drawback, as the server can have several connected clients, all of them blocked until the operation with a single client is finished.

I think that the neatest solution would be using my own thread, but at this point I don't know what else I can do. I already tried different thread's priorities, changed the way I store the IdContext, tested TIdThread instead of TThread... with no results.

Any ideas?

Sorry for the delay. I have been testing the demos...

First of all, you're absolutely right about OnExecute triggering. It triggers for each client independently, I made a bad reading of documentation. This helps me greatly because I can use Indy's own threads to control de TCP flow in a safe manner. I guess I will have to retype the code to move all TCP commands inside the OnExecute thread in the main form. I tested PostThreadMessage to notify server's thread about starting operations, and it works fine.

It would be great if I could just understand why using a different thread to access TIdTCPServer connection ends up in unpredictably freezing. I made it all really simple: I protect socket access with critical sections and it should be all thread-safe! Could it be a bug somewhere in indy's thread management? I couldn't find any example in the web which used a custom thread to read/write to a server. I even found a post somewhere that recommended (without further explanations) to use indy connections always from the same thread spawned in the OnConnect event, even though synchronizing objects could be used to protect access from other threads.

If no one comes with an explanation to this problem I will award you the points. Your demo code was useful to clarify things.

Indy uses a TidThread which has a few extra methods. I expect using your own TThread means that you can't inherently call some of the TiDThread's OnExecute code which probably handles some other syncing that isn't obvious before the published Execute method.
e.g.
Indy "execute"
 "do indy stuff"
 if assigned(fOnExecute) then fOnExecute(self, AThread);
 "do other indy stuff"
end;

I still do a lot of guess work when it comes to Indy

I have done inter thread communication on Indy using TCriticalSection, but it probably wouldn't have been a good match for your situation.

Yes, that makes sense. I even tried with my own TIdThread, just in case the origin of the problem was on synchronizing between a TThread and a TIdThread, but it didn't work either.

As I said, I award you the point for your response with the demo. Thanks for your help, guys.

Hi Praxalia,

i think I have the same problem as you did.
Did you ever solved this?

Thanks!

Hi DKropf.

Yes I did solve the freezing issue. What I discovered is that, for some reason, TIdTCPServer socket is only safe  to use inside the Onexecute thread of the TIdTCPServer. My problem arosed from the fact that I called socket operations (read and write) from my own custom thread. The program freezed randomly at these I/O operations no matter which blocking system I used (semaphores, Critical sections, signaled events...) to avoid conflicts with Indy's own thread. At last, everything worked fine when I moved all reading/writing to the socket inside the OnExecute loop.

Hope it helps,

praxalia