c# software limits / server processing limits
Hi
We are facing some issues with performance, we cannot seem to process data fast enough.
Firstly what I would need to know is what we are trying is technically possible
I'm trying to process hundreds of thousands of records
each record takes approx 500ms to process due to things like hitting external API's
I'm only able to process 85-100 records per second
The server is in a datacenter, its am 8 cpu skylake cpu, however when the app runs it uses approx 140 threads, and cpu sits around 6%
The app is multithreaded and i've tried threads, threadpool, Async/await, parallel foreach/invoke etc, most of the 500ms is waiting for API's to respond.
My question is related to processing throughput with the given parameters
i.e. if it takes 500ms to process 1 record, (and its running consistently on 140 threads), does 100 records per second so like a near limit?
I know this question is tough to answer but I need to know is i should be focusing on code performance tuning or look at other solutions
We are facing some issues with performance, we cannot seem to process data fast enough.
Firstly what I would need to know is what we are trying is technically possible
I'm trying to process hundreds of thousands of records
each record takes approx 500ms to process due to things like hitting external API's
I'm only able to process 85-100 records per second
The server is in a datacenter, its am 8 cpu skylake cpu, however when the app runs it uses approx 140 threads, and cpu sits around 6%
The app is multithreaded and i've tried threads, threadpool, Async/await, parallel foreach/invoke etc, most of the 500ms is waiting for API's to respond.
My question is related to processing throughput with the given parameters
i.e. if it takes 500ms to process 1 record, (and its running consistently on 140 threads), does 100 records per second so like a near limit?
I know this question is tough to answer but I need to know is i should be focusing on code performance tuning or look at other solutions
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
David Johnson, CD
and the api is waiting for what? disk i/o ?
ASKER
Api is external service which returns a string of data, I call over http
I think the bigger point that David and I were both trying to make is that if the API is a bottleneck - is it one that you have control over or can improve on usage?
ASKER
Thanks, I did think the same, and is why I mentioned it, wasn't sure if there was some magic work area
It appears that the threads are all in use while these requests are waiting for response and not available to the rest of the system, so it's reaching a limit (approx 140 threads at any one point)
The http api is open source, I have it on my own server but its being access via public IP in same data centre (different hosts)
I intended to bring it onto a Lan, and access it this was to see if it will speed up things significantly
There seems to be a significant ttfb (time to first Byte) when checking the same request via chrome Dev tool bar
We have seen it at 300ms, but not much faster than this
The http api sends a lat lon into the api, and returns a road name.
It appears that the threads are all in use while these requests are waiting for response and not available to the rest of the system, so it's reaching a limit (approx 140 threads at any one point)
The http api is open source, I have it on my own server but its being access via public IP in same data centre (different hosts)
I intended to bring it onto a Lan, and access it this was to see if it will speed up things significantly
There seems to be a significant ttfb (time to first Byte) when checking the same request via chrome Dev tool bar
We have seen it at 300ms, but not much faster than this
The http api sends a lat lon into the api, and returns a road name.
So it's a reverse geocoding API? There might not be too much you can do unless you can load the database onto a ramdisk or something. Usually with geocoding (regardless of direction), there's a lot of data and it just takes a while to look up, so I/O can be the biggest bottleneck. It also sounds like it's a 3rd party / open source API, which might constrain the ability for you to make significant modifications to it unless you want to really dive into it.
ASKER
yes its reverse
I'll take a look at the Source / see if i can connect directly to the DB
I'll take a look at the Source / see if i can connect directly to the DB
ASKER
Thanks, this had led me to investigate much more, i'll close this question here now as i've got a separate question with a lot more detail including an environment to test, and source code.
https://www.experts-exchange.com/questions/29145595/Increase-concurrent-requests-to-API-from-one-machine.html
It may result in the same answer as here, but its a very important area so I need to be 100% sure how to proceed.
I would appreciate if you could participate on the other question,
https://www.experts-exchange.com/questions/29145595/Increase-concurrent-requests-to-API-from-one-machine.html
It may result in the same answer as here, but its a very important area so I need to be 100% sure how to proceed.
I would appreciate if you could participate on the other question,