We are facing some issues with performance, we cannot seem to process data fast enough.
Firstly what I would need to know is what we are trying is technically possible
I'm trying to process hundreds of thousands of records
each record takes approx 500ms to process due to things like hitting external API's
I'm only able to process 85-100 records per second
The server is in a datacenter, its am 8 cpu skylake cpu, however when the app runs it uses approx 140 threads, and cpu sits around 6%
The app is multithreaded and i've tried threads, threadpool, Async/await, parallel foreach/invoke etc, most of the 500ms is waiting for API's to respond.
My question is related to processing throughput with the given parameters
i.e. if it takes 500ms to process 1 record, (and its running consistently on 140 threads), does 100 records per second so like a near limit?
I know this question is tough to answer but I need to know is i should be focusing on code performance tuning or look at other solutions