In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.
Solved
Proper way to use C# Tasks?
Posted on 2016-09-06
Hi,
I am trying to get clarification on how to use C# Tasks.
Lets say I have 10,000 objects I want to process.
Is it sensible to create 10,000 Task objects with the Task.Factory?
https://msdn.microsoft.com/en-us/library/dd321439(v=vs.110).aspx
TaskFactory.StartNew Method (Action)
Or should I be tiring to re-use a limited number?
However from the doco I am not sure how to do this. (Sure it something I have missed).
I have had some problems with things in the WaitingToRun state for longer then I would of expected.
So I am hoping to get some general guidance on all of this. (And some examples if possible).
Thanks,
I am trying to get clarification on how to use C# Tasks.
Lets say I have 10,000 objects I want to process.
Is it sensible to create 10,000 Task objects with the Task.Factory?
https://msdn.microsoft.com/en-us/library/dd321439(v=vs.110).aspx
TaskFactory.StartNew Method (Action)
Or should I be tiring to re-use a limited number?
However from the doco I am not sure how to do this. (Sure it something I have missed).
I have had some problems with things in the WaitingToRun state for longer then I would of expected.
So I am hoping to get some general guidance on all of this. (And some examples if possible).
Thanks,
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
4 Comments
Active today
It depends on the kind of tasks to perform and your machine, where you run those 10k tasks.
But generally speaking: No, its normally not a good idea.
Most for the reason that you may run you program on different machines, on capable of running only 50 tasks and another of running 500 tasks.
The normal approaches I use in such cases:
1) Use a blocking queue to store the task description. Use a single thread to spawn new threads when new task descriptions arrive and fewer then the maximum worker threads are alive.
Advantage: Each thread and its description are isolated from each other. State of threads does not matter.
Disadvantage: Needs some more time to instantiate.
Usage: When isolation is more important than speed. Like high-level jobs as print or download jobs.
2) Use a blocking queue to store the task description. Create a configurable amount of tasks. Each task takes its task description from the queue. When no more description is available, the tasks quits.
Advantage: Lesser time to instantiate.
Disadvantage: Threads keep state.
Usage: Low-level jobs like object processing for drawing graphics.
And then try to find a good number for your worker threads. E.g. for download tasks the bandwidth of your server is the bottleneck. Here even 100 or lesser tasks are optimal to balance memory and CPU consumption due to low bandwidth waits. Make this a configuration parameter in your app.config.
But generally speaking: No, its normally not a good idea.
Most for the reason that you may run you program on different machines, on capable of running only 50 tasks and another of running 500 tasks.
The normal approaches I use in such cases:
1) Use a blocking queue to store the task description. Use a single thread to spawn new threads when new task descriptions arrive and fewer then the maximum worker threads are alive.
Advantage: Each thread and its description are isolated from each other. State of threads does not matter.
Disadvantage: Needs some more time to instantiate.
Usage: When isolation is more important than speed. Like high-level jobs as print or download jobs.
2) Use a blocking queue to store the task description. Create a configurable amount of tasks. Each task takes its task description from the queue. When no more description is available, the tasks quits.
Advantage: Lesser time to instantiate.
Disadvantage: Threads keep state.
Usage: Low-level jobs like object processing for drawing graphics.
And then try to find a good number for your worker threads. E.g. for download tasks the bandwidth of your server is the bottleneck. Here even 100 or lesser tasks are optimal to balance memory and CPU consumption due to low bandwidth waits. Make this a configuration parameter in your app.config.
Active 5 days ago
Hi ste5an,
Thanks for your response. Unless I am mistaken is there not some parameter you can get from the threadpool that says how many tasks you can run and the same time. And if you run more it block you?
ThreadPool.GetMaxThreads Method (Int32, Int32) --> Should I be looking at this figure ?
Also I have got some of my code below.
Is this kind of the right way. the function fnCreate3 deques data from the fake_pc_queue.
Thanks,
Ward
Thanks for your response. Unless I am mistaken is there not some parameter you can get from the threadpool that says how many tasks you can run and the same time. And if you run more it block you?
ThreadPool.GetMaxThreads Method (Int32, Int32) --> Should I be looking at this figure ?
Also I have got some of my code below.
Is this kind of the right way. the function fnCreate3 deques data from the fake_pc_queue.
Thanks,
Ward
private void fnRun_Queue(BlockingCollection<Fake_PC> processing_list,
BlockingCollection<Fake_PC> fake_pc_results,
BlockingCollection<Fake_PC> pc_error_list,
Task_Stats my_stats,
int number_of_threads,
int iteration_count)
{
ConcurrentQueue<Fake_PC> fake_pc_queue = new ConcurrentQueue<Fake_PC>();
int total_machines = processing_list.Count;
int queue_count = 0;
int results_count = 0;
int error_count = 0;
Task[] taskArray = null;
string duration = null;
DateTime start_time = DateTime.Now;
fnUpdate_Status(string.Format("Starting iteration: {0} Records: {1}", iteration_count, processing_list.Count));
Thread.Sleep(3000);
foreach (Fake_PC my_pc in processing_list)
{
fake_pc_queue.Enqueue(my_pc);
}
Boolean execution_flag = true;
int counter = 1;
while (execution_flag == true)
{
// Task my_task = Task.Factory.StartNew(() => fnGet_Record_count());
if (queue_count > 0)
{
taskArray = new Task[number_of_threads];
for (int i = 0; i < number_of_threads; i++)
{
if (taskArray[i] == null || taskArray[i].Status == TaskStatus.RanToCompletion)
{
taskArray[i] = Task.Factory.StartNew(() => { return fnCreate3(fake_pc_queue, fake_pc_results, pc_error_list); });
}
}
}
else
{
fnUpdate_Status("Waiting on threads to finish.");
Thread.Sleep(2000);
}
Task.WaitAny(taskArray);
while (fake_pc_queue.IsEmpty == false)
{
Fake_PC my_pc;
if (fake_pc_queue.TryDequeue(out my_pc))
{
pc_error_list.Add(my_pc);
}
}
fnUpdate_Task_Status("");
}
Active today
Are you actually calling the same code for each object? Obviously just sample data you provided.
If so, any reason not to just use Parallel.For (or .ForEach)?
You don't have as much lower level control, but does a very good job (in my experience) managing the different threads and tasks and splitting them out as needed across processors.
And is dirt simple to implement, with far less code.
If so, any reason not to just use Parallel.For (or .ForEach)?
You don't have as much lower level control, but does a very good job (in my experience) managing the different threads and tasks and splitting them out as needed across processors.
And is dirt simple to implement, with far less code.
Featured Post
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
A long time ago (May 2011), I have written an article showing you how to create a DLL using Visual Studio 2005 to be hosted in SQL Server 2005. That was valid at that time and it is still valid if you are still using these versions. You can still re…
Exception Handling is in the core of any application that is able to dignify its name.
In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash"
After Shake : http://www.videocopilot.net/presets/after_shake/
All lightning effects with instructions : http://www.mediaf…
Sometimes it takes a new vantage point, apart from our everyday security practices, to truly see our Active Directory (AD) vulnerabilities. We get used to implementing the same techniques and checking the same areas for a breach. This pattern can re…
Suggested Courses
Course of the Month10 days, 17 hours left to enroll
719 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.