Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Parallel.ForEach using SendMailAsync() detect when loop is complete

Posted on 2016-11-21
24
Medium Priority
?
413 Views
Last Modified: 2017-01-17
Hi Guys,

I am trying to know when a parallel.foreach is complete using tasks etc. I know this isn't easily possible as it runs on mulitple threads but I am sending emails using SendMailAsync and want to enable a button using MVVM property when the loop is complete.

Currently my code after the parallel for each is run before the loop is finished. With a standard foreach() it works as expected but I want to speed things up with parallel.foreach?

Does anybody know how I can do this?

HEre is my code:

    Public Function SendEmail(email As String, bodystuff As String) As Task


        Dim smtp As New SmtpClient("webmail.databarracks.com")
        Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
        Dim [to] As New MailAddress(email)
        Dim message As New MailMessage(from, [to])
        message.Body = "The message I want to send is to this <b>contact: " & vbCrLf & bodystuff & "</b>"
        message.IsBodyHtml = True
        message.BodyEncoding = System.Text.Encoding.UTF8
        message.Subject = "The subject of the email"
        message.SubjectEncoding = System.Text.Encoding.UTF8

        ' Set the method that is called back when the send operation ends.
        'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted



        Return smtp.SendMailAsync(message)

    End Function


    Public Sub StartEmailRun()

        Try

            Dim sWatch As New System.Diagnostics.Stopwatch()

            sWatch.Start()

            Dim po As New ParallelOptions()
            'Create a cancellation token so you can cancel the task.
            _cancelToken = New CancellationTokenSource()
            po.CancellationToken = _cancelToken.Token
            'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
            po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
            Dim v As Long = 0


            Parallel.ForEach(contacts.Cast(Of Object), po,
                            Function(row)

                                Return SendEmail(row.Email.ToString, row.Name.ToString)

                            End Function)


            'Here I would like to update my property to True when the above loop is complete??
            MyProperty = True


            sWatch.Stop()
            Timing = sWatch.ElapsedMilliseconds

        Catch ex As Exception
            MsgBox(ex.ToString)
        End Try




    End Sub

Open in new window


P.S - C# code is welcome
0
Comment
Question by:databarracks
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 6
  • 4
  • +1
24 Comments
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41895881
Will you actually speed anything up (with async) if you have to wait for it to complete before doing the next thing?  I doubt it will.
0
 

Author Comment

by:databarracks
ID: 41895889
I do see what you mean, but I am not looking to await within the loop just when the loopresult is completed? When I run my standard loop it takes double the time.

I am probably out of my depth but would you have any other recommendations to send bulk emails? Because I cannot even report back progress with how many emails have been sent using parallel and interlocked.increment either?

Would you be able to recommend an alternative?
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41895905
Simplest is probably to let another thread (or app - process.start) send the emails and not bother to report any progress.  
Unless you have coded an email client (or have one that you can interrogate programatically) you won't know if a send had worked or failed.  All you know is that it was instructed to send.
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 

Author Comment

by:databarracks
ID: 41895908
Ok I am aware of that so would you just say I stick to a standard foreach loop with a SendMailAsync and not bother with parallel.foreach?
0
 
LVL 34

Expert Comment

by:it_saige
ID: 41895935
Take a look at Task.WaitAll.

-saige-
0
 

Author Comment

by:databarracks
ID: 41895943
it_saige that is interesting I have seen that before, just figuring out how I can use that with the parallel.foreach?
0
 
LVL 34

Expert Comment

by:it_saige
ID: 41895968
Parrallel.ForEach would be used to create a list (or array) of tasks.  Each task is the code used to process the smtp send request.  After you build your list (or array) of tasks, you would use the Task.WaitAll in order to wait for all of the tasks to complete; e.g. -
ReadOnly tasks As New List(Of Task)

Public Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Return smtp.SendMailAsync(message)
End Function

Public Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					 Sub(row)
						 tasks.Add(SendEmail(row.Email.ToString, row.Name.ToString))
					 End Sub)

		Task.WaitAll(tasks.ToArray())

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds

	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window


-saige-
0
 

Author Comment

by:databarracks
ID: 41896023
Hi it_saige,

Thanks for the code snippet but would you happen to know why I can only manage to send if there is only one item in the list. Basically it appears as though it isn't actually looping through records?
0
 
LVL 34

Accepted Solution

by:
it_saige earned 2000 total points
ID: 41896168
Try using WhenAll:
Public Async Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					 Sub(row)
						 tasks.Add(SendEmail(row.Email.ToString, row.Name.ToString))
					 End Sub)

		Await Task.WhenAll(tasks)

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds

	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window


-saige-
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41896183
You still seem to want to wait until all the sends have finished.  I suspect you are going to end up with next to no performance advantage - maybe even taking longer because of the overhead of the tasks - when compared to the simple for loop and SendMailAsync.  The .net environment ought to pass the load through the available processors.
0
 

Author Comment

by:databarracks
ID: 41896192
Ok it_saige, your new workaround works correctly:)

Andy, I have tested the new method by it_saige and there appears to a performance advantage after all?? Maybe I am seeing things but the stopwatch seems to be reporting a much shorter time to run it_saige's method.
0
 
LVL 34

Expert Comment

by:it_saige
ID: 41896209
This makes me wonder though if the answer could have simply been adding the associated Async and Await keywords into your SendEmail function instead; e.g. -
Public Async Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Await smtp.SendMailAsync(message)
End Function

Public Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					  Function(row)
						  Return SendEmail(row.Email.ToString, row.Name.ToString)
					  End Function)

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds
	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window

Especially since the Parallel class essentially wraps each loop iteration into it's own task.

-saige-
0
 

Author Comment

by:databarracks
ID: 41896219
Hi,

I tried that before I submitted this question. The problem with that approach is that it would run the _isCompleted = True before it actually has looped through all records.
0
 

Author Comment

by:databarracks
ID: 41896235
Your previous approach is correct, also  because if I change the SendEmail to:

Public Async Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Await smtp.SendMailAsync(message)
End Function

Open in new window


And used this:

Public Async Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					 Sub(row)
						 tasks.Add(SendEmail(row.Email.ToString, row.Name.ToString))
					 End Sub)

		Await Task.WhenAll(tasks)

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds

	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window


I can now increment a property...call it totalsent inside the SendEmail like so:

Public Async Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Await smtp.SendMailAsync(message)
TotalSent = TotalSent + 1
End Function

Open in new window


And seems to be stable in incrementing?
0
 

Author Comment

by:databarracks
ID: 41896242
I just compared the two different approaches.

The standard loop with SendMail Async took 5.42 seconds

The parallel foreach using your method took 2.3 seconds
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41896263
Interesting.  With that result it looks like the Async doesn't in fact use all the processors available.
0
 

Author Comment

by:databarracks
ID: 41896282
Indeed
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 41896491
With that result it looks like the Async doesn't in fact use all the processors available.
Async isn't about using more processors--that's parallel processing. Your async code might all execute on one processor, or it might get distributed across all processors. You don't control this***. Async is about not waiting on external resources when other work can be done. You can send off multiple network requests asynchronously, and you won't block on processing that is happening on the external server (e.g. the SMTP server). But you still have to process n results. Those n results will trickle in as they finish--the OS/hardware will dictate this. Async doesn't imply that these n requests process any faster; it only implies that you won't be blocked waiting on n network requests to finish.

*** I feel like I read somewhere that maybe you can control thread affinity for the tasks that are generated by async code, but such a use case will be rare in general usage, I think.
0
 

Author Comment

by:databarracks
ID: 41897075
käµfm³d 👽 thanks for the comment very good to know:)
0
 

Author Closing Comment

by:databarracks
ID: 41897076
Awesome stuff by it_saige as usual
0
 

Author Comment

by:databarracks
ID: 41897078
Also thanks to Andy and käµfm³d 👽
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41897112
@Kaufmed,
My understanding was that an async method would use other processors if available wheras a parallel method tells the framework to use other processors preferentially should they be available. Both would allow the code in the main thread to continue (non blocking).
0
 
LVL 75

Expert Comment

by:käµfm³d 👽
ID: 41897424
@Andy

Async methods register continuations which get resolved when the asynchronous operations finishes. A continuation is basically a way for the runtime to say, "Take this code here, and shelve it for the moment. I'll let you know when to run this as I am waiting on something else to complete first." The OS uses an I/O completion port to signal that the operation is finished. So while your network or file processing request is ongoing, your processor isn't doing anything. It's I/O work:  there's nothing for the processor to do.

Async/await is for I/O-bound processing; threads are for CPU-bound processing.
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41897460
@kaufmed.
Interesting link.  If I understand it correctly the call to the CreateIoCompletionPort may result in the framework requesting all processors be used to spread the load over many threads - depends on just how the Async method sets things up in the background.
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

We all know that functional code is the leg that any good program stands on when it comes right down to it, however, if your program lacks a good user interface your product may not have the appeal needed to keep your customers happy. This issue can…
Introduction Hi all and welcome to my first article on Experts Exchange. A while ago, someone asked me if i could do some tutorials on object oriented programming. I decided to do them on C#. Now you may ask me, why's that? Well, one of the re…
In this video, Percona Solution Engineer Dimitri Vanoverbeke discusses why you want to use at least three nodes in a database cluster. To discuss how Percona Consulting can help with your design and architecture needs for your database and infras…
Want to learn how to record your desktop screen without having to use an outside camera. Click on this video and learn how to use the cool google extension called "Screencastify"! Step 1: Open a new google tab Step 2: Go to the left hand upper corn…
Suggested Courses

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question