Solved

Parallel.ForEach using SendMailAsync() detect when loop is complete

Posted on 2016-11-21
24
29 Views
Last Modified: 2016-11-22
Hi Guys,

I am trying to know when a parallel.foreach is complete using tasks etc. I know this isn't easily possible as it runs on mulitple threads but I am sending emails using SendMailAsync and want to enable a button using MVVM property when the loop is complete.

Currently my code after the parallel for each is run before the loop is finished. With a standard foreach() it works as expected but I want to speed things up with parallel.foreach?

Does anybody know how I can do this?

HEre is my code:

    Public Function SendEmail(email As String, bodystuff As String) As Task


        Dim smtp As New SmtpClient("webmail.databarracks.com")
        Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
        Dim [to] As New MailAddress(email)
        Dim message As New MailMessage(from, [to])
        message.Body = "The message I want to send is to this <b>contact: " & vbCrLf & bodystuff & "</b>"
        message.IsBodyHtml = True
        message.BodyEncoding = System.Text.Encoding.UTF8
        message.Subject = "The subject of the email"
        message.SubjectEncoding = System.Text.Encoding.UTF8

        ' Set the method that is called back when the send operation ends.
        'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted



        Return smtp.SendMailAsync(message)

    End Function


    Public Sub StartEmailRun()

        Try

            Dim sWatch As New System.Diagnostics.Stopwatch()

            sWatch.Start()

            Dim po As New ParallelOptions()
            'Create a cancellation token so you can cancel the task.
            _cancelToken = New CancellationTokenSource()
            po.CancellationToken = _cancelToken.Token
            'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
            po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
            Dim v As Long = 0


            Parallel.ForEach(contacts.Cast(Of Object), po,
                            Function(row)

                                Return SendEmail(row.Email.ToString, row.Name.ToString)

                            End Function)


            'Here I would like to update my property to True when the above loop is complete??
            MyProperty = True


            sWatch.Stop()
            Timing = sWatch.ElapsedMilliseconds

        Catch ex As Exception
            MsgBox(ex.ToString)
        End Try




    End Sub

Open in new window


P.S - C# code is welcome
0
Comment
Question by:databarracks
  • 12
  • 6
  • 4
  • +1
24 Comments
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41895881
Will you actually speed anything up (with async) if you have to wait for it to complete before doing the next thing?  I doubt it will.
0
 

Author Comment

by:databarracks
ID: 41895889
I do see what you mean, but I am not looking to await within the loop just when the loopresult is completed? When I run my standard loop it takes double the time.

I am probably out of my depth but would you have any other recommendations to send bulk emails? Because I cannot even report back progress with how many emails have been sent using parallel and interlocked.increment either?

Would you be able to recommend an alternative?
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41895905
Simplest is probably to let another thread (or app - process.start) send the emails and not bother to report any progress.  
Unless you have coded an email client (or have one that you can interrogate programatically) you won't know if a send had worked or failed.  All you know is that it was instructed to send.
0
 

Author Comment

by:databarracks
ID: 41895908
Ok I am aware of that so would you just say I stick to a standard foreach loop with a SendMailAsync and not bother with parallel.foreach?
0
 
LVL 32

Expert Comment

by:it_saige
ID: 41895935
Take a look at Task.WaitAll.

-saige-
0
 

Author Comment

by:databarracks
ID: 41895943
it_saige that is interesting I have seen that before, just figuring out how I can use that with the parallel.foreach?
0
 
LVL 32

Expert Comment

by:it_saige
ID: 41895968
Parrallel.ForEach would be used to create a list (or array) of tasks.  Each task is the code used to process the smtp send request.  After you build your list (or array) of tasks, you would use the Task.WaitAll in order to wait for all of the tasks to complete; e.g. -
ReadOnly tasks As New List(Of Task)

Public Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Return smtp.SendMailAsync(message)
End Function

Public Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					 Sub(row)
						 tasks.Add(SendEmail(row.Email.ToString, row.Name.ToString))
					 End Sub)

		Task.WaitAll(tasks.ToArray())

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds

	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window


-saige-
0
 

Author Comment

by:databarracks
ID: 41896023
Hi it_saige,

Thanks for the code snippet but would you happen to know why I can only manage to send if there is only one item in the list. Basically it appears as though it isn't actually looping through records?
0
 
LVL 32

Accepted Solution

by:
it_saige earned 500 total points
ID: 41896168
Try using WhenAll:
Public Async Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					 Sub(row)
						 tasks.Add(SendEmail(row.Email.ToString, row.Name.ToString))
					 End Sub)

		Await Task.WhenAll(tasks)

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds

	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window


-saige-
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41896183
You still seem to want to wait until all the sends have finished.  I suspect you are going to end up with next to no performance advantage - maybe even taking longer because of the overhead of the tasks - when compared to the simple for loop and SendMailAsync.  The .net environment ought to pass the load through the available processors.
0
 

Author Comment

by:databarracks
ID: 41896192
Ok it_saige, your new workaround works correctly:)

Andy, I have tested the new method by it_saige and there appears to a performance advantage after all?? Maybe I am seeing things but the stopwatch seems to be reporting a much shorter time to run it_saige's method.
0
 
LVL 32

Expert Comment

by:it_saige
ID: 41896209
This makes me wonder though if the answer could have simply been adding the associated Async and Await keywords into your SendEmail function instead; e.g. -
Public Async Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Await smtp.SendMailAsync(message)
End Function

Public Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					  Function(row)
						  Return SendEmail(row.Email.ToString, row.Name.ToString)
					  End Function)

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds
	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window

Especially since the Parallel class essentially wraps each loop iteration into it's own task.

-saige-
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Author Comment

by:databarracks
ID: 41896219
Hi,

I tried that before I submitted this question. The problem with that approach is that it would run the _isCompleted = True before it actually has looped through all records.
0
 

Author Comment

by:databarracks
ID: 41896235
Your previous approach is correct, also  because if I change the SendEmail to:

Public Async Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Await smtp.SendMailAsync(message)
End Function

Open in new window


And used this:

Public Async Sub StartEmailRun()
	Try
		Dim sWatch As New Stopwatch()

		sWatch.Start()

		Dim po As New ParallelOptions()
		'Create a cancellation token so you can cancel the task.
		_cancelToken = New CancellationTokenSource()
		po.CancellationToken = _cancelToken.Token
		'Manage the MaxDegreeOfParallelism instead of .NET Managing this. We dont need 500 threads spawning for this.
		po.MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
		Dim v As Long = 0

		Parallel.ForEach(_contacts.Cast(Of Object), po,
					 Sub(row)
						 tasks.Add(SendEmail(row.Email.ToString, row.Name.ToString))
					 End Sub)

		Await Task.WhenAll(tasks)

		'Here I would like to update my property to True when the above loop is complete??
		_isCompleted = True

		sWatch.Stop()
		_timing = sWatch.ElapsedMilliseconds

	Catch ex As Exception
		MsgBox(ex.ToString)
	End Try
End Sub

Open in new window


I can now increment a property...call it totalsent inside the SendEmail like so:

Public Async Function SendEmail(email As String, bodystuff As String) As Task
	Dim smtp As New SmtpClient("webmail.databarracks.com")
	Dim from As New MailAddress("info@databarracks.com", "Info", System.Text.Encoding.UTF8)
	Dim [to] As New MailAddress(email)
	Dim message As New MailMessage(from, [to])
	message.Body = String.Format("The message I want to send is to this <b>contact: {0}{1}</b>", vbCrLf, bodystuff)
	message.IsBodyHtml = True
	message.BodyEncoding = System.Text.Encoding.UTF8
	message.Subject = "The subject of the email"
	message.SubjectEncoding = System.Text.Encoding.UTF8

	' Set the method that is called back when the send operation ends.
	'AddHandler smtp.SendCompleted, AddressOf smtpClient_SendCompleted

	Await smtp.SendMailAsync(message)
TotalSent = TotalSent + 1
End Function

Open in new window


And seems to be stable in incrementing?
0
 

Author Comment

by:databarracks
ID: 41896242
I just compared the two different approaches.

The standard loop with SendMail Async took 5.42 seconds

The parallel foreach using your method took 2.3 seconds
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41896263
Interesting.  With that result it looks like the Async doesn't in fact use all the processors available.
0
 

Author Comment

by:databarracks
ID: 41896282
Indeed
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 41896491
With that result it looks like the Async doesn't in fact use all the processors available.
Async isn't about using more processors--that's parallel processing. Your async code might all execute on one processor, or it might get distributed across all processors. You don't control this***. Async is about not waiting on external resources when other work can be done. You can send off multiple network requests asynchronously, and you won't block on processing that is happening on the external server (e.g. the SMTP server). But you still have to process n results. Those n results will trickle in as they finish--the OS/hardware will dictate this. Async doesn't imply that these n requests process any faster; it only implies that you won't be blocked waiting on n network requests to finish.

*** I feel like I read somewhere that maybe you can control thread affinity for the tasks that are generated by async code, but such a use case will be rare in general usage, I think.
0
 

Author Comment

by:databarracks
ID: 41897075
käµfm³d 👽 thanks for the comment very good to know:)
0
 

Author Closing Comment

by:databarracks
ID: 41897076
Awesome stuff by it_saige as usual
0
 

Author Comment

by:databarracks
ID: 41897078
Also thanks to Andy and käµfm³d 👽
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41897112
@Kaufmed,
My understanding was that an async method would use other processors if available wheras a parallel method tells the framework to use other processors preferentially should they be available. Both would allow the code in the main thread to continue (non blocking).
0
 
LVL 74

Expert Comment

by:käµfm³d 👽
ID: 41897424
@Andy

Async methods register continuations which get resolved when the asynchronous operations finishes. A continuation is basically a way for the runtime to say, "Take this code here, and shelve it for the moment. I'll let you know when to run this as I am waiting on something else to complete first." The OS uses an I/O completion port to signal that the operation is finished. So while your network or file processing request is ongoing, your processor isn't doing anything. It's I/O work:  there's nothing for the processor to do.

Async/await is for I/O-bound processing; threads are for CPU-bound processing.
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 41897460
@kaufmed.
Interesting link.  If I understand it correctly the call to the CreateIoCompletionPort may result in the framework requesting all processors be used to spread the load over many threads - depends on just how the Async method sets things up in the background.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

It’s quite interesting for me as I worked with Excel using vb.net for some time. Here are some topics which I know want to share with others whom this might help. First of all if you are working with Excel then you need to Download the Following …
Introduction Hi all and welcome to my first article on Experts Exchange. A while ago, someone asked me if i could do some tutorials on object oriented programming. I decided to do them on C#. Now you may ask me, why's that? Well, one of the re…
It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now