Delete File after Download

I generate a JPEG from a Graphic, and then download the graphic, using:

Response.Clear()
Response.ContentType = "application/octet-stream"
Response.AddHeader("Content-Disposition", _
  "attachment; filename=""" & varFileName & """")
Response.Flush()
Response.WriteFile(varFilePath & varFileName)

This download works fine.  Now I would like to delete the workfile at varFilePath and varFileName, so I do this:

System.IO.File.Delete(varFilePath & varFileName)

The delete works, except that it causes the "open/save" dialog to not appear, as if it were deleting the file before the
response can get to it.   Take out the delete line, response then works again.

What might I do to automatically delete the work file, ensuring that this occurs after the "open/save" dialog  of the download as been completed or cancelled?

Thanks!