System.IO.File.Move Error Access to the path is denied.

On my Windows service written in C#, there is a method that makes an app domain and then loads some assemblies for that app domain (all of these are inside a thread).
Im trying to add the following functionality:
1.      Check to see if assemblies are loaded?
2.      If yes then unload the app domain
3.      Create a directory using System.IO.Directory.CreateDirectory
4.      Back up DLL assemblies into the backup directory using System.IO.File.Move
5.      Copy over the new DLL assemblies using the System.IO.File.Move
6.      Reload the app domain

Only step 4 is not working and Im getting the Access to the path is denied. in the Application event where I log the exceptions.
I cannot rename, delete, or move the DLL assemblies.
I used the sysinternal procexp to make sure when I try to move the DLLs they are not using by anyone and still I cannot move them.

I wonder if anyone knows the right way of checking the securities and permissions so this can be done (I mean programmatically).
In other words, if you were I, how do you write your code?
Here is mine (please copy and paste into VS, the text format is changed here):

Assembly[] allAssemblies;
                              allAssemblies = appDomain.GetAssemblies();
                              // Check if a PARSER_*.dll assembly is loaded
                              foreach (Assembly assemblyFile in allAssemblies)
                              {
                                    string assemblyName = assemblyFile.FullName;
                                    if (assemblyName.Contains("PARSER"))
                                          isParserModuleLoaded = true;
                              }
                              // If parser assembly is already loaded then unload the app domain CopilotParserController
                              if (isParserModuleLoaded)
                                    AppDomain.Unload(appDomain);

                              System.IO.DirectoryInfo sourceDir = new DirectoryInfo(Application.StartupPath + "\\PhoneHomeServicesUpdate");
                              System.IO.Directory.CreateDirectory(Application.StartupPath + "\\PARSER Modules Backup");
                              System.IO.DirectoryInfo backupDir = new DirectoryInfo(Application.StartupPath + "\\PARSER Modules Backup");
                              FileInfo[] destinFileList = dir.GetFiles("PARSER_*.*");
                              // Make backup
                              foreach (FileInfo destinFile in destinFileList)
                                    try
                                    {
                                          CommonLoggingProvider.LogMessageDebug("Make backup to {0}", backupDir.FullName);
                                          System.IO.File.Move(destinFile.FullName, backupDir.FullName + "\\" + destinFile.Name);
                                    }
                                    catch (Exception ex)
                                    {
                                          CPDataService.stats.Errors++;
                                          CommonLoggingProvider.LogEventWarning(ex.Message);
                                          ErrorReporter.ReportException(ex, "Copilot Data Service <CPDataService.ExecuteParsers> exception");
                                    }