Editor's Choice: This article has been selected by our editors as an exceptional contribution.

Automatically download files from the web - AutoHotkey Script

Joe WinogradDeveloper
CERTIFIED EXPERT
50+ years in computers
EE FELLOW 2017 — first ever recipient of Fellow award
MVE 2015,2016,2018
CERTIFIED GOLD EXPERT
DISTINGUISHED EXPERT
Published:
Updated:
Edited by: Andrew Leniart
In a recent question here at Experts Exchange about Windows 10 automation, a member asked for a solution "to download one by one automatically" 10 specific files from the Internet. This article presents an AutoHotkey script with the solution, enhanced to allow any number of files to be downloaded.

In an interesting question here at Experts Exchange with a subject of "Windows 10 Automation...", a member asked for a solution "to download one by one automatically" ten (10) specific files from the web. I posted a quick-and-dirty script with the solution at the question, then decided to develop a more robust script, which I'm presenting in this article.

I wrote the script in the AutoHotkey language. If you are not familiar with AutoHotkey and this article piques your interest in it, one of my other EE articles will get you started on it:

AutoHotkey - Getting Started

The script presented in this article works as follows:

• Supports the specification of a plain text input file (in the FileListInput variable) that contains one or two fields on each line. The first field (required) is the web address (URL) of the file to download. The second field (optional) is the full path/name of the local file on the PC to store the downloaded file. If the second field is not specified, the downloaded file will be stored in a folder assigned to a variable in the script (OutputFolder). The name of the file will be automatically derived from the URL.


• Allows any number of lines in the input file, with each line representing a file to download.

• Expects the two fields to be separated from each other with a vertical bar (the "pipe" character) followed by a space. I chose this two-character separator because the vertical bar is invalid in a file name and the space is invalid in a URL. So, the two characters together provide a perfect way to separate the fields. You may change this via the assignment statement that puts a value in the FieldSeparator variable, but keep in mind that you don't want the StrSplit (string split) function to find the separator in the web address or the local file name. Here's a sample input line without specifying the local file name:


https://www.google.com/logos/2003/einstein.gif


Here's a sample input line with the local file name specified (it may wrap to two lines in this article, but, of course, it is on one line in the input file):

https://filedb.experts-exchange.com/flags/24x24/us.png| c:\My Folders\AutoDown Output\test good.png

Attached is a sample (working) input file — AutoDown.txt.

• Ignores blank (empty/null) lines in the input file (helpful for improving readability).

• Ignores lines in the input file with a semi-colon (;) in the first column. In other words, they're comments (helpful for documenting the input file).

• Creates a logfile (optional) that makes an entry showing the result of processing each line in the input file. There are three possible entries:

Success — means the download succeeded (but see Two Important Caveats below)

Failure — means the download failed

BadItem — means the input line was bad


The first line of the logfile contains the script's Version, the date and time that the run began, the full path/name of the input file, and the output folder (the latter may or may not be specified). The last line of the logfile contains the date and time that the run ended, along with summary statistics. Each logfile entry begins with the local time in YYYYMMDDhhmmss format. Here's a screenshot of a sample logfile:


The posted code creates a file name for the logfile that has the date and time in it (including milliseconds) and stores it in the Windows %temp% folder (the AutoHotkey built-in variable A_Temp). As with anything in the script, feel free to change it to suit your needs. In particular, there are numerous variables in the ***begin variables to change*** section of the InitializeVars subroutine that you may want to change — and one that you definitely need to change, namely, the location of the input file (FileListInput).

The sample (plain text) logfile shown in the screenshot above is attached (AutoDown_2021-04-23_12.23.16.715.log).


Two Important Caveats


(1) An existing file will be overwritten by the downloaded file with no warning.


(2) The download may seem successful (ErrorLevel=0) even when the file on the web does not exist. This is because some web servers send an error page when the file is missing, and the downloaded error page will look like a success when it has been successfully downloaded.


• Provides a QuietMode option that allows unattended operation, meaning it does not display any dialog boxes (except Fatal Error ones). The results of the run are available in the logfile (if, of course, the logfile was specified).

• Checks the return code (ErrorLevel) on each download and counts the number of successes and failures (but see Two Important Caveats above). These counts are shown in a closing dialog (along with the number of bad items in the input file), unless it's a QuietMode run. Here's a sample closing dialog:


Clicking Yes opens the logfile in the program that owns the LOG file extension on your system, such as Notepad, and then the script exits. The script exits immediately if you click No.


• Displays a system tray (notification area) icon when it is running that looks like this:



Hovering over the icon produces a tooltip:



This allows you to see easily what file is currently downloading.

Right-clicking on the icon produces this context menu:



Note that the context menu does not appear until it is finished the download that is in progress — be patient — the context menu will eventually appear.

Selecting the About menu choice displays this dialog:



Selecting the Exit menu choice exits (terminates/quits) the script, thereby stopping any remaining downloads. To make sure that this wasn't selected by accident, it displays the dialog below, where No is the default button (so that an accidental Enter key won't cause an unwanted exit):


Here is the script in a code block (it is also attached as a file — AutoDown.ahk — at the end of the article for easy downloading):

; Joe Winograd 23-Apr-2021
#Warn,UseUnsetLocal ; warning on uninitialized variables
#NoEnv ; avoid checking empty variables to see if they are environment variables
#SingleInstance Force ; replace old instance immediately
SetBatchLines,-1 ; run at maximum speed
Version:="2" ; version number to put in logfile, "About" dialog box, system tray tip

Gosub,InitializeVars ; initialize all variables
Gosub,ConfigureInitialTray ; configure initial system tray (notification area)

Loop,Parse,FileList,`n,`r ; loop through all files in list
{
  If (A_LoopField="")
    Continue ; ignore blank lines
  If (SubStr(A_LoopField,1,1)=";") ; semi-colon in column 1 is a comment
    Continue ; ignore comments
  FileListArray:=StrSplit(A_LoopField,FieldSeparator) ; split input line at separator
  NumFields:=FileListArray.MaxIndex() ; get number of fields on line
  If (NumFields>2) ; number of fields must be 1 or 2 (no need to check for 0 because blank lines have already been ignored)
  {
    NumBad:=NumBad+1
    WriteLogfile("BadItem: Too many fields - " . A_LoopField)
    Continue
  }
  URLweb:=FileListArray[1] ; link to file on web
  If (NumFields=1) ; only one field so store in OutputFolder
  {
    If (OutputFolder="Not specified")
    {
      NumBad:=NumBad+1
      WriteLogfile("BadItem: No filename on line and output folder not specified - " . A_LoopField)
      Continue
    }
    SplitPath,URLweb,FileNameOnly
    FileLocal:=OutputFolder . FileNameOnly
  }
  Else
  {
    FileLocal:=FileListArray[2] ; full path to file on computer
    SplitPath,FileLocal,,FolderOnly
    If (!Instr(FileExist(FolderOnly),"D"))
    {
      NumBad:=NumBad+1
      WriteLogfile("BadItem: Output folder does not exist - " . A_LoopField)
      Continue
    }
  }
  TrayTip:=ProgramName . " Version " . Version . "`nDownloading: " . URLweb
  Menu,Tray,Tip,%TrayTip%
  UrlDownloadToFile,%URLweb%,%FileLocal% ; download file
  If (ErrorLevel=0)
  {
    NumSuccess:=NumSuccess+1
    Result:="Success: "
  }
  Else
  {
    NumFailure:=NumFailure+1
    Result:="Failure: "
  }
  WriteLogfile(Result . A_LoopField)
}
EndTimeStd:=A_Now
FormatTime,EndTime,%EndTimeStd%,yyyy-MM-dd_HH.mm.ss ; end time nicely formatted
WriteLogfile("Ending date and time: " . EndTime . "  Succeeded: " . NumSuccess . "  Failed: " . NumFailure . "  Bad: " . NumBad)
If (!QuietMode) ; do not display closing dialog box in quiet mode run
{
  If (Logfile="") ; no logfile - just display closing dialog and exit
  {
    MsgBox,4160,%ProgramName% Done,Succeeded: %NumSuccess%`nFailed: %NumFailure%`nBad: %NumBad%
    ExitApp
  }
  Else ; logfile assigned
  {
    MsgBox,4164,%ProgramName% Done,Succeeded: %NumSuccess%`nFailed: %NumFailure%`nBad: %NumBad%`n`nDo you want to open logfile now?"
    IfMsgBox,No
      ExitApp
    If (!FileExist(Logfile))
      MsgBox,4112,Fatal Error,Unable to create logfile:`n%Logfile%
    Else
      Run,%Logfile%
  }
}
ExitApp

InitializeVars:
BeginTimeStd:=A_Now ; begin time in standard time format
FormatTime,BeginTime,%BeginTimeStd%,yyyy-MM-dd_HH.mm.ss ; begin time nicely formatted and valid in file name
SplitPath,A_ScriptName,,,,ProgramName ; get name of script without path or extension

; *** begin variables to change ***
FileListInput:="c:\My Files\AutoDownV2 Test.txt" ; full path to plain text file with a URL on each line (and an optional comment)
OutputFolder:="c:\My Folders\AutoDown Output" ; output folder - works with or without ending backslash - set to null [OutputFolder:=""] if all lines have local filename
Logfile:=A_Temp . "\" . ProgramName . "_" . BeginTime . "." . A_Msec . ".log" ; logfile full path - set to null [Logfile:=""] for no logfile
TrayIconFile:=A_WinDir . "\System32\wpdshext.dll" ; icon from built-in wpdshext.dll file
TrayIconNum:="-751" ; blue down arrow
FieldSeparator:="| " ; pipe (vertical bar) and space - pipe invalid in file name - space invalid in URL
QuietMode:=False ; True ==> suppress all dialogs except Fatal Error for unattended runs
; *** end variables to change ***

; validate input file and read it
If (!FileExist(FileListInput))
{
  MsgBox,4112,Fatal Error,Input file does not exist:`n`n%FileListInput%
  ExitApp
}
FileRead,FileList,%FileListInput% ; read list of files to download
If (ErrorLevel!=0)
{
  MsgBox,4112,Fatal Error,Error Level %ErrorLevel% trying to read file list:`n`n%FileListInput%
  ExitApp
}

; validate output folder if it is specified
If (OutputFolder="")
  OutputFolder:="Not specified"
Else
{
  If (SubStr(OutputFolder,0,1)!="\")
    OutputFolder:=OutputFolder . "\" ; add trailing backslash if there is not one
  If (!Instr(FileExist(OutputFolder),"D"))
  {
    MsgBox,4112,Fatal Error,Output folder does not exist:`n`n%OutputFolder%
    ExitApp
  }
}
NumSuccess:=NumFailure:=NumBad:=0 ; set Success, Failure, and Bad counters to zero
WriteLogfile(ProgramName . " Version " . Version . "   Beginning date and time: " . BeginTime . "   Input File: " . FileListInput . "   Output Folder: " . OutputFolder) ; write opening logfile record
Return

ConfigureInitialTray:
Menu,Tray,NoStandard ; do not use standard AutoHotkey context menu
Menu,Tray,Add,&About,ContextMenu ; first item is About
Menu,Tray,Add,E&xit,ContextMenu ; last item is Exit
TrayTip:=ProgramName . " Version " . Version
Menu,Tray,Tip,%TrayTip%
Menu,Tray,Icon,%TrayIconFile%,%TrayIconNum%
Menu,Tray,Default,&About ; default (double-click) is About
Return

ContextMenu:
; check which context menu selected
If (A_ThisMenuItem="&About")
{
  MsgBox,4160,About %ProgramName%,%ProgramName% Version %Version%
  Return
}
If (A_ThisMenuItem="E&xit") ; true means Exit selected, but make sure it was not accidental
{
  MsgBox,4388,Terminate %ProgramName%?,Are you sure you want to quit? ; 4388 means No is default button, in case of accidental Enter key
  IfMsgBox,No
    Return ; user does not really want to quit - stay running
}
ExitApp ; user wants to quit - terminate app

WriteLogfile(Record)
{
  global Logfile
  If (Logfile="")
    Return
  FileAppend,%A_Now% %Record%`n,%Logfile%
  If (ErrorLevel!=0)
  {
    MsgBox,4112,Fatal Error,Error Level %ErrorLevel% occurred while trying to write record:`n%Record%`nto logfile:`n%Logfile%
    ExitApp
  }
  Return
}


I hope that the descriptive variable names and the comments in the script provide enough documentation for readers to understand and modify it, but if you have any questions, please post them here and I'll try to help.

The script should work in all versions of Windows from XP through W10, 32-bit and 64-bit. I tested it on W7/64-bit and W10/64-bit — both worked perfectly.

If you find this article to be helpful, please click the thumbs-up icon below. This lets me know what is valuable for EE members and provides direction for future articles. Thanks very much! Regards, Joe

AutoDown.ahk

AutoDown.txt

AutoDown_2021-04-23_12.23.16.715.log





2
6,491 Views
Joe WinogradDeveloper
CERTIFIED EXPERT
50+ years in computers
EE FELLOW 2017 — first ever recipient of Fellow award
MVE 2015,2016,2018
CERTIFIED GOLD EXPERT
DISTINGUISHED EXPERT

Comments (2)

NVITEnd-user support
CERTIFIED EXPERT

Commented:
Thanks for this article, Joe. This example assumes one knows the link of the file to download. But, in my recent question here, there doesn't seem to be an obvious download link when I hover my cursor. It's for powershell but I'm open to other methods.

Any ideas?

https://www.experts-exchange.com/questions/29180262/How-to-download-file-from-website.html
Joe WinogradDeveloper
CERTIFIED EXPERT
Fellow
Most Valuable Expert 2018

Author

Commented:
Hi NVIT,

I hope you're doing well and staying safe in paradise. If one has to shelter in place, I'd certainly like to be doing it in your neck of the woods. :)

I looked at your question and one idea that comes to mind is downloading the HTML for that page and finding the XLSX file in the HTML text. Do you think that would work?

I don't code in PowerShell, but I'm sure you know how to download from a URL to a file. In AutoHotkey, it is this line of code:

UrlDownloadToFile,%URL%,%FileFullPath%

Open in new window

The command name and variable names make it clear what that does...there must be an equivalent in PowerShell. Regards, Joe

Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.