Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

Saving a File from the Internet onto server

Posted on 2004-03-20
13
915 Views
Last Modified: 2010-08-05
How do I download a file from the internet onto the server? (there are all kinds of files.. html / ASP / PHP / PDF / Excel / Word etc.) (Assume that the directory to which it is being written to is write-enabled)

While saving ...
1. I need to check if the file already exists.
2. I need to rename each file with a unique id.

:) SD

Assuming that the URLS are like this and they are all in an array....

http://www.sciencegroups.com/viewtopic.php?t=15958&view=next
http://jama.ama-assn.org/cgi/reprint/287/21/2869.pdf
http://gollem.swi.psy.uva.nl/cgi-bin/pl-cvsweb/pl/man/lib/url.doc
http://search.ngfl.gov.uk/senco-forum/senco-forum.archive.0303/msg00411.html
http://groups.msn.com/VestibularDisordersImbalance/general.msnw?action=get_message&mview=1&ID_Message=565
www.counterhack.net/url.ppt
http://www.zlw-ima.rwth-aachen.de/forschung/eoi/Dokumente/EoI-FP6_2002_IP_esquare_IfU.pdf
0
Comment
Question by:sciber_dude
  • 5
  • 4
  • 2
  • +2
13 Comments
 
LVL 6

Assisted Solution

by:bvinson
bvinson earned 20 total points
ID: 10642844
Try this - and make sure you have the proper rights to upload files.

<%Response.Buffer = true
Function BuildUpload(RequestBin)
     'Get the boundary
     PosBeg = 1
     PosEnd = InstrB(PosBeg,RequestBin,getByteString(chr(13)))
     boundary = MidB(RequestBin,PosBeg,PosEnd-PosBeg)
     boundaryPos = InstrB(1,RequestBin,boundary)
     'Get all data inside the boundaries
     Do until (boundaryPos=InstrB(RequestBin,boundary & getByteString("--")))
          'Members variable of objects are put in a dictionary object
          Dim UploadControl
          Set UploadControl = CreateObject("Scripting.Dictionary")
          'Get an object name
          Pos = InstrB(BoundaryPos,RequestBin,getByteString("Content-Disposition"))
          Pos = InstrB(Pos,RequestBin,getByteString("name="))
          PosBeg = Pos+6
          PosEnd = InstrB(PosBeg,RequestBin,getByteString(chr(34)))
          Name = getString(MidB(RequestBin,PosBeg,PosEnd-PosBeg))
          PosFile = InstrB(BoundaryPos,RequestBin,getByteString("filename="))
          PosBound = InstrB(PosEnd,RequestBin,boundary)
          'Test if object is of file type
          If PosFile<>0 AND (PosFile<PosBound) Then
               'Get Filename, content-type and content of file
               PosBeg = PosFile + 10
               PosEnd = InstrB(PosBeg,RequestBin,getByteString(chr(34)))
               FileName = getString(MidB(RequestBin,PosBeg,PosEnd-PosBeg))
               'Add filename to dictionary object
               UploadControl.Add "FileName", FileName
               Pos = InstrB(PosEnd,RequestBin,getByteString("Content-Type:"))
               PosBeg = Pos+14
               PosEnd = InstrB(PosBeg,RequestBin,getByteString(chr(13)))
               'Add content-type to dictionary object
               ContentType = getString(MidB(RequestBin,PosBeg,PosEnd-PosBeg))
               UploadControl.Add "ContentType",ContentType
               'Get content of object
               PosBeg = PosEnd+4
               PosEnd = InstrB(PosBeg,RequestBin,boundary)-2
               Value = MidB(RequestBin,PosBeg,PosEnd-PosBeg)
               Else
               'Get content of object
               Pos = InstrB(Pos,RequestBin,getByteString(chr(13)))
               PosBeg = Pos+4
               PosEnd = InstrB(PosBeg,RequestBin,boundary)-2
               Value = getString(MidB(RequestBin,PosBeg,PosEnd-PosBeg))
          End If
          UploadControl.Add "Value" , Value    
          UploadRequest.Add name, UploadControl    
          BoundaryPos=InstrB(BoundaryPos+LenB(boundary),RequestBin,boundary)
     Loop
End Function

Function getByteString(StringStr)
     For i = 1 to Len(StringStr)
          char = Mid(StringStr,i,1)
          getByteString = getByteString & chrB(AscB(char))
     Next
End Function

Function getString(StringBin)
     getString =""
     For intCount = 1 to LenB(StringBin)
          getString = getString & chr(AscB(MidB(StringBin,intCount,1)))
     Next
End Function

If request("Action")="1" then
     Response.Clear
     byteCount = Request.TotalBytes
     
     RequestBin = Request.BinaryRead(byteCount)
     
     Set UploadRequest = CreateObject("Scripting.Dictionary")

     BuildUpload(RequestBin)
     
     If UploadRequest.Item("blob").Item("Value") <> "" Then

          contentType = UploadRequest.Item("blob").Item("ContentType")
          filepathname = UploadRequest.Item("blob").Item("FileName")
          filename = Right(filepathname,Len(filepathname)-InstrRev(filepathname,"\"))
          FolderName = UploadRequest.Item("where").Item("Value")
          Path = server.mappath(".")  & "\"  
          ToFolder = Path & "\" & FolderName
          value = UploadRequest.Item("blob").Item("Value")
          filename = ToFolder & "\" & filename
          Set MyFileObject = Server.CreateObject("Scripting.FileSystemObject")
          Set objFile = MyFileObject.CreateTextFile(filename)
                     
          For i = 1 to LenB(value)
               objFile.Write chr(AscB(MidB(value,i,1)))
          Next          
          objFile.Close
          Set objFile = Nothing
          Set MyFileObject = Nothing
     End If
     Set UploadRequest = Nothing


End If%>
<form METHOD="POST" ENCTYPE="multipart/form-data" action="Upload.asp?Action=1" name="form1" id="form1">
<table cellspacing="0" cellpadding="3" width="100%" border="0">
     <tr>
          <td>
               <b>Select a file to upload:</b><br>
          </td>
    <tr>
    </tr>
          <td>
               <input TYPE="file" NAME="blob" value>
               <input TYPE="HIDDEN" NAME="where" value="Images">
          </td>
     <tr>
     <tr>
          <td align="center">
               <input TYPE="submit" NAME="Upload" Value="Click to Upload">
          </td>
     </tr>
</table>
</form>

bvinson
0
 
LVL 8

Expert Comment

by:fozylet
ID: 10642846
VB with iNet controls would be a better way of doing this (as compared to ASP with HTTP controls)

How to do part...

- You need a database as files are being saved with a unique ID and not in its real name. The table could have two fields ID,URL. For the ID to be unique, you could write an algorithm for it or let the db autogenerate it.

loop thru the array
  - Check DB to see if the URL is already there: SELECT ID FROM urltable WHERE URL IN (SELECT URL FROM urltable)
  - if the above query returns an empty recordset
     - Use the iNet control to GET each file in the array and save.
  - skip the URL if the recordset not empty
loop
0
 
LVL 11

Author Comment

by:sciber_dude
ID: 10642859
@ bvinson -

(Correct me if I am wrong) It seems that you have given me code to upload a file from local machine onto the server.

What I want is code for downloading the files from URLs onto the machine where the script is run.

@ fozylet -

I need an ASP solution :(

:) SD

0
Networking for the Cloud Era

Join Microsoft and Riverbed for a discussion and demonstration of enhancements to SteelConnect:
-One-click orchestration and cloud connectivity in Azure environments
-Tight integration of SD-WAN and WAN optimization capabilities
-Scalability and resiliency equal to a data center

 
LVL 11

Author Comment

by:sciber_dude
ID: 10642866
@ fozylet -

I kinda know the logic as to what is to be done.

In my database, I have the following fields.

ID | URL

I want to code to actually download the files from the URL onto the local machine and rename the file with ID.extension.

(depends on whatever the extension is).

:) SD
0
 
LVL 6

Expert Comment

by:bvinson
ID: 10642867
You're right, I'm wrong (again)...I completely misread your question.

Sorry for any potential confusion.
bvinson
0
 
LVL 11

Author Comment

by:sciber_dude
ID: 10642870
not a problem! Sorry for wasting your effort.

SD
0
 
LVL 8

Expert Comment

by:fozylet
ID: 10642895
@sciber_dude,
Guess the same thing can be done with XMLHTTP controls (instead of iNet controls)

right now i'm fighting with my deadlines at work (working on a sunday!)
will be happy to write a code for u when i get some free time i.e. if u still need it then ;)
0
 
LVL 11

Assisted Solution

by:mouatts
mouatts earned 80 total points
ID: 10643394
As has been said XmlHttp server or client or ASPHTTP components are the key to doing this. The former is probably the easier for binary files. Your starter for 10 is

http://support.microsoft.com/default.aspx?scid=kb;EN-US;290761&product=msxml

HTH
Steve
0
 
LVL 8

Accepted Solution

by:
fozylet earned 320 total points
ID: 10647955
The database integration part is not there... This one saves with the same filename. It should not be big deal for you to write the DB part...

<%
Response.Buffer = False
Dim url(50)
url(0) = "http://www.sciencegroups.com/viewtopic.php?t=15958&view=next"
url(0) = "http://172.21.9.50:6006/default.asp"
url(1) = "http://jama.ama-assn.org/cgi/reprint/287/21/2869.pdf"
url(2) = "http://gollem.swi.psy.uva.nl/cgi-bin/pl-cvsweb/pl/man/lib/url.doc"
url(3) = "http://search.ngfl.gov.uk/senco-forum/senco-forum.archive.0303/msg00411.html"
url(4) = "http://groups.msn.com/VestibularDisordersImbalance/general.msnw?action=get_message&mview=1&ID_Message=565"
url(5) = "http://www.counterhack.net/url.ppt"
url(6) = "http://www.zlw-ima.rwth-aachen.de/forschung/eoi/Dokumente/EoI-FP6_2002_IP_esquare_IfU.pdf"

Set xmlhttp = Server.CreateObject("Msxml2.ServerXMLHTTP")

For Each Link In url
      Response.Write "Fetching " & Link & "... "
      SaveToFile Link
      Response.Write "Done!<br/>"
Next

Set xmlhttp = Nothing

Sub SaveToFile(strURL)
response.write strurl
   xmlhttp.Open "GET", strURL, False
   xmlhttp.send
   If xmlhttp.Status = 200 Then
        strValue = xmlhttp.responseBody
        iSt1 = InStrRev(strURL,"/")+1
        iSt2 = InStr(iSt1,strURL,".")+1
        iEd = InStr(iSt1,strURL,"?")
        If iEd = 0 Then iEd = Len(strURL)+1
        FileName = Mid(strURL,iSt1,iEd-iSt1)
        FileExt = Mid(strURL,iSt2,iEd-iSt2)
        Response.Write FileName & FileExt
        FileName = Server.MapPath(".") & "\" & FileName

        Set oStream = Server.CreateObject("ADODB.Stream")
        oStream.Open
        oStream.Position = 0
        oStream.Type = 1
        oStream.Write strValue
        oStream.SaveToFile FileName,2
        oStream.Close
        Set oStream=nothing
    End If
End Sub
%>
0
 
LVL 8

Expert Comment

by:fozylet
ID: 10647969
Minor change...

If Link<>"" Then SaveToFile Link

Also, please take care of declaration/error handling.
0
 
LVL 28

Assisted Solution

by:sybe
sybe earned 80 total points
ID: 10648023
My best suggestion is not to use ASP but use VBS. That is because your problem is caused by time-outs, and VBS does not have a timeout on the server it's running on.

Take fozulet's code (it looks ok to me), remove the ASP tags (<% and %>) and replace Server.CreateObject with CreateObject and save the file with the extension .vbs. Then run it from your machine.
0
 
LVL 11

Author Comment

by:sciber_dude
ID: 10651239
Thanks fozylet and sybe,

I will try these out and get back to you by today evening. Mondays :(

:) SD
0
 
LVL 11

Author Comment

by:sciber_dude
ID: 10680007
@ fozylet  -
Thanks for the code. That worked perfectly.

@ sybe -
Thanks for the ASP Vs VBS idea. :)

@ mouatts -
Thank you for the link, It was a good read. Now I feel better empowered to approach it.

@ bvinson -
Though your answer was not what i was looking for, I didnt want ur effort to be wasted. Thank you once again.

:) SD
0

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Add server behaviors to Dreamweaver CC 2015 2 185
Select distinct 25 115
Get hold of longitude and latitude in iframe string 11 48
Html Table looping 4 25
I have helped a lot of people on EE with their coding sources and have enjoyed near about every minute of it. Sometimes it can get a little tedious but it is always a challenge and the one thing that I always say is:  The Exchange of information …
I would like to start this tip/trick by saying Thank You, to all who said that this could not be done, as it forced me to make sure that it could be accomplished. :) To start, I want to make sure everyone understands the importance of utilizing p…
This video shows how to quickly and easily add an email signature for all users on Exchange 2016. The resulting signature is applied on a server level by Exchange Online. The email signature template has been downloaded from: www.mail-signatures…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question