[Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 206
  • Last Modified:

problem transfering large files to using selfbuild webserver

Hello experts,

I am trying to build a webserver with some special functions in it for personal use. File transfer actually works fine but with large files > 5mb my program will not respond anymore till the file is "loaded" in the computers memory( and this time the file was only 100mb so if it gets even bigger like > 700mb I think my computer will crash ). I think this is because of bad code but i don't know any other solutions for it so i hope some of you can help me out.

here the piece of code i use to transmit data:

Open App.Path & "\" & page For Binary As #1
        entireFile = Space(LOF(1))
        Get #1, , entireFile
        Close #1
Socket(Index).SendData entireFile

so actually the file is loaded in 'entirefile' and that variabel is send with winsock to the user connected to it.

So what im asking if someone knows a better solution to my problem.

Greetings War Lord
0
WarLord
Asked:
WarLord
  • 10
  • 9
  • 3
1 Solution
 
Michael_DCommented:
Try something like this

Open App.Path & "\" & page For Binary As #1
Do While not EOF(#1)
        CurrentPortion= Space(PORTION_SIZE)
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
Loop

Close #1

Try different values of PORTION_SIZE for best performance.
0
 
WarLordAuthor Commented:
hmz,

Do While not EOF(#1)

gives me a Expected: expression error

for the rest the idea of the solution looks good.

ps. PORTION_SIZE = size in bytes?
0
 
Michael_DCommented:
Sorry, remove the #

Do While not EOF(1)
:::
:::

Yes PORTION_SIZEis size in bytes


Also I think you should control portion's size. (It must be not more than rest of the size)

tell me if you need more help

0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
WarLordAuthor Commented:
Indeed I already figured it out that I should remove the #. But nou I have the problem as you describe that the portion size must be no larger as the file itself.

any examples on how to fix that?
0
 
Michael_DCommented:
Dim filesize as long
Dim Transfered as long
Open App.Path & "\" & page For Binary As #1
Filesize=LOF(1)
Transfered=0
Do While not EOF(#1)
       If Transfered+portion_size>filesize then
              Currentportion=space(Filesize-transfered)
       else
              CurrentPortion= Space(PORTION_SIZE)
       end if
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        transfered=transfered + Portion_size
Loop

Close #1

I hope it helps
0
 
WarLordAuthor Commented:
hmz,

Dim filesize As Long
Dim Transfered As Long
Dim CurrentPortion As Long
Dim Portion_size As Long
Open App.Path & "\" & page For Binary As #1
filesize = LOF(1)
Transfered = 0
CurrentPortion = 0
Portion_size = 1000000
Do While Not EOF(1)
       If Transfered + Portion_size > filesize Then
              CurrentPortion = Space(filesize - Transfered)
       Else
              CurrentPortion = Space(Portion_size)
       End If
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        Transfered = Transfered + Portion_size
Loop

Close #1

with this code i get a type mismatch with CurrentPortion = Space(Portion_size)
also tried some other stuff but that got overflows.

ps. because i ask so much questions i will raise the point value for this question.

0
 
sgayatriCommented:
This module is working code -- tested for transfering 30mb file also.
It actually makes use of ftp.exe of windows
test it ....

Creates three text files
ftpmain.bat - which contains the actual ftp command and takes ftptest.bat as an input
ftptest.bat - which is a command file that gets created in the code
ftp.log - the log file written during ftp


Write your own interface (i.e., vb form) call the following module
assign values where necessary

(variables fname -- file to be FTPied
 FTPSERVER - IPadrr or alias of the server )

this code gets executed in background
 

Private Sub ftp_send_receive_dos()
''' Give the ip address or server alias at the place
''' where FTPSERVER is mentioned in this code

comd = "put"   ''' change this to put or get as per requirement

Set fso = CreateObject("scripting.filesystemobject")
Set f = CreateObject("scripting.filesystemobject")

outfile = fname ''' fname is the file to be FTPied


If comd = "put" And f.FileExists(outfile) = False Then
    MsgBox " File is not existing "
    Exit Sub
End If

''''delete old files

If f.FileExists("c:\windows\temp\ftpmain.bat") = True Then
    f.DeleteFile ("c:\windows\temp\ftpmain.bat")
End If
If f.FileExists("c:\windows\temp\ftptest") = True Then
    f.DeleteFile ("c:\windows\temp\ftptest")
End If

''' create ftpmain.bat

Set mainfile = fso.CreateTextFile("c:\windows\temp\ftpmain.bat")
mainfile.WriteLine ("ftp -s:c:\windows\temp\ftptest.bat FTPSERVER > c:\windows\desktop\ftp.log")

mainfile.Close

''' create ftptest.bat

Set batfile = fso.CreateTextFile("c:\windows\temp\ftptest.bat")
batfile.WriteLine ("USERID")
batfile.WriteBlankLines (1)
batfile.WriteLine ("PASSWORD")
batfile.WriteBlankLines (1)
batfile.WriteLine ("bin")
batfile.WriteBlankLines (1)
batfile.WriteBlankLines (1)
batfile.WriteLine ("cd ftpdocs")
batfile.WriteBlankLines (1)
batfile.WriteLine (comd & " " & outfile)
batfile.WriteBlankLines (1)
batfile.WriteLine ("bye")
batfile.Close

''' run the ftpmain.bat

Shell "command.com  /c  c:\windows\temp\ftpmain.bat", vbHide

length = -1
While f.FileExists("c:\windows\desktop\ftp.log") = False
   Sleep 1000
Wend
'
Label2.Caption = "FTP Started..."
Sleep 100
Label2.Refresh


ftpprg = "FTP in Progress "
polinterval = 10000   ' 10 sec
mins = 0
cnt = 0
While FileLen("c:\windows\desktop\ftp.log") <> length

    If FileLen("c:\windows\desktop\ftp.log") > 0 Then
       length = FileLen("c:\windows\desktop\ftp.log")
    End If
    Sleep polinterval

    cnt = cnt + 1
    'If Cnt = 6 Then
    If cnt = (60000 / polinterval) Then
        mins = mins + 1
        cnt = 0
        ftpprg = "FTP in Progress " & mins 'ftpprg & mins
    Else
        ftpprg = ftpprg & "."
        Label2.Caption = ftpprg
        Label2.Refresh
    End If
   'Sleep 3000
Wend
'pbar.Value = pbar.Max
MsgBox "file length is " & length


f.DeleteFile ("c:\windows\temp\ftpmain.bat")
f.DeleteFile ("c:\windows\temp\ftptest.bat")

end sub
0
 
Michael_DCommented:
Here you go, This time it's TESTED!!!
(I should do it before, sorry)


Dim filesize As Long
Dim Transfered As Long
Dim CurrentPortion As String  ' <= was Long
Dim Portion_size As Long
Portion_size = 1000 <= I've redused size for testing

'Open "C:\text.txt" For Binary As #1
Open App.Path & "\" & page For Binary As #1
filesize = LOF(1)
Transfered = 0
CurrentPortion = ""
Do While Transfered < filesize ' <= Notice this change
       If Transfered + Portion_size > filesize Then
              CurrentPortion = Space$(filesize - Transfered)
       Else
              CurrentPortion = Space$(Portion_size)
       End If
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        Transfered = Transfered + Len(CurrentPortion)
Loop

Close #1
0
 
WarLordAuthor Commented:
@michael D

hmz, how should i put this. when i tried a 100mb file it worked fine, but when i tried a 1 gig it did the same again and froze my whole computer. :(
0
 
Michael_DCommented:
This code should work up to 2GB without . What is the value of Portion_size?
0
 
Michael_DCommented:
...without any problem  
0
 
WarLordAuthor Commented:
tried different sizes, 10000 100000 1000000
0
 
sgayatriCommented:
Warlord,

Yu aren't attempting the code posted ...
I have been using it for quite some time without any problem.
Very large files also have been transfered thru that.

You just have to paste the code and change a few parameters ...

0
 
WarLordAuthor Commented:
i know sgayatri but i don't want to an external program i like michael his solution except is doesn't work here very well.
0
 
Michael_DCommented:
How much time took to transfer 100Mb?
0
 
WarLordAuthor Commented:
the transfer speed is fine the point where it's going wrong is when you click a link and the server tries to load the file
0
 
Michael_DCommented:
I think the problem in other place.
Could you post the code how do you obtain the file. What does it mean  "when you click a link and the server tries to load the file"?
0
 
WarLordAuthor Commented:
If Mid(strData, 1, 10) = "GET /file/" Then
strGet = InStr(strData, "GET ")
spc2 = InStr(strGet + 5, strData, " ")
page = Trim(Mid(strData, strGet + 10, spc2 - (strGet + 10)))
If Right(page, 1) = "/" Then page = Left(page, Len(page) - 1)

List1.AddItem "Page being requested: [" & page & "] by " & Socket(Index).RemoteHostIP

Open App.Path & "\log.txt" For Append As #1
    Write #1, Date & " " & Time & "   " & "Page being requested: [" & page & "] by " & Socket(Index).RemoteHostIP
    Close #1


Dim filesize As Long
Dim Transfered As Long
Dim CurrentPortion As String  ' <= was Long
Dim Portion_size As Long
Portion_size = 10000 '<= i 've redused size for testing

Open App.Path & "\" & page For Binary As #1
filesize = LOF(1)
Transfered = 0
CurrentPortion = ""
Do While Transfered < filesize ' <= Notice this change
       If Transfered + Portion_size > filesize Then
              CurrentPortion = Space$(filesize - Transfered)
       Else
              CurrentPortion = Space$(Portion_size)
       End If
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        Transfered = Transfered + Len(CurrentPortion)
Loop

Close #1

End If


Hope this helps. the server checks which command is recieved then strips the recieved command writes information of the client to a log file and at last it opens the file and sends it to the client.  
0
 
WarLordAuthor Commented:
No one has an answer?
0
 
sgayatriCommented:
Another answer also has been posted earlier. You aren't even attempting....
Good Luck !!!
0
 
WarLordAuthor Commented:
maybe it works but i don't like the solution.
0
 
Michael_DCommented:
The code works fine for me.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 10
  • 9
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now