Solved

problem transfering large files to using selfbuild webserver

Posted on 2004-09-20
23
193 Views
Last Modified: 2013-11-13
Hello experts,

I am trying to build a webserver with some special functions in it for personal use. File transfer actually works fine but with large files > 5mb my program will not respond anymore till the file is "loaded" in the computers memory( and this time the file was only 100mb so if it gets even bigger like > 700mb I think my computer will crash ). I think this is because of bad code but i don't know any other solutions for it so i hope some of you can help me out.

here the piece of code i use to transmit data:

Open App.Path & "\" & page For Binary As #1
        entireFile = Space(LOF(1))
        Get #1, , entireFile
        Close #1
Socket(Index).SendData entireFile

so actually the file is loaded in 'entirefile' and that variabel is send with winsock to the user connected to it.

So what im asking if someone knows a better solution to my problem.

Greetings War Lord
0
Comment
Question by:WarLord
  • 10
  • 9
  • 3
23 Comments
 
LVL 13

Expert Comment

by:Michael_D
ID: 12106525
Try something like this

Open App.Path & "\" & page For Binary As #1
Do While not EOF(#1)
        CurrentPortion= Space(PORTION_SIZE)
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
Loop

Close #1

Try different values of PORTION_SIZE for best performance.
0
 

Author Comment

by:WarLord
ID: 12106628
hmz,

Do While not EOF(#1)

gives me a Expected: expression error

for the rest the idea of the solution looks good.

ps. PORTION_SIZE = size in bytes?
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12106726
Sorry, remove the #

Do While not EOF(1)
:::
:::

Yes PORTION_SIZEis size in bytes


Also I think you should control portion's size. (It must be not more than rest of the size)

tell me if you need more help

0
 

Author Comment

by:WarLord
ID: 12107670
Indeed I already figured it out that I should remove the #. But nou I have the problem as you describe that the portion size must be no larger as the file itself.

any examples on how to fix that?
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12108257
Dim filesize as long
Dim Transfered as long
Open App.Path & "\" & page For Binary As #1
Filesize=LOF(1)
Transfered=0
Do While not EOF(#1)
       If Transfered+portion_size>filesize then
              Currentportion=space(Filesize-transfered)
       else
              CurrentPortion= Space(PORTION_SIZE)
       end if
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        transfered=transfered + Portion_size
Loop

Close #1

I hope it helps
0
 

Author Comment

by:WarLord
ID: 12109493
hmz,

Dim filesize As Long
Dim Transfered As Long
Dim CurrentPortion As Long
Dim Portion_size As Long
Open App.Path & "\" & page For Binary As #1
filesize = LOF(1)
Transfered = 0
CurrentPortion = 0
Portion_size = 1000000
Do While Not EOF(1)
       If Transfered + Portion_size > filesize Then
              CurrentPortion = Space(filesize - Transfered)
       Else
              CurrentPortion = Space(Portion_size)
       End If
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        Transfered = Transfered + Portion_size
Loop

Close #1

with this code i get a type mismatch with CurrentPortion = Space(Portion_size)
also tried some other stuff but that got overflows.

ps. because i ask so much questions i will raise the point value for this question.

0
 
LVL 3

Expert Comment

by:sgayatri
ID: 12109577
This module is working code -- tested for transfering 30mb file also.
It actually makes use of ftp.exe of windows
test it ....

Creates three text files
ftpmain.bat - which contains the actual ftp command and takes ftptest.bat as an input
ftptest.bat - which is a command file that gets created in the code
ftp.log - the log file written during ftp


Write your own interface (i.e., vb form) call the following module
assign values where necessary

(variables fname -- file to be FTPied
 FTPSERVER - IPadrr or alias of the server )

this code gets executed in background
 

Private Sub ftp_send_receive_dos()
''' Give the ip address or server alias at the place
''' where FTPSERVER is mentioned in this code

comd = "put"   ''' change this to put or get as per requirement

Set fso = CreateObject("scripting.filesystemobject")
Set f = CreateObject("scripting.filesystemobject")

outfile = fname ''' fname is the file to be FTPied


If comd = "put" And f.FileExists(outfile) = False Then
    MsgBox " File is not existing "
    Exit Sub
End If

''''delete old files

If f.FileExists("c:\windows\temp\ftpmain.bat") = True Then
    f.DeleteFile ("c:\windows\temp\ftpmain.bat")
End If
If f.FileExists("c:\windows\temp\ftptest") = True Then
    f.DeleteFile ("c:\windows\temp\ftptest")
End If

''' create ftpmain.bat

Set mainfile = fso.CreateTextFile("c:\windows\temp\ftpmain.bat")
mainfile.WriteLine ("ftp -s:c:\windows\temp\ftptest.bat FTPSERVER > c:\windows\desktop\ftp.log")

mainfile.Close

''' create ftptest.bat

Set batfile = fso.CreateTextFile("c:\windows\temp\ftptest.bat")
batfile.WriteLine ("USERID")
batfile.WriteBlankLines (1)
batfile.WriteLine ("PASSWORD")
batfile.WriteBlankLines (1)
batfile.WriteLine ("bin")
batfile.WriteBlankLines (1)
batfile.WriteBlankLines (1)
batfile.WriteLine ("cd ftpdocs")
batfile.WriteBlankLines (1)
batfile.WriteLine (comd & " " & outfile)
batfile.WriteBlankLines (1)
batfile.WriteLine ("bye")
batfile.Close

''' run the ftpmain.bat

Shell "command.com  /c  c:\windows\temp\ftpmain.bat", vbHide

length = -1
While f.FileExists("c:\windows\desktop\ftp.log") = False
   Sleep 1000
Wend
'
Label2.Caption = "FTP Started..."
Sleep 100
Label2.Refresh


ftpprg = "FTP in Progress "
polinterval = 10000   ' 10 sec
mins = 0
cnt = 0
While FileLen("c:\windows\desktop\ftp.log") <> length

    If FileLen("c:\windows\desktop\ftp.log") > 0 Then
       length = FileLen("c:\windows\desktop\ftp.log")
    End If
    Sleep polinterval

    cnt = cnt + 1
    'If Cnt = 6 Then
    If cnt = (60000 / polinterval) Then
        mins = mins + 1
        cnt = 0
        ftpprg = "FTP in Progress " & mins 'ftpprg & mins
    Else
        ftpprg = ftpprg & "."
        Label2.Caption = ftpprg
        Label2.Refresh
    End If
   'Sleep 3000
Wend
'pbar.Value = pbar.Max
MsgBox "file length is " & length


f.DeleteFile ("c:\windows\temp\ftpmain.bat")
f.DeleteFile ("c:\windows\temp\ftptest.bat")

end sub
0
 
LVL 13

Accepted Solution

by:
Michael_D earned 250 total points
ID: 12111162
Here you go, This time it's TESTED!!!
(I should do it before, sorry)


Dim filesize As Long
Dim Transfered As Long
Dim CurrentPortion As String  ' <= was Long
Dim Portion_size As Long
Portion_size = 1000 <= I've redused size for testing

'Open "C:\text.txt" For Binary As #1
Open App.Path & "\" & page For Binary As #1
filesize = LOF(1)
Transfered = 0
CurrentPortion = ""
Do While Transfered < filesize ' <= Notice this change
       If Transfered + Portion_size > filesize Then
              CurrentPortion = Space$(filesize - Transfered)
       Else
              CurrentPortion = Space$(Portion_size)
       End If
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        Transfered = Transfered + Len(CurrentPortion)
Loop

Close #1
0
 

Author Comment

by:WarLord
ID: 12137264
@michael D

hmz, how should i put this. when i tried a 100mb file it worked fine, but when i tried a 1 gig it did the same again and froze my whole computer. :(
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12138204
This code should work up to 2GB without . What is the value of Portion_size?
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12138251
...without any problem  
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 

Author Comment

by:WarLord
ID: 12140495
tried different sizes, 10000 100000 1000000
0
 
LVL 3

Expert Comment

by:sgayatri
ID: 12140642
Warlord,

Yu aren't attempting the code posted ...
I have been using it for quite some time without any problem.
Very large files also have been transfered thru that.

You just have to paste the code and change a few parameters ...

0
 

Author Comment

by:WarLord
ID: 12142207
i know sgayatri but i don't want to an external program i like michael his solution except is doesn't work here very well.
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12142228
How much time took to transfer 100Mb?
0
 

Author Comment

by:WarLord
ID: 12146267
the transfer speed is fine the point where it's going wrong is when you click a link and the server tries to load the file
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12147679
I think the problem in other place.
Could you post the code how do you obtain the file. What does it mean  "when you click a link and the server tries to load the file"?
0
 

Author Comment

by:WarLord
ID: 12149643
If Mid(strData, 1, 10) = "GET /file/" Then
strGet = InStr(strData, "GET ")
spc2 = InStr(strGet + 5, strData, " ")
page = Trim(Mid(strData, strGet + 10, spc2 - (strGet + 10)))
If Right(page, 1) = "/" Then page = Left(page, Len(page) - 1)

List1.AddItem "Page being requested: [" & page & "] by " & Socket(Index).RemoteHostIP

Open App.Path & "\log.txt" For Append As #1
    Write #1, Date & " " & Time & "   " & "Page being requested: [" & page & "] by " & Socket(Index).RemoteHostIP
    Close #1


Dim filesize As Long
Dim Transfered As Long
Dim CurrentPortion As String  ' <= was Long
Dim Portion_size As Long
Portion_size = 10000 '<= i 've redused size for testing

Open App.Path & "\" & page For Binary As #1
filesize = LOF(1)
Transfered = 0
CurrentPortion = ""
Do While Transfered < filesize ' <= Notice this change
       If Transfered + Portion_size > filesize Then
              CurrentPortion = Space$(filesize - Transfered)
       Else
              CurrentPortion = Space$(Portion_size)
       End If
        Get #1, , CurrentPortion
        Socket(Index).SendData CurrentPortion
        Transfered = Transfered + Len(CurrentPortion)
Loop

Close #1

End If


Hope this helps. the server checks which command is recieved then strips the recieved command writes information of the client to a log file and at last it opens the file and sends it to the client.  
0
 

Author Comment

by:WarLord
ID: 12187529
No one has an answer?
0
 
LVL 3

Expert Comment

by:sgayatri
ID: 12188344
Another answer also has been posted earlier. You aren't even attempting....
Good Luck !!!
0
 

Author Comment

by:WarLord
ID: 12189251
maybe it works but i don't like the solution.
0
 
LVL 13

Expert Comment

by:Michael_D
ID: 12562585
The code works fine for me.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Suggested Solutions

Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
Whether you’re a college noob or a soon-to-be pro, these tips are sure to help you in your journey to becoming a programming ninja and stand out from the crowd.
The goal of this video is to provide viewers with basic examples to understand and use conditional statements in the C programming language.
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…

759 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

26 Experts available now in Live!

Get 1:1 Help Now