Mad_Dennis
asked on
Searching data at high speed
Hi there,
I got a tough one for you guys out there, let's first start out with what my application is supposed to do.
My application acts as a barrier between a game server and player clients. It forwards all packages to the player so it can adjust the data on demand, everything is in UDP and for the server not to get confused for EACH player there has to be made a new socket on a new port. It requires reconizing technique's, at first i used a LIST ... which was not fast enough. Then i used a Database which AGAIN did not search fast enough, the thing is here: 'The server sends several dozen packages per second ... between 20-50 packages per second' for each package comming from the server the IP and Index of the socket the data has to be send to can be found easily by using a TAG in the winsocket. But when getting the data from a player (all players send their data to one main socket which they think is the server) the Index of the winsock which sends the data to the server has to be found. This has to happen at each package, and when not happening fast enough the client/server discards them as lost. Resulting in major lag and getting kicked from the server....
Here's a little piece of code to show you the situation. What i am looking for is an extremely fast search method that will do the trick ...
<----- Socket Code with Searching (FindIt2 is a seperate function to search in the database) ----->
If Index = 0 Then
'Main Server where does it originate from?
If Left$(Hexer, 8) = "00000000" Then
Winsock2(1).RemoteHost = "127.0.0.1"
Winsock2(1).RemotePort = 8912
pMes = Winsock2(Index).RemoteHost IP
pMesP = Winsock2(Index).RemotePort
Winsock2(1).SendData Data
Exit Sub '
End If
Dim Datam As Variant
Datam = FindIt2("Ip", Ipe).Socket
If Datam = "" Or Datam = 0 Then GoTo insiderr:
Winsock2(Datam).RemotePort = 8912
Winsock2(Datam).RemoteHost = "127.0.0.1"
Winsock2(Datam).SendData Data
Exit Sub
insiderr:
'It is not yet in the list
Load Winsock2(Winsock2.UBound + 1)
Data1.Recordset.AddNew
Data1.Recordset("Ip") = Winsock2(Index).RemoteHost IP
Data1.Recordset("Port") = Winsock2(Index).RemotePort
Data1.Recordset("Socket") = Winsock2.UBound
Data1.Recordset.Update
cPort = cPort + 1
Winsock2(Winsock2.UBound). LocalPort = cPort
Winsock2(Winsock2.UBound). Bind cPort
Winsock2(Winsock2.UBound). Tag = Winsock2(Index).RemoteHost IP & ":" & Winsock2(Index).RemotePort
Winsock2(Winsock2.UBound). RemoteHost = "127.0.0.1"
Winsock2(Winsock2.UBound). RemotePort = 8912
Winsock2(Winsock2.UBound). SendData Data
End If
<-------> FindIt2 function <------->
Public Type FindIt
Ip As String
Socket As Integer
Port As Integer
Name As String
End Type
Public Function FindIt2(Section As Variant, Value As Variant, Optional ChangeVal, Optional ChangeSection) As FindIt
On Error Resume Next
frmMain.Data1.RecordSource = "SELECT * FROM [Sup] WHERE [" & Section & "] ='" & Value & "'"
frmMain.Data1.Refresh
If frmMain.Data1.Recordset.EO F = True Then
Exit Function
End If
frmMain.Data1.Recordset.Mo veFirst
If Len(ChangeVal) > 0 And Len(ChangeSection) < 1 Then
frmMain.Data1.Recordset.Ed it
frmMain.Data1.Recordset(Se ction).Val ue = ChangeVal
End If
If Len(ChangeVal) > 0 And Len(ChangeSection) > 0 Then
frmMain.Data1.Recordset.Ed it
frmMain.Data1.Recordset(Ch angeSectio n).Value = ChangeVal
End If
FindIt2.Ip = frmMain.Data1.Recordset("I p")
FindIt2.Socket = frmMain.Data1.Recordset("S ocket")
FindIt2.Port = frmMain.Data1.Recordset("P ort")
FindIt2.Name = frmMain.Data1.Recordset("N ame")
frmMain.Data1.Recordset.Up date
End Function
I hope you guys can find a good and fast solution, remember that this tool will need to handle 18 to 20 players at a time so it has to search over a hundred times per second without loosing too much milliseconds.
Thanks in advance
I got a tough one for you guys out there, let's first start out with what my application is supposed to do.
My application acts as a barrier between a game server and player clients. It forwards all packages to the player so it can adjust the data on demand, everything is in UDP and for the server not to get confused for EACH player there has to be made a new socket on a new port. It requires reconizing technique's, at first i used a LIST ... which was not fast enough. Then i used a Database which AGAIN did not search fast enough, the thing is here: 'The server sends several dozen packages per second ... between 20-50 packages per second' for each package comming from the server the IP and Index of the socket the data has to be send to can be found easily by using a TAG in the winsocket. But when getting the data from a player (all players send their data to one main socket which they think is the server) the Index of the winsock which sends the data to the server has to be found. This has to happen at each package, and when not happening fast enough the client/server discards them as lost. Resulting in major lag and getting kicked from the server....
Here's a little piece of code to show you the situation. What i am looking for is an extremely fast search method that will do the trick ...
<----- Socket Code with Searching (FindIt2 is a seperate function to search in the database) ----->
If Index = 0 Then
'Main Server where does it originate from?
If Left$(Hexer, 8) = "00000000" Then
Winsock2(1).RemoteHost = "127.0.0.1"
Winsock2(1).RemotePort = 8912
pMes = Winsock2(Index).RemoteHost
pMesP = Winsock2(Index).RemotePort
Winsock2(1).SendData Data
Exit Sub '
End If
Dim Datam As Variant
Datam = FindIt2("Ip", Ipe).Socket
If Datam = "" Or Datam = 0 Then GoTo insiderr:
Winsock2(Datam).RemotePort
Winsock2(Datam).RemoteHost
Winsock2(Datam).SendData Data
Exit Sub
insiderr:
'It is not yet in the list
Load Winsock2(Winsock2.UBound + 1)
Data1.Recordset.AddNew
Data1.Recordset("Ip") = Winsock2(Index).RemoteHost
Data1.Recordset("Port") = Winsock2(Index).RemotePort
Data1.Recordset("Socket") = Winsock2.UBound
Data1.Recordset.Update
cPort = cPort + 1
Winsock2(Winsock2.UBound).
Winsock2(Winsock2.UBound).
Winsock2(Winsock2.UBound).
Winsock2(Winsock2.UBound).
Winsock2(Winsock2.UBound).
Winsock2(Winsock2.UBound).
End If
<-------> FindIt2 function <------->
Public Type FindIt
Ip As String
Socket As Integer
Port As Integer
Name As String
End Type
Public Function FindIt2(Section As Variant, Value As Variant, Optional ChangeVal, Optional ChangeSection) As FindIt
On Error Resume Next
frmMain.Data1.RecordSource
frmMain.Data1.Refresh
If frmMain.Data1.Recordset.EO
Exit Function
End If
frmMain.Data1.Recordset.Mo
If Len(ChangeVal) > 0 And Len(ChangeSection) < 1 Then
frmMain.Data1.Recordset.Ed
frmMain.Data1.Recordset(Se
End If
If Len(ChangeVal) > 0 And Len(ChangeSection) > 0 Then
frmMain.Data1.Recordset.Ed
frmMain.Data1.Recordset(Ch
End If
FindIt2.Ip = frmMain.Data1.Recordset("I
FindIt2.Socket = frmMain.Data1.Recordset("S
FindIt2.Port = frmMain.Data1.Recordset("P
FindIt2.Name = frmMain.Data1.Recordset("N
frmMain.Data1.Recordset.Up
End Function
I hope you guys can find a good and fast solution, remember that this tool will need to handle 18 to 20 players at a time so it has to search over a hundred times per second without loosing too much milliseconds.
Thanks in advance
ASKER
Oh no just ignore my above post ... i'm being dumb again, isn't there an 'edit' function to edit posts?
Everything you gotta know is in the Question
Everything you gotta know is in the Question
A few suggestions to speed-up your code...
1)
How many columns are in the [Sup] table? Do you really need to "SELECT *"?
Why not just select the columns (fields) you need?
2)
Is the column referred to in the "Section" variable an indexed field?
3)
Why are you returning 4 items in the User-Defined Type return of FindIt2 routine, if you are only ever using the "Socket" [Datam = FindIt2("Ip", Ipe).Socket]? If you just return the "Socket" (As Integer) rather than as part of the "FindIt" user-defined type?
BFN,
fp.
1)
How many columns are in the [Sup] table? Do you really need to "SELECT *"?
Why not just select the columns (fields) you need?
2)
Is the column referred to in the "Section" variable an indexed field?
3)
Why are you returning 4 items in the User-Defined Type return of FindIt2 routine, if you are only ever using the "Socket" [Datam = FindIt2("Ip", Ipe).Socket]? If you just return the "Socket" (As Integer) rather than as part of the "FindIt" user-defined type?
BFN,
fp.
ASKER
I don't think that will make much of a difference, searching in a database just goes to slow.
I only need the Socket and i need to search on the user's IP. But searching in a Database goes way to slow, it isn't the code, it's the microsoft database that can't handle several dozen searches per second.... This is a matter of miliseconds i don't want a laggy server with my 10 MBit connection.
I only need the Socket and i need to search on the user's IP. But searching in a Database goes way to slow, it isn't the code, it's the microsoft database that can't handle several dozen searches per second.... This is a matter of miliseconds i don't want a laggy server with my 10 MBit connection.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
As you said, each player should have a dedicated (or a shared, with low contention ratio) port number.
Do you have concurrently running datagram handlers associated to each port allocated, so that each process hanging off each port only handles a subset of the throughput of 20-50 packets per second, or do you have one application reading each port number in turn? In the latter case, the speed of execution will suffer. I would therefore suggest the former option, if you do not implement this approach already.
Yes, as Idle_Mind suggests, a collection may be a way forward... but no, you do not need to implement as a class.
However, I am concerned that your network traffic is too great anyway, and of course the speed of lookup is still constrained by the speed of the slowest component in the chain; the network.
UDP is also not 100% perfect, in that you are never sure you have received every packet; unless you set-up a content-based protocol so that you know what to expect & when, and hence know if you have missed a packet & can request a re-send.
This in turn adds to the speed of execution.
Presumably this application is running on a dedicated server?
BFN,
fp.
Do you have concurrently running datagram handlers associated to each port allocated, so that each process hanging off each port only handles a subset of the throughput of 20-50 packets per second, or do you have one application reading each port number in turn? In the latter case, the speed of execution will suffer. I would therefore suggest the former option, if you do not implement this approach already.
Yes, as Idle_Mind suggests, a collection may be a way forward... but no, you do not need to implement as a class.
However, I am concerned that your network traffic is too great anyway, and of course the speed of lookup is still constrained by the speed of the slowest component in the chain; the network.
UDP is also not 100% perfect, in that you are never sure you have received every packet; unless you set-up a content-based protocol so that you know what to expect & when, and hence know if you have missed a packet & can request a re-send.
This in turn adds to the speed of execution.
Presumably this application is running on a dedicated server?
BFN,
fp.
...and of course an obvious suggestion is to not use VB, but to use C++, Java, or even Assembler, if speed of execution is critical.
PS. What database technology are you using?
PS. What database technology are you using?
ASKER
I used a Microsoft Access database, i'm now looking into collections...
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
First of all i post this reply very quickly i'll come back to this later. My internet has been down alot lately. I yet haven;t been able to test it ....
I'll tell you when i did
I'll tell you when i did
That's fine with me, thanks Gerry.
ASKER
---------------
If Index <> 0 Then
'Probably comming from the server
If Left$(Hexer, 4) = "0001" And Mid$(Hexer, 7, 4) = "0091" Then
Winsock2(0).RemoteHost = Left$(Winsock2(Index).Tag,
Winsock2(0).RemotePort = Mid$(Winsock2(Index).Tag, InStr(Winsock2(Index).Tag,
Winsock2(0).SendData Data
FindIt2 "Socket", Index, 0, "Port"
FindIt2 "Socket", Index, " ", "Ip"
FindIt2 "Socket", Index, 0, "Socket"
Exit Sub
End If
Winsock2(0).RemoteHost = Left$(Winsock2(Index).Tag,
Winsock2(0).RemotePort = Mid$(Winsock2(Index).Tag, InStr(Winsock2(Index).Tag,
Winsock2(0).SendData Data
End If
--------------------------