• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 301
  • Last Modified:

Searching data at high speed

Hi there,

I got a tough one for you guys out there, let's first start out with what my application is supposed to do.

My application acts as a barrier between a game server and player clients. It forwards all packages to the player so it can adjust the data on demand, everything is in UDP and for the server not to get confused for EACH player there has to be made a new socket on a new port. It requires reconizing technique's, at first i used a LIST ... which was not fast enough. Then i used a Database which AGAIN did not search fast enough, the thing is here: 'The server sends several dozen packages per second ... between 20-50 packages per second' for each package comming from the server the IP and Index of the socket the data has to be send to can be found easily by using a TAG in the winsocket. But when getting the data from a player (all players send their data to one main socket which they think is the server) the Index of the winsock which sends the data to the server has to be found. This has to happen at each package, and when not happening fast enough the client/server discards them as lost. Resulting in major lag and getting kicked from the server....

Here's a little piece of code to show you the situation. What i am looking for is an extremely fast search method that will do the trick ...

<----- Socket Code with Searching (FindIt2 is a seperate function to search in the database) ----->

If Index = 0 Then
    'Main Server where does it originate from?
        If Left$(Hexer, 8) = "00000000" Then
            Winsock2(1).RemoteHost = ""
            Winsock2(1).RemotePort = 8912
            pMes = Winsock2(Index).RemoteHostIP
            pMesP = Winsock2(Index).RemotePort
            Winsock2(1).SendData Data
            Exit Sub '
        End If
        Dim Datam As Variant
        Datam = FindIt2("Ip", Ipe).Socket
        If Datam = "" Or Datam = 0 Then GoTo insiderr:
        Winsock2(Datam).RemotePort = 8912
        Winsock2(Datam).RemoteHost = ""
        Winsock2(Datam).SendData Data
    Exit Sub
        'It is not yet in the list
        Load Winsock2(Winsock2.UBound + 1)
        Data1.Recordset("Ip") = Winsock2(Index).RemoteHostIP
        Data1.Recordset("Port") = Winsock2(Index).RemotePort
        Data1.Recordset("Socket") = Winsock2.UBound

        cPort = cPort + 1
        Winsock2(Winsock2.UBound).LocalPort = cPort
        Winsock2(Winsock2.UBound).Bind cPort
        Winsock2(Winsock2.UBound).Tag = Winsock2(Index).RemoteHostIP & ":" & Winsock2(Index).RemotePort
        Winsock2(Winsock2.UBound).RemoteHost = ""
        Winsock2(Winsock2.UBound).RemotePort = 8912
        Winsock2(Winsock2.UBound).SendData Data
End If

<-------> FindIt2 function <------->

Public Type FindIt
  Ip                          As String
  Socket                      As Integer
  Port                        As Integer
  Name                        As String
End Type

Public Function FindIt2(Section As Variant, Value As Variant, Optional ChangeVal, Optional ChangeSection) As FindIt
On Error Resume Next
frmMain.Data1.RecordSource = "SELECT * FROM [Sup] WHERE [" & Section & "] ='" & Value & "'"

If frmMain.Data1.Recordset.EOF = True Then
    Exit Function
End If


If Len(ChangeVal) > 0 And Len(ChangeSection) < 1 Then
    frmMain.Data1.Recordset(Section).Value = ChangeVal
End If

If Len(ChangeVal) > 0 And Len(ChangeSection) > 0 Then
    frmMain.Data1.Recordset(ChangeSection).Value = ChangeVal
End If

FindIt2.Ip = frmMain.Data1.Recordset("Ip")
FindIt2.Socket = frmMain.Data1.Recordset("Socket")
FindIt2.Port = frmMain.Data1.Recordset("Port")
FindIt2.Name = frmMain.Data1.Recordset("Name")
End Function

I hope you guys can find a good and fast solution, remember that this tool will need to handle 18 to 20 players at a time so it has to search over a hundred times per second without loosing too much milliseconds.

Thanks in advance

3 Solutions
Mad_DennisAuthor Commented:
Oh BTW, the searching has to be done in this code ... put the wrong code above


If Index <> 0 Then
    'Probably comming from the server
        If Left$(Hexer, 4) = "0001" And Mid$(Hexer, 7, 4) = "0091" Then
            Winsock2(0).RemoteHost = Left$(Winsock2(Index).Tag, InStr(Winsock2(Index).Tag, ":") - 1)
            Winsock2(0).RemotePort = Mid$(Winsock2(Index).Tag, InStr(Winsock2(Index).Tag, ":") + 1)
            Winsock2(0).SendData Data
            FindIt2 "Socket", Index, 0, "Port"
            FindIt2 "Socket", Index, " ", "Ip"
            FindIt2 "Socket", Index, 0, "Socket"
            Exit Sub
        End If
        Winsock2(0).RemoteHost = Left$(Winsock2(Index).Tag, InStr(Winsock2(Index).Tag, ":") - 1)
        Winsock2(0).RemotePort = Mid$(Winsock2(Index).Tag, InStr(Winsock2(Index).Tag, ":") + 1)
        Winsock2(0).SendData Data
End If

Mad_DennisAuthor Commented:
Oh no just ignore my above post ... i'm being dumb again, isn't there an 'edit' function to edit posts?

Everything you gotta know is in the Question
[ fanpages ]IT Services ConsultantCommented:
A few suggestions to speed-up your code...

How many columns are in the [Sup] table?  Do you really need to "SELECT *"?
Why not just select the columns (fields) you need?

Is the column referred to in the "Section" variable an indexed field?

Why are you returning 4 items in the User-Defined Type return of FindIt2 routine, if you are only ever using the "Socket" [Datam = FindIt2("Ip", Ipe).Socket]?  If you just return the "Socket" (As Integer) rather than as part of the "FindIt" user-defined type?


Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Mad_DennisAuthor Commented:
I don't think that will make much of a difference, searching in a database just goes to slow.

I only need the Socket and i need to search on the user's IP. But searching in a Database goes way to slow, it isn't the code, it's the microsoft database that can't handle several dozen searches per second.... This is a matter of miliseconds i don't want a laggy server with my 10 MBit connection.
Mike TomlinsonMiddle School Assistant TeacherCommented:
Have you considered using a Collection?  You can store items in a collection by key, which allows you to very quickly pull that same item back out via the key.  If your dataset is not too large then this may be the way to go.  I believer you would need to convert your FindIt type to a class instead though.
[ fanpages ]IT Services ConsultantCommented:
As you said, each player should have a dedicated (or a shared, with low contention ratio) port number.

Do you have concurrently running datagram handlers associated to each port allocated, so that each process hanging off each port only handles a subset of the throughput of 20-50 packets per second, or do you have one application reading each port number in turn?  In the latter case, the speed of execution will suffer.  I would therefore suggest the former option, if you do not implement this approach already.

Yes, as Idle_Mind suggests, a collection may be a way forward... but no, you do not need to implement as a class.

However, I am concerned that your network traffic is too great anyway, and of course the speed of lookup is still constrained by the speed of the slowest component in the chain; the network.

UDP is also not 100% perfect, in that you are never sure you have received every packet; unless you set-up a content-based protocol so that you know what to expect & when, and hence know if you have missed a packet & can request a re-send.

This in turn adds to the speed of execution.

Presumably this application is running on a dedicated server?


[ fanpages ]IT Services ConsultantCommented:
...and of course an obvious suggestion is to not use VB, but to use C++, Java, or even Assembler, if speed of execution is critical.

PS. What database technology are you using?
Mad_DennisAuthor Commented:
I used a Microsoft Access database, i'm now looking into collections...
using a multithreaded architecture (i.e. not vb) in the first place, would be of significant benefit..
[ fanpages ]IT Services ConsultantCommented:
FYI: A collection can be created from a recordset, initially, if desired, then used as a discrete object thereafter.

Here is a question I responded to recently on this subject:
Mad_DennisAuthor Commented:
First of all i post this reply very quickly i'll come back to this later. My internet has been down alot lately. I yet haven;t been able to test it ....

I'll tell you when i did
[ fanpages ]IT Services ConsultantCommented:
That's fine with me, thanks Gerry.

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now