XK8ER
asked on
Using loop to find low prices
Hello,
I am trying to find the fastest possible way to loop through a ConcurrentDictionary to find low prices in the market.
I am averaging 450 loops per second on a computer using 8 cores. Someone suggested to me to look into (intel xeon phi) with 60 cores.
the problem with that is that I don't know if it would work with programming languages like vb.net or c#.
in this example I am using regular For Each and also using Parallel.ForEach with a bit higher load.
I need to find a way to loop through this dictionary about 5,000 times per second with low CPU load and not just 450p/s..
if new hardware is required then I'm open for any suggestions.
I am trying to find the fastest possible way to loop through a ConcurrentDictionary to find low prices in the market.
I am averaging 450 loops per second on a computer using 8 cores. Someone suggested to me to look into (intel xeon phi) with 60 cores.
the problem with that is that I don't know if it would work with programming languages like vb.net or c#.
in this example I am using regular For Each and also using Parallel.ForEach with a bit higher load.
I need to find a way to loop through this dictionary about 5,000 times per second with low CPU load and not just 450p/s..
if new hardware is required then I'm open for any suggestions.
Option Explicit On
Option Strict On
Imports System.Collections.Concurrent
Public Class Form1
Public StoredStockPrices As New ConcurrentDictionary(Of Integer, StockPrices)
Dim r As New Random
Dim iLoops As Boolean
Dim iTotalLoopsPM As Long
Private Sub Form1_Closing(sender As Object, e As System.ComponentModel.CancelEventArgs) Handles MyBase.Closing
iLoops = False
End Sub
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
If StoredStockPrices.Count = 0 Then
For x = 1 To 15000
StoredStockPrices(x) = New StockPrices(r.Next, r.Next.ToString, r.Next.ToString, r.Next.ToString,
r.Next.ToString, r.Next, r.Next, True,
New TimeSpan(1, 0, 0), #07/29/2016 2:00 PM#, #07/30/2016 2:00 PM#, r.Next, r.Next,
r.Next, r.Next, r.Next)
Next
End If
Debug.Print(StoredStockPrices(1).Name.ToString & "|" & StoredStockPrices.Count)
End Sub
Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
Call StartLoops()
End Sub
Public Sub StartLoops()
iLoops = True
Dim iThreadLoops As New Threading.Thread(New Threading.ThreadStart(AddressOf threadLoops))
iThreadLoops.Start()
End Sub
Private Sub threadLoops()
If StoredStockPrices.Count > 0 Then
Do While iLoops = True
Dim copyStoredStockPrices As New ConcurrentDictionary(Of Integer, StockPrices)
copyStoredStockPrices = StoredStockPrices
'360 p/s with 50% CPU usage
Threading.Tasks.Parallel.ForEach(copyStoredStockPrices.Values.Reverse, Sub(myStockPrices)
Call CheckLowPrices(myStockPrices)
End Sub)
'450 p/s with 20% CPU usage
'For Each myStockPrices In copyStoredStockPrices.Values.Reverse
' Call CheckLowPrices(myStockPrices)
'Next
iTotalLoopsPM += 1
copyStoredStockPrices = Nothing
Threading.Thread.Sleep(1)
Loop
End If
End Sub
Private Sub CheckLowPrices(myStockPrices As StockPrices)
If myStockPrices.StrikePrice < 10 AndAlso myStockPrices.Index = 10 Then
If myStockPrices.Sequence < 500 AndAlso myStockPrices.Index < 20 Then
'submit order
End If
End If
End Sub
Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick
Dim PerSec As Double = (iTotalLoopsPM / Date.UtcNow.Second)
Label1.Text = Math.Round(PerSec, 0) & "p/s"
Label3.Text = Math.Round(iTotalLoopsPM, 0) & "p/m"
If Date.UtcNow.Second = 0 Then
iTotalLoopsPM = 0
End If
Label2.Text = Date.UtcNow.ToString
'Stock prices are always changing, hundreds of times per second.
If StoredStockPrices.Count > 0 Then
For Each eachRecord In StoredStockPrices
StoredStockPrices(eachRecord.Key) = New StockPrices(r.Next, r.Next.ToString, r.Next.ToString, r.Next.ToString,
r.Next.ToString, r.Next, r.Next, True,
New TimeSpan(1, 0, 0), #07/29/2016 2:00 PM#, #07/30/2016 2:00 PM#, r.Next, r.Next,
r.Next, r.Next, r.Next)
Next
End If
End Sub
End Class
Public Class StockPrices
Private m_Index As Integer
Private m_Symbol As String
Private m_Name As String
Private m_ContractStatus As String
Private m_UnderlyingSymbol As String
Private m_UnderlyingSettledPrice As Double
Private m_StrikePrice As Double
Private m_HasTraded As Boolean
Private m_LifeSpan As TimeSpan
Private m_OpensAt As DateTime
Private m_ClosesAt As DateTime
Private m_CeilingPrice As Double
Private m_FloorPrice As Double
Private m_Latency As Long
Private m_Sequence As Integer
Private m_TickMath As Double
Public Sub New(ByVal iIndex As Integer, ByVal iSymbol As String, ByVal iName As String, ByVal iContractStatus As String,
ByVal iUnderlyingSymbol As String, ByVal iUnderlyingSettledPrice As Double, ByVal iStrikePrice As Double,
ByVal iHasTraded As Boolean, ByVal iLifeSpan As TimeSpan, ByVal iOpensAt As DateTime, ByVal iClosesAt As DateTime,
ByVal iCeilingPrice As Double, ByVal iFloorPrice As Double, ByVal iLatency As Long, ByVal iSequence As Integer,
ByVal iTickMath As Double)
Me.m_Index = iIndex
Me.m_Symbol = iSymbol
Me.m_Name = iName
Me.m_ContractStatus = iContractStatus
Me.m_UnderlyingSymbol = iUnderlyingSymbol
Me.m_UnderlyingSettledPrice = iUnderlyingSettledPrice
Me.m_StrikePrice = iStrikePrice
Me.m_HasTraded = iHasTraded
Me.m_LifeSpan = iLifeSpan
Me.m_OpensAt = iOpensAt
Me.m_ClosesAt = iClosesAt
Me.m_CeilingPrice = iCeilingPrice
Me.m_FloorPrice = iFloorPrice
Me.m_Latency = iLatency
Me.m_Sequence = iSequence
Me.m_TickMath = iTickMath
End Sub
Public Property Index() As Integer
Get
Return m_Index
End Get
Set
m_Index = Value
End Set
End Property
Public Property Symbol() As String
Get
Return m_Symbol
End Get
Set
m_Symbol = Value
End Set
End Property
Public Property Name() As String
Get
Return m_Name
End Get
Set
m_Name = Value
End Set
End Property
Public Property ContractStatus() As String
Get
Return m_ContractStatus
End Get
Set
m_ContractStatus = Value
End Set
End Property
Public Property UnderlyingSymbol() As String
Get
Return m_UnderlyingSymbol
End Get
Set
m_UnderlyingSymbol = Value
End Set
End Property
Public Property UnderlyingSettledPrice() As Double
Get
Return m_UnderlyingSettledPrice
End Get
Set
m_UnderlyingSettledPrice = Value
End Set
End Property
Public Property StrikePrice() As Double
Get
Return m_StrikePrice
End Get
Set
m_StrikePrice = Value
End Set
End Property
Public Property LifeSpan() As TimeSpan
Get
Return m_LifeSpan
End Get
Set
m_LifeSpan = Value
End Set
End Property
Public Property HasTraded() As Boolean
Get
Return m_HasTraded
End Get
Set
m_HasTraded = Value
End Set
End Property
Public Property OpensAt() As DateTime
Get
Return m_OpensAt
End Get
Set
m_OpensAt = Value
End Set
End Property
Public Property ClosesAt() As DateTime
Get
Return m_ClosesAt
End Get
Set
m_ClosesAt = Value
End Set
End Property
Public Property CeilingPrice() As Double
Get
Return m_CeilingPrice
End Get
Set
m_CeilingPrice = Value
End Set
End Property
Public Property FloorPrice() As Double
Get
Return m_FloorPrice
End Get
Set
m_FloorPrice = Value
End Set
End Property
Public Property Latency() As Long
Get
Return m_Latency
End Get
Set
m_Latency = Value
End Set
End Property
Public Property Sequence() As Integer
Get
Return m_Sequence
End Get
Set
m_Sequence = Value
End Set
End Property
Public Property TickMath() As Double
Get
Return m_TickMath
End Get
Set
m_TickMath = Value
End Set
End Property
End Class
have you run your code through a profiler and found any bottlenecks?
ASKER
yes I have used profiler but the issue is not bottlenecks is speed
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
there are two connections to two different market data feed.
the function CheckLowPrices is just something very similar to what we have in place.
example:
data feed #1 says EUR/USD contract 1.1166 is selling for $50
data feed #2 says the underlying price for EUR/USD is at 1.11652
if we only check one time it could work yes but sometimes data feed #1 doesn't move for several second so if its a good price according to data feed #2 then we miss out the window to submit a new order, and when market data #1 moves again its already not a good price
the function CheckLowPrices is just something very similar to what we have in place.
example:
data feed #1 says EUR/USD contract 1.1166 is selling for $50
data feed #2 says the underlying price for EUR/USD is at 1.11652
if we only check one time it could work yes but sometimes data feed #1 doesn't move for several second so if its a good price according to data feed #2 then we miss out the window to submit a new order, and when market data #1 moves again its already not a good price