• Status: Solved
• Priority: Medium
• Security: Public
• Views: 332

Managing huge picture boxes in VB 6

Hi Everyone,

Short version of the question:
Is there any way to make a picturebox control 15,000 pixels square or bigger? Do I just need to have vb allocate more memory or are there some fundamental limitations to doing this? Given the longer explanation below, is there a better or faster way to store and access the data I need?

Background:
Why would anyone need a picturebox that big? Well, I'm building a model of how army ants swarm for part of my PhD. These ants are blind and find their way around by leaving trails made out of chemicals called “pheromones”.  The ants swarm 100 meters or more in a day and they move around on a 1mm scale so this makes a lot of numbers to be tracked. Basic model is this: each ant samples the “pheromone” values from the area in front of it with its antennae, it moves, then it increments the value of the “pheromone” at its new location. Pretty simple except for the huge scale. (More background if you’re interested at: http://www.infiniteworld.org/research/research.htm)

More detailed explanation of the problem(s):
1) Pheromone map: I'd like to be able to create an array of about 225 million integer values (i.e. that’s the size of an area of 15,000 x 15,000) that can be quickly accessed with x/y coordinates. That is, an ant at a given x/y position needs to be able to get the value of the point where it is and leave a new value at a nearby point (and maybe a few adjacent points). If it makes any difference, the data will be mostly zeros. Since 200 million 2byte integers is only about 425 MB of memory it doesn’t seem unreasonable that this could be done somehow.

2) Plotting and Zooming: To be able to debug the program, I need to be able to see what the ants are seeing by quickly plotting a visual representation of this data to the screen and to be able to "zoom in" to smaller sections of the ant-world and see the pheromone map and ants there. This means that some (possibly lower resolution) version of the pheromone has to be stored as an image.

3) Time constraint: Ideally a single loop of 4000 or so ants would take less than 30millisec since that would be "realtime" for the model, i.e. the model needs to run for a few hours in "ant time" and the model time step is 1/30 of a second so loop times longer than 30msec would make models runs take hours.

Things I’ve tried:
Change scalemode -- In a previous version I was just setting the “scalemode” to whatever width I wanted but I’m pretty sure VB only tracks color at the pixel scale so that doesn’t work <-- Anyone know this for sure?

Scaling the picturebox to worldsize -- The simplest solution seemed to be to have the ants leave their "pheromone" as a change to color in a PictureBox Control since that way I can just look at the image to see what's going on, and the ant can just sample the x/y value of where it is and increment the color values to add pheromone. But making pictureboxes more than 4000 pixels wide gives the dreaded “Error ‘480’ - Can’t create AutoRedraw” memory failure. <-- Is there an easy API way to allocate more memory; will VB try and make it as one consecutive chunk and crash?

A solution that kind of worked: The ants store their pheromone values in a set of arrays (which have to be allocated in smaller chunks to prevent problems with oversized blocks of memory) and then have them also leave their pheromone value on a small picture box just for the visual representation (since the picture only has to be visually accurate, not numerically accurate).

So, the main questions:
1) Can this be done using picture boxes? (So far I’m not sure if it isn’t working because of basic limitation of VB or just because of memory allocation problems that could be fixed)
2) Is there a better or faster way?

This is a big question, so I’m mainly looking for semi-specific suggestions of directions to take and answers to the last two questions.

Thanks!
Tim
0
Intersection
1 Solution

Commented:
Are you getting any run time errors or what? I was able to make my picture box that big with no RTE. The size you need will go beyond the monitor so you may need to scale down.
0

Author Commented:
It doesn't seem to choke until you actually call it ... try accessing a point at the end of the box. 4,000px seems to be as big as it can get.  I assume this is because VB hasn't set aside enough memory - so is there some way to warn it to make a big block before you make the pictureBox huge?
This should crash it:
Dim i as long,  pix As Long
For i = 1 To 15000
pix = GetPixel(picture.hdc, i, i) '(or picture.point(x,y))
DoEvents
Next
0

Commented:
Hi Intersection,
I've lead a project similar to your case that required huge data to be stored and displayed on screen using a picturebox. From our experience this kind of projects should use rendering techniques that will render only the visible part of data, that means if I had a picturebox that has a size of 400x400 I will render only a 400x400 picture with the desired offset and no need for a huge picturebox.

Then came the data storage and manipulating issue. This issue is best approached using OOP. That means designing classes that represents your system and interfacing them with each other to communicate. A class you may need is an Ant Class: clsAnt. This class will have the properties of an ant and the path data (pheromones) as an array of a user defined structure (point):

Type AntPoint
X as Integer
Y as Integer
Value as Integer
End Type

And the array will look like this: Dim AntPath() as AntPoint. You can always use a collection instead of an array because arrays need much more coding and experience to deal with them, but give better performance and less overhead.

In your array you only store the nonzero values (The path) that will reduce the data size dramatically.

And for the display you only clear the picturebox to a white screen and then draw the points of the path that is visible from the AntPath array.

In this case you have a small bitmap drawn and a small sized array.

As for the zoom issue, to implement it you simply can control the size of the point you are drawing on screen, and for every zoom change you need to calculate the size of the visible data that is represented in the picturebox.

This is a very brief explanation of an approach for your project. If you have any further questions just ask, and maybe we know each other
0

Author Commented:

Hi,

Thanks for the suggestions.

A few questions...
1 - So you've tried to make huge PictureBoxes and found it doesn't work?

2 - Classes: Way back the first time I wrote this program, I did it with an array that held every pheromone value and each ant just searched through the entire thing to see if there was data at its current point.  If I remember correctly, this got pretty slow really fast because each ant had to search through the entire array which could be 10,000 points long or longer and there was no way to order the values sequentially to speed things up. Perhaps there is a way around this?

I was thinking this approach might work if I could store values in a collection as an ordered stack somehow (i.e. sorted by small to large X values or something) then the ant could just go to the part of the stack where it was and look at adjacent values. Is this what you were thinking? Is it faster to search through a big collection than a big array?

0

Commented:
This link shows how you can map an array to the memory used by a picture box.

http://www.visualbasicforum.com/t25347.html

I modiefied the example to map a udt array to the image so that  you can get at each colour without maths.  But even a normal sized picture takes about a second to process so to obtain the speeds that you need you will need to pass the pointer to the array memory to a C++ DLL.

The advantage of C++ is that it is a lot faster, but to achieve the speeds you need C++ also supports the ASM command.

Using ASM you can write machine code instructions (Assembler) this is the only way you will get to your required speeds.

Further, there is no pint in display the whole image, once you have an array image of the data you only need to specific sections so create a virtual image, like Google Earth you only display the bit you need.

0

Commented:
Hi
Sorry for the delay, but my ISP had some technical problems so I was offline for a while
First of all the answer to your first question is yes. VB has a restriction on the size of a picturebox so huge pictureboxes isn't an option. Go for visible data process only.

As for the arrays and collections stuff, consider the following array

0000001001000000000000000
0000001111000000000000000
0000001001000000000000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000
0000000000000000000000000

Method 1 Storing the digits as is in a two dimension array:
If the above data is stored in an array and each digit reserved a byte the array size will be about 225 bytes (Ignoring array's type data)

Method 2 Storing the 1s coordinates as UDT in the array:
If we store the 1s coordinates it will look like this
(0,6), (1,6), (2,6), (1,7), (1,8), (0,9), (1,9), (2,9) and each element will take up to 4 bytes (two integers) and that will be a total of 32 bytes so its less memory usage than the above, but its not always that way. If the ones are more than 25% of all the digits then it will consume more memory.

But we can always store the digits in the first method as bits (needs more coding) that will consume 29 bytes. So in you question 425 megas will shrink to 26.6 megas

Now we'll look at the performance issue:

In method 1 we have a lot of iterations to go through (225) when processing data where we use bits or bytes.

In method 2 we have only 8 iterations because everything other than the coordinates we have are zeros so no need to list them

You can benefit from the key property of the collection to access the desired item quickly if you set it right.

And yes I go with inthedark to use ASM within VB for fast performance
0

Author Commented:
HI everyone,

Thanks for the comments and sorry for the delay -- didn't have time until today to implement the suggestions.

inthedark, I've tried DIB sections, etc as suggested in the link but get outofmemory errors still and since a lot of the calls involve loops, they end up being too slow anyway.  C++ would clearly be the way to go but alas I have a very tight deadline so I don't have the time to learn how.

iHadi, I tried using a UDT, actually I tried the Dictionary type which works great because you can make the key be anything you want so I can just complex the x/y values to make unique keys (i.e. x= 135, y = 26 goes to "135 26" as a key). This works very well and memory only gets as big as the amount of data I store in the dictionary.  Turns out it is still to slow though, which answer my question -- no it can't be done.  So I've re-worked the model and will do it on a smaller spatial scale using pixels and the get/setpixel api's to draw them.  This seems to be fast enough and work pretty well.

Thanks for all the suggestions.

Cheers,

Tim

Just for reference, here's the test loop I wrote:

Option Explicit

Dim pher As New Dictionary

'API's
Private Declare Function GetTickCount Lib "kernel32" () As Long
Private Declare Function SetPixel Lib "gdi32" (ByVal hDC As Long, ByVal x As Long, ByVal y As Long, ByVal Color As Long) As Long

Private Sub cmdTest_Click()

Dim x As Long
Dim y As Long
Dim pherval, i
Dim tailxy As String
Dim looptime As Single
Dim numloops As Long

Randomize

looptime = GetTickCount()

numloops = 1000

For i = 1 To numloops

'Pick a random x and y values (scalemode is pixels)
x = Round(Rnd * Picture1.Width)
y = Round(Rnd * Picture1.Height)

'Generate the unique key (very simple,
' just the x and y values with a space between them):
tailxy = x & " " & y

If Not (pher.Exists(tailxy)) Then 'Check that the key doesn't exist left
'If no pheromone in that spot, add some
pherval = 50
Else
' If key value does exist, get current value of pixel and increment it
pherval = (pher.Item(tailxy) + 50)
pher.Item(tailxy) = pherval
'    If pherval > 256 Then pherval = 256

End If
' Leave a mark on the screen
SetPixel Picture1.hDC, x, y, RGB(pherval, 0, 0)
DoEvents
If i Mod 1000 = 0 Then txtCurLoop.Text = i
Next

Picture1.Refresh 'Need to refresh for setpixel to show up

txtLoopTime.Text = (GetTickCount() - looptime) ' Report loop time in milliseconds
txtTypeSize.Text = pher.Count 'Report size of "pher"

End Sub

0

Featured Post

Tackle projects and never again get stuck behind a technical roadblock.