Displaying lot of points - How ?

Posted on 2011-05-02
Last Modified: 2013-12-06
Hello experts !
I have an OpenGL view in which I would like to display points.
So far, this is something I can handle ;)

For every point, I have its coordinates (X ; Y ; Z) and a value (unsigned char).
I have a color array giving the link between one value and a color.
For example, 255 is red, 0 is blue, and so on...
The problem is that since I would like that dynamically, I can change a threshold to "hide/show" the points depending on their corresponding value and as I will have lot of points to display (maximum 5 billions but in a standard usage, maybe 1~2 millions), I am worried that the performance will be pretty bad.

So I am now looking for the best way to handle this.
I am interested in the VBO. I have read that it will allow some good performance and also that I can modify the buffer as I want without recalculating it from scratch (as with display list).

1) Do you have other ideas ?
2) With VBO I don't understand how it is possible to select the color of the points in the buffers. How to do that ?

Thanks in advance !
Question by:DBTechnique
    LVL 12

    Expert Comment

    1~2 million is a lot of points, no matter how you render them.

    VBO's will help because you can store the VBO in graphics memory, saving the time taken to transfer the data across the graphics bus. Also like vertex array and display lists they save the time the program would spend calling gl functions to draw the points.

    However, that may not help in this case.  When you modify the VBO, the changes have to be sent across the bus.  There is no real gain if you modify the VBO frequenly.  Also you have so many points that they might not fit in graphics memory anyway.  You might not be able to create VBO's that big, or they may be kept in system memory and transferred across the bus when needed.  I'm certain that 5 billion points is beyond the capacity of any graphics hardware I know of.

    The buffer object can be set to stream mode, and that will probably be fast.  However GL gives no guarantee about the speed, only that it will work, as stated here Buffer Object Streaming Reference.  I imagine this is especially true with large VBOs.

    You could group the points into 256 VBO's and render only the one that should be visible.  Though in much the same way, you could put them all in a vertex array (sorted by color) and then render parts of it.

    Not drawing points that are outside of the view frustrum might be useful.  Assuming the program allows the user to focus on a area, rather than the whole cloud.

    Can you use shaders in the context of this program?  The reason I ask is that shaders can modify the colour of the points as they are rendered.  This moves a lot of the update work to the GPU.  If you can use shaders, the problem is sending the vertex data to the card as fast as possible, the color problem is fixed.

    When it gets to the 5 billions points range, could you clump them together somehow?  If you have 1000 points in almost the same place, you could probably render that as one slightly larger point.  I'd guess that even with a high resolution display, it's unlikely those 1000 points will be discernable.

    Author Comment

    Hi Satsumo.
    Thank you really much for this long and detailed reply :)

    I don't have any limitations regarding the tools that can be used.
    Hence I am totaly open-minded regarding this question.
    Let me give more information about the context around the needed implementation.

    I have a 3D space (XYZ).
    X and Y have a maximum range of 2500.0 millimeters.
    Z has a maximum range of 800.0 millimeters.
    Then from a bunch of file, i will read the points' data.
    As I said in the first post, the data contain the X/Y/Z coordinates, and an 8 bits value for each points.
    From a GUI attached to the display, I have tools and parameters to select which data to display (for example, select one region, set a threshold on the value, ...).
    The limitation I have is that the step on XYZ is "minimum" one millimeter.
    As you said, anyway there is no need to display everything since the screen doesn't have this kind of resolution.

    Regarding the shaders, I am not sure to understand your explanation.
    But maybe we can discuss about it later,

    So I would like to split my previous question to two different questions.
    One is about the way to reduce the number of points and the second would to about how to use the VBO with the previous constraint of huge amount of points theoretically solved with a solution for the first question.

    1) How to reduce the number of points to be displayed ?
    At first, I can define a bigger step. But I guess that even with this, the points' quantity can still be huge.
    Then I can define the step depending on the screen resolution and the zoom applied in the view.
    Is there a standard way to implement that on OpenGL ?

    2) How to use VBO for this implementation ?
    I have posted the same question on StackOverflow

    The answer that interested me the most was from Kvark :

    "Put everything into one VBO. Draw it as an array of points: glDrawArrays(GL_POINTS,0,num). Calculate alpha in a pixel shader (using threshold passed as uniform).
    If you want to change a small subset of points - you can map a sub-range of the VBO. If you need to update large parts frequently - you can use Transform Feedback to utilize GPU."

    I read some tutorial about the VBO but I am confused about how to use it regarding the colors.
    Do you have any code snippet or information about this ?
    LVL 12

    Accepted Solution

    I think it would be a good idea to make part one of the problem into a separate question.  I have seen related questions in other areas.  The experts who answered had a lot of background knowledge, but they wouldn't know OpenGL

    Part two of your question.  I think the shaders might be woth discussing here. You don't actually need to send color data.  The decription suggests a range of colour from red at extreme to blue on the other.  A single value define where each vertex is in this range.

    You can define a vertex with up to four values,  x, y, z and w.  A shader could use the w value to decide how much blue or red to render this vertex with.  A shader could even dynamically scale the red-blue range across the w range, so that 0 is red, 0.5 is blue and anything above is not drawn.  A shader can make the point darker with distance from the camera, to give a sense of depth.  Because this work would be done by GPU, it wouldn't affect the rendering time.

    Otherwise, you need to put the appropriate color for each vertex into an array (usually in the same VBO).  Assuming the colors stay the same when you looking at different parts of the data, this is not a problem.  Even the depth shading effect I described above could be done with fog.  If the idea is to scale the color with the visible range, you will have to rewrite the values in the color buffer when the range changes.  For  1~2 million points that wouldn't especially quick, though I guess the range isn't going to change very often.

    The main issue is that the graphics card might to process more data.  Assuming the extra w value would be a float (4 bytes) and the color values would be a 4 8-bit values (RGBA), then both methods use the same bus bandwidth; in theory.  The card or the driver will probably covert the RGB values to floats at some point and that might slow things down.

    There is a clear example of setting up and using a VBO on this page OpenGL Vertex Buffer Object.  If thats not helpful, I can give you a step-by-step guide once you've decided what data you to send.
    LVL 20

    Assisted Solution

    Hi DBTechnique,

    The brood force method of just rendering the points as they are will neither give visually acceptable results nor will it be acceptable performance-wise.
    The key is to calculate renderable surface(s) based on the input points. Without surfaces its just a point-soup without shape and depth, with or without VBOs and Shaders.

    The computer science approach to render such a high number of points is called "Point Cloud Rendering"

    Basically what it does is taking a (huge) set of points, called the Point Cloud and uses several techniques to calculate surfaces that can be rendered easily with nowadays graphics-hardware.

    Have a look at this project

    it comes with full source code and sample datasets, just tried it here with VS 2005, it looks pretty amazing!

    There are pics and videos made with that project on the site, also here:

    Point Cloud Rendering is a huge interesting topic in recent graphics computer science, you will find a lot of papers, projects and libraries.

    Btw. sounds like a very interesting project. What data is it exactly?

    Hope it helps :)


    Featured Post

    What Should I Do With This Threat Intelligence?

    Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

    Join & Write a Comment

    Suggested Solutions

    Artificial Intelligence comes in many forms, and for game developers, Path-Finding is an important ability for making an NPC (Non-Playable Character) maneuver through terrain.  A* is a particularly easy way to approach it.  I’ll start with the algor…
    As game developers, we quickly learn that Artificial Intelligence (AI) doesn’t need to be so tough.  To reference Space Ghost: “Moltar, I have a giant brain that is able to reduce any complex machine into a simple yes or no answer. (http://www.youtu…
    Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
    Illustrator's Shape Builder tool will let you combine shapes visually and interactively. This video shows the Mac version, but the tool works the same way in Windows. To follow along with this video, you can draw your own shapes or download the file…

    734 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    23 Experts available now in Live!

    Get 1:1 Help Now