Solved

c# Monte Carlo Incremental Risk Calculation optimisation, random numbers, parallel execution

Posted on 2009-07-06
6
779 Views
Last Modified: 2013-12-10
My current task is to optimise a Monte Carlo Simulation that calculates Capital Adequacy figures by region for a set of Obligors.

It is running about 10 x too slow for where it will need to be in production and number or daily runs required. Additionally the granularity of the result figures will need to be improved down to desk possibly book level at some stage, the code I've been given is basically a prototype which is used by business units in a semi production capacity.

The app is currently single threaded so I'll need to make it multi-threaded, may look at ThreadPool or the Microsoft Parallels library but I'm constrained to .Net 2 on the server at this bank so may have to consider this guy's port http://www.codeproject.com/KB/cs/aforge_parallel.aspx am trying my best to get them to upgrade to .net 3.5 SP1 but it's a major exercise in an organisation of this size and might not be possible in my contract time frames.

I've profiled the app using the trial of dottrace (http://www.jetbrains.com/profiler) any other good profilers people know of? free ones?

A lot of the execution time is spent generating uniform random numbers and then translating this to a normally distributed random number, they are using a c# mersenne twister implementation not sure where they got it or if it's the best way to go about this (or best implementation) to generate the uniform random numbers, then this is translated to a normally distributed version for use in the calculation (haven't delved into the translation code yet).

Also has anyone used:

http://quantlib.org
http://www.qlnet.org (c# port of quantlib)
http://www.boost.org

Any alternatives you know of?

I'm a c# developer so would prefer c#, but a wrapper to c++ shouldn't be a problem should it? maybe even faster leveraging the the c++ implementations. Am thinking some of these libraries will have the fastest method to directly generate normally distributed random numbers, without the translation step. Also they may have some other functions that will be helpful in the subsequent calculations.

Also the box this is on is a quad core Opteron 275, 8GB memory but Windows Server 2003 Enterprise 32 bit, should I advise then to upgrade to a 64 bit OS ? any links to articles supporting this decision would really be appreciated.

Anyway any advice, help you may have is really appreciated.

I'm looking for experts to weigh in on the best way to:

- profile this
- take this from single threaded to multithreaded, whether to go for ThreadPool, some Parallel library...
- best way to generate normally distributed random numbers enmass and consume by the monte carlo simulation
- any other architechural considerations I should be thinking about

Thanks
0
Comment
Question by:craigdev
  • 2
  • 2
6 Comments
 
LVL 84

Expert Comment

by:ozo
Comment Utility
0
 
LVL 1

Author Comment

by:craigdev
Comment Utility
Below is the current code that takes a uniform number and returns a Normally distributed random number, there are numerous calls to ICDF(random.nextUniform()), is the best way of doing things? uniform numbers are never used directly in this program so is there some way to generate normally distributed random numbers in one step?
Is being called like this millions of times for example:

IDistribution dist = new DiscreteNormalDistribution(1048576);

...

SimMat[i, j] = dist.ICDF(rnd.nextUniform());
 
 

class DiscreteNormalDistribution : IDistribution

    {

        private long msize;

        private double[] vals;

        public DiscreteNormalDistribution(long size)

        {

            NormalDistribution dist = new  NormalDistribution();

            msize = size;

            vals = new double[size+1];

            double z;
 

            for (long i = 1; i < size-1; i++)

            {

                z = (1.0/size)*i;

                vals[i] = dist.ICDF(z);

            }

            vals[0] = vals[1];

            vals[size] = vals[size - 1];

        }
 

        public double PDF(double q)

        {

            return 0;

        }

        public double CDF(double q)

        {

            return 0;

        }

        public double ICDF(double q)

        {

            long z = Convert.ToInt64(q * (msize));

            if (z < 0 || z > msize)

            {

                return 0;

            }

            else return vals[z];            

        }

        public double Generate(IRandomGenerator x)

        {

            return ICDF(x.nextUniform());

        }

    }

Open in new window

0
 
LVL 1

Author Comment

by:craigdev
Comment Utility
Whoops you will also need the NormalDistribution code
class NormalDistribution: IDistribution

    {

        public  double PDF(double q)

        {

            return 0;

        }

        public  double CDF(double q)

        {

            return 0;

        }

        public  double ICDF(double q)

        {

            if (q == .5)

                return 0;
 

            q = 1.0 - q;
 

            double p = (q > 0.0 && q < 0.5) ? q : (1.0 - q);

            double t = Math.Sqrt(Math.Log(1.0 / Math.Pow(p, 2.0)));
 

            double c0 = 2.515517;

            double c1 = 0.802853;

            double c2 = 0.010328;
 

            double d1 = 1.432788;

            double d2 = 0.189269;

            double d3 = 0.001308;
 

           // double x = t - (c0 + c1 * t + c2 * Math.Pow(t, 2.0)) /

           //                (1.0 + d1 * t + d2 * Math.Pow(t, 2.0) + d3 * Math.Pow(t, 3.0));
 
 

            double x = t - (c0 + c1 * t + c2 *(t*t)) /

                          (1.0 + d1 * t + d2 * (t*t) + d3 * (t*t*t));
 

            if (q > .5)

                x *= -1.0;

            return x;

        }

        public double  Generate(IRandomGenerator x)

        {

            return ICDF(x.nextUniform());

        }
 

    }

Open in new window

0
 
LVL 84

Accepted Solution

by:
ozo earned 500 total points
Comment Utility
Box-Muller generates 2 normally distributed numbers for every 2 uniformly distributed numbers
The referenced article also gives links to other methods, if you prefer one of them,
0
 
LVL 69

Expert Comment

by:Callandor
Comment Utility
If you want to consider a hardware/software solution, there is the CUDA language (http://www.nvidia.com/object/cuda_home.html#) designed to work with stream computation engines in nVidia video cards (8800GT and above). It is already in use in some financial institutions that need parallel processing in existing servers (you install a dedicated computation card). 10x speed increases are not unheard of.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

This article is for Object-Oriented Programming (OOP) beginners. An Interface contains declarations of events, indexers, methods and/or properties. Any class which implements the Interface should provide the concrete implementation for each Inter…
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now