?
Solved

c# Monte Carlo Incremental Risk Calculation optimisation, random numbers, parallel execution

Posted on 2009-07-06
6
Medium Priority
?
830 Views
Last Modified: 2013-12-10
My current task is to optimise a Monte Carlo Simulation that calculates Capital Adequacy figures by region for a set of Obligors.

It is running about 10 x too slow for where it will need to be in production and number or daily runs required. Additionally the granularity of the result figures will need to be improved down to desk possibly book level at some stage, the code I've been given is basically a prototype which is used by business units in a semi production capacity.

The app is currently single threaded so I'll need to make it multi-threaded, may look at ThreadPool or the Microsoft Parallels library but I'm constrained to .Net 2 on the server at this bank so may have to consider this guy's port http://www.codeproject.com/KB/cs/aforge_parallel.aspx am trying my best to get them to upgrade to .net 3.5 SP1 but it's a major exercise in an organisation of this size and might not be possible in my contract time frames.

I've profiled the app using the trial of dottrace (http://www.jetbrains.com/profiler) any other good profilers people know of? free ones?

A lot of the execution time is spent generating uniform random numbers and then translating this to a normally distributed random number, they are using a c# mersenne twister implementation not sure where they got it or if it's the best way to go about this (or best implementation) to generate the uniform random numbers, then this is translated to a normally distributed version for use in the calculation (haven't delved into the translation code yet).

Also has anyone used:

http://quantlib.org
http://www.qlnet.org (c# port of quantlib)
http://www.boost.org

Any alternatives you know of?

I'm a c# developer so would prefer c#, but a wrapper to c++ shouldn't be a problem should it? maybe even faster leveraging the the c++ implementations. Am thinking some of these libraries will have the fastest method to directly generate normally distributed random numbers, without the translation step. Also they may have some other functions that will be helpful in the subsequent calculations.

Also the box this is on is a quad core Opteron 275, 8GB memory but Windows Server 2003 Enterprise 32 bit, should I advise then to upgrade to a 64 bit OS ? any links to articles supporting this decision would really be appreciated.

Anyway any advice, help you may have is really appreciated.

I'm looking for experts to weigh in on the best way to:

- profile this
- take this from single threaded to multithreaded, whether to go for ThreadPool, some Parallel library...
- best way to generate normally distributed random numbers enmass and consume by the monte carlo simulation
- any other architechural considerations I should be thinking about

Thanks
0
Comment
Question by:craigdev
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
6 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 24789924
0
 
LVL 1

Author Comment

by:craigdev
ID: 24792675
Below is the current code that takes a uniform number and returns a Normally distributed random number, there are numerous calls to ICDF(random.nextUniform()), is the best way of doing things? uniform numbers are never used directly in this program so is there some way to generate normally distributed random numbers in one step?
Is being called like this millions of times for example:
IDistribution dist = new DiscreteNormalDistribution(1048576);
...
SimMat[i, j] = dist.ICDF(rnd.nextUniform());
 
 
class DiscreteNormalDistribution : IDistribution
    {
        private long msize;
        private double[] vals;
        public DiscreteNormalDistribution(long size)
        {
            NormalDistribution dist = new  NormalDistribution();
            msize = size;
            vals = new double[size+1];
            double z;
 
            for (long i = 1; i < size-1; i++)
            {
                z = (1.0/size)*i;
                vals[i] = dist.ICDF(z);
            }
            vals[0] = vals[1];
            vals[size] = vals[size - 1];
        }
 
        public double PDF(double q)
        {
            return 0;
        }
        public double CDF(double q)
        {
            return 0;
        }
        public double ICDF(double q)
        {
            long z = Convert.ToInt64(q * (msize));
            if (z < 0 || z > msize)
            {
                return 0;
            }
            else return vals[z];            
        }
        public double Generate(IRandomGenerator x)
        {
            return ICDF(x.nextUniform());
        }
    }

Open in new window

0
 
LVL 1

Author Comment

by:craigdev
ID: 24792684
Whoops you will also need the NormalDistribution code
class NormalDistribution: IDistribution
    {
        public  double PDF(double q)
        {
            return 0;
        }
        public  double CDF(double q)
        {
            return 0;
        }
        public  double ICDF(double q)
        {
            if (q == .5)
                return 0;
 
            q = 1.0 - q;
 
            double p = (q > 0.0 && q < 0.5) ? q : (1.0 - q);
            double t = Math.Sqrt(Math.Log(1.0 / Math.Pow(p, 2.0)));
 
            double c0 = 2.515517;
            double c1 = 0.802853;
            double c2 = 0.010328;
 
            double d1 = 1.432788;
            double d2 = 0.189269;
            double d3 = 0.001308;
 
           // double x = t - (c0 + c1 * t + c2 * Math.Pow(t, 2.0)) /
           //                (1.0 + d1 * t + d2 * Math.Pow(t, 2.0) + d3 * Math.Pow(t, 3.0));
 
 
            double x = t - (c0 + c1 * t + c2 *(t*t)) /
                          (1.0 + d1 * t + d2 * (t*t) + d3 * (t*t*t));
 
            if (q > .5)
                x *= -1.0;
            return x;
        }
        public double  Generate(IRandomGenerator x)
        {
            return ICDF(x.nextUniform());
        }
 
    }

Open in new window

0
 
LVL 84

Accepted Solution

by:
ozo earned 2000 total points
ID: 24792729
Box-Muller generates 2 normally distributed numbers for every 2 uniformly distributed numbers
The referenced article also gives links to other methods, if you prefer one of them,
0
 
LVL 69

Expert Comment

by:Callandor
ID: 24793460
If you want to consider a hardware/software solution, there is the CUDA language (http://www.nvidia.com/object/cuda_home.html#) designed to work with stream computation engines in nVidia video cards (8800GT and above). It is already in use in some financial institutions that need parallel processing in existing servers (you install a dedicated computation card). 10x speed increases are not unheard of.
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Ready to improve network connectivity? Watch this webinar to learn how SD-WANs and a one-click instant connect tool can boost provisions, deployment, and management of your cloud connection.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
The article shows the basic steps of integrating an HTML theme template into an ASP.NET MVC project
I've attached the XLSM Excel spreadsheet I used in the video and also text files containing the macros used below. https://filedb.experts-exchange.com/incoming/2017/03_w12/1151775/Permutations.txt https://filedb.experts-exchange.com/incoming/201…
Visualize your data even better in Access queries. Given a date and a value, this lesson shows how to compare that value with the previous value, calculate the difference, and display a circle if the value is the same, an up triangle if it increased…

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question