find distribution for strange data

Posted on 2014-02-05
Last Modified: 2014-02-11

I have data which represent the time for process. I have spent more than two weeks to find a good distribution for it, but I coudln't. Actually I have sued exponentiol with Lambda 142.43 but when I insert that in my simulation I faced a problem due to the varionce between sample huge and that value for Lambda would genertate wrong values. So, please I need any hints or help in this point

The data attached to this question
Question by:obad62
LVL 19

Expert Comment

by:Ken Butters
ID: 39836327
Not sure if this is exactly what you are looking for or not.  

What I did..

insert a chart to map the points, then inserted a trendline using the option for automatic polynomial.  

selected to show the polynomial in the chart.   It seems to capture the distribution of points rather well.

Attaching updated spreadsheet with chart.
LVL 27

Expert Comment

ID: 39836791
You can fit any set of data points to a polynomial curve, but the real question is , Does the curve mean anything?"

What is the precision of your data?  (It looks pretty good)
What is the relation you expect between y- axis and x- axis?
(mu guess is you expect it to be flat.
Butters give you a nice graph.
My guess is that something major happened in your process between points 1392 and 1393.
And that an equation reproducing your points will be worthless (except for pinpointing the fact that something drastic happented
LVL 27

Expert Comment

ID: 39839187
Here is one way to analyze the data in Excel.

1.  Calculate the time interval between successive events.
2.  Calculate the log of the time differences.
3.  Assign an Index to each log value.
4.  Copy the values of the Log and Index columns to two new columns.
5.  Sort the new columns by log value.
6.  Plot the sorted Log (Top) and Index (Bottom) columns.

There are a few outliers, but most of the Log values are uniformly distributed between -3 and +1.  This corresponds to time intervals between 0.001 and 1.0 seconds (or days or time units).

The Index plot suggests that small interval values may occur anywhere in the sequence.
As the values get larger, they are more likely to appear later.

I still don't know what the data means or what questions you are trying to answer.
LVL 21

Accepted Solution

Ejgil Hedegaard earned 500 total points
ID: 39842182
Another approach is to find the number of values in a series of intervals, to see how the values are distributed, and find a matching curve, to give the probability for any time value to occur.

First step is to plot the data to get the frequency distribution, and see what it looks like.
I used a step of 10 up to 1060, counting how many values are in each interval, using the frequency function, divided with the number of datapoints (1396), giving the distribution.
The result for 10 is 0.2048, meaning that the probability is 20 % for a value between 0 and 10. For a value between 10 and 20 it is 11 % etc.

It looks exponential, but a check with the logarithmic function shows that it is only the case for the lower values, the upper values follows a more linear function, and a constant will be ok.
See the comments on the sheet.

I end up with a formula like this P(X) = a*Exp(-b*X) + c

The best approximation is then found by calculating the sum of P(X)-Data(X) squared, finding the minimum. For that I used the add-in Solver function.
It gives a correlation of 0.97 witch is ok with a dataset as this.
And the chart shows that the approximation is ok.

Author Closing Comment

ID: 39849867
Thanks, That what I am looking for exactly.

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Some code to ensure data integrity when using macros within Excel. Also included code that helps secure your data within an Excel workbook.
When you see single cell contains number and text, and you have to get any date out of it seems like cracking our heads.
Many functions in Excel can make decisions. The most simple of these is the IF function: it returns a value depending on whether a condition you describe is true or false. Once you get the hang of using the IF function, you will find it easier to us…
Although Jacob Bernoulli (1654-1705) has been credited as the creator of "Binomial Distribution Table", Gottfried Leibniz (1646-1716) did his dissertation on the subject in 1666; Leibniz you may recall is the co-inventor of "Calculus" and beat Isaac…

785 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question