Improve company productivity with a Business Account.Sign Up

x
?
Solved

? MS ACCESS 2010 sample to calculate median quickly?

Posted on 2014-02-15
6
Medium Priority
?
439 Views
Last Modified: 2014-02-20
I have a data base with several payments for each participant.
Each payment has participant number and date-of-payment.

I'd like to easily and QUICKLY calculate the Median number of days between payments for each participant.

I found a method using a sub-query, but, it ran for over 12 hours on only about 30,000 rows.
I'd like to be able to run it on a half-million rows in less than an hour.

help?
0
Comment
Question by:philkryder
  • 2
  • 2
  • 2
6 Comments
 
LVL 50

Expert Comment

by:Dale Fye
ID: 39862121
What did your query look like?

To start with, you have to get the payment date and the previous payment date,  I generally do that something like:

SELECT P1.ParticipantID, P1.PaymentDate, Max(P2.PaymentDate) as PrevPayment
FROM yourTable as P1
LEFT JOIN yourTable as P2
ON P1.ParticipantID = P2.ParticipantID
AND P1.PaymentDate >P2.PaymentDate
GROUP By P1.ParticipantID, P1.PaymentDate

This should run relatively quickly, but is not editable in the query grid, only in the SQL view.  Once you have that working, you can either save it, or make it a subquery to get the difference in dates between the PaymentDate and PrevPayment.  You might want to make that a temp table or even make the above into a temp table, to speed up succeeding logic.

What logic are you using to compute the median for each participant, once you know the number of days between payments?
0
 
LVL 50

Expert Comment

by:Dale Fye
ID: 39862123
BTW, you might want to check out this article by MatthewsPatrick that discusses Median, Mode, Skewness, and Kurtosis
0
 
LVL 93

Accepted Solution

by:
Patrick Matthews earned 2000 total points
ID: 39863383
Have you had a chance to review the approach described in my article above?  (Thanks for referring it, Dale!)
0
Easily Design & Build Your Next Website

Squarespace’s all-in-one platform gives you everything you need to express yourself creatively online, whether it is with a domain, website, or online store. Get started with your free trial today, and when ready, take 10% off your first purchase with offer code 'EXPERTS'.

 
LVL 1

Author Comment

by:philkryder
ID: 39863470
yes - I used the DMEDIAN on an INTERVAL value that I obtained using the code Dale showed for finding the interval between the two dates.

After finding the INTERVAL it would find the DMEDIAN.

I saved intermediate tables after each step.

I was running this on an I3 intel and it pegged 100% of one of the CPU engines for about 12 hours to do my 32k rows.


My typical grouping on Participant will have from 7 to 20 intervals.


Have you run this on similar data quantities?

I'll post the code when I get back to work after the holiday tomorrow.
Phil
0
 
LVL 93

Expert Comment

by:Patrick Matthews
ID: 39863542
A sample file would be very helpful
0
 
LVL 1

Author Comment

by:philkryder
ID: 39868810
I'd like to close this.
On closer examination, I found the major delay was in calculating the interval between two timestamps.

I subsequently calculated the interval by sorting in order by the key, and the timestamp, then assigning a sequential ID, then doing a self-join on the next row. This seemed much faster than the sub-query to find the interval.

Matthew -
regarding the median -
would you consider adding a couple of global variables such that if the CRITERIA was the same between calls to DMEDIAN, then the DMEDIAN returned would be the same as on the prior call.
Not as elegant,
But, It seems it would avoid creating and running the subquery much of the time  ...
something like:
' =================
Global DMedianSave As Long
Global CriteriaSave As String
...

    If Criteria <> "" Then
   
        If CriteriaSave = Criteria Then
            DMedian = DMedianSave
            GoTo Cleanup
        End If
    End If
    CriteriaSave = Criteria
...
 '        calculate DMEDIAN as before
...
   End If        
    DMedianSave = DMedian
    GoTo Cleanup
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

In this article, we will see two different methods to recover deleted data. The first option will be using the transaction log to identify the operation and restore it in a specified section of the transaction log. The second option is simpler and c…
Usually, rounding is performed by some power of 10 - to thousands, hundreds, tens, or integer - or to one, two, or more decimals. But rounding can also be done to a power of two, say, 16 or 64, or 1/32 or 1/1024, even for extreme values.
In Microsoft Access, learn how to use Dlookup and other domain aggregate functions and one method of specifying a string value within a string. Specify the first argument, which is the expression to be returned: Specify the second argument, which …
With Secure Portal Encryption, the recipient is sent a link to their email address directing them to the email laundry delivery page. From there, the recipient will be required to enter a user name and password to enter the page. Once the recipient …

607 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question