Assessing the reliability of a repeated measures (within group) continuous variable

Hi Folks,

I have a 10 item construct with each item measures on a 7 point Lickert scale. The construct is measured at two points in time - a repeated measure. I can run a Cronbach alpha for each seperately and I can combine the variables and run an alpha for the combination but there is dependence here so these aren't correct approaches - so how do I assess the reliability of a repeated measures construct like this?

LVL 4
freeversedAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

richdiesalProfessorCommented:
Well remember, you're never assessing the reliability of a construct - just the reliability of a scale for a particular administration.  I think the problem you're having is that you're trying to come up with a single reliability estimate for your scale, and this is not necessary.  You can (and should) simply compute and report alpha at each time point.  

Of course, you should keep in mind the kind of information that alpha gives you at each time point - you are only assessing internal consistency reliability (and not from other perspectives).  For further information, I suggest:

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98-104.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
freeversedAuthor Commented:
The scale is the same, implemented twice - a single reliability estimate seems logical. Is there a valid way of assessing the reliability of the scale taking into account that it is a repeated measure?

Otherwise, I guess the first instance is the only valid one for reliability analysis.
0
richdiesalProfessorCommented:
That is not an appropriate way to think of reliability - I strongly recommend reading that article.  Reliability is specific to the administration, not to the scale.  A particular measure does not have a specific reliability associated with it, just a typical range of observed reliabilities.

You are computing internal consistency reliability with alpha.  That method of assessing reliability ignores temporal instability, so it is inappropriate to use it over multiple time points.

If you're trying to assess reliability over time, you are no longer concerned with internal consistency - you are interested in assessing both stability and equivalence.  In which case, alpha isn't appropriate in the first place.

Regardless of this, reliability is assessed in a similar metric as most types of correlation.  If you REALLY wish to combine the two estimates (and again, this is mathematically possible but probably not logically sound), you would just need to take the square root of the mean square reliability estimates.
0
Learn SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

freeversedAuthor Commented:
In this situation the solution is to report the reliability of the first instance only because there is dependence for the second.
0
freeversedAuthor Commented:
In this situation the solution is to report the reliability of the first instance only because there is dependence for the second.
0
richdiesalProfessorCommented:
That depends on the specifics of what you're doing.  I'd strongly recommend giving more information about your measurement model the next time you post a question on this topic.  

In many models, a researcher needs to model that dependence as part of their reliability estimate.  Using only the first instance in those cases is incorrect.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Programming Theory

From novice to tech pro — start learning today.