Reliability Measurement

Topic about reliability have been published in this blog in July 2008, and this topic with same theme will complete previous topic.

The similarity of results provided by independent but comparable measures of the same object, trait, or construct is called reliability. Data said to be reliable if it consistently procedures the same measurement time after time for a given amount or type of a response, regardless of who or when does measurement. Relation between reliability and validity is data can be reliable without valid, but it cannot be valid without being reliable.

Some definition from reliability:
1. In general, reliability (systemic def.) is the ability of a person or system to perform and maintain its functions in routine circumstances, as well as hostile or unexpected circumstances.

2. In statistics, reliability is the consistency of a set of measurements or measuring instrument, often used to describe a test. This can either be whether the measurements of the same instrument give or are likely to give the same measurement (test-retest), or in the case of more subjective instruments, such as personality or trait inventories, whether two independent assessors give similar scores (inter-rater reliability). Reliability is inversely related to random error.

3. In experimental sciences, reliability is the extent to which the measurements of a test remain consistent over repeated tests of the same subject under identical conditions. An experiment is reliable if it yields consistent results of the same measure. It is unreliable if repeated measurements give different results. It can also be interpreted as the lack of random error in measurement.

4. Reliability has to do with the quality of measurement. Reliability is the "consistency" or "repeatability" of your measures.

5. In research, the term reliability means "repeatability" or "consistency". A measure is considered reliable if it would give us the same result over and over again.

6. A scale is said to be reliable if it consistently produces the same measurement or category time after time for a given amount or type of a response, regardless of who does the measurement or when.

7. Reliability refers to the consistency of a measure. A test is considered reliable if we get the same result repeatedly.

8. 'Reliability' of any research is the degree to which it gives an accurate score across a range of measurement. It can thus be viewed as being 'repeatability' or 'consistency'.

9. Reliability means "repeatability" or "consistency". A measure is considered reliable if it would give us the same result over and over again.


Reliability does not imply validity. That is, a reliable measure is measuring something consistently, but not necessarily what it is supposed to be measuring. It is distinguished from validity in that validity is represented in agreement between two attempts to measure the same trait through maximally different methods, whereas reliability is the agreement between two efforts to measure the same trait through maximally similar methods.

If a measure were valid, there would be little need to worry about its reliability. If a measure is valid, it reflects the characteristic that it is supposed to measure and id not distorted by other factors, either systematic or transitory. For example, while there are many reliable tests of specific abilities, not all of them would be valid for predicting, say, job performance. In terms of accuracy and precision, reliability is precision, while validity is accuracy.

There are four general classes of reliability estimates, each of which estimates reliability in a different way. They are:
1. Inter-Rater or Inter-Observer Reliability
Used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon.
Inter-rater: Different people, same test.

2. Test-Retest Reliability
Used to assess the consistency of a measure from one time to another.
Test-retest: Same people, different times.

3. Parallel-Forms Reliability
Used to assess the consistency of the results of two tests constructed in the same way from the same content domain.
Parallel-forms: Different people, same time, different test.

4. Internal Consistency Reliability
Used to assess the consistency of results across items within a test.
Internal consistency: Different questions, same construct.

Although lack of reliability provides negative evidence of the validity of a measure, the mere presence of reliability does not mean that the measure is valid. Reliability is a necessary, but not a sufficient, condition for validity. Reliability is more easily measured than validity.


Source:
-. Marketing Research, Methodological Foundations, 5th edition, The Dryden Press International Edition, author Gilbert A. Churchill, Jr.
-. Managerial Application of Multivariate – Analysis in Marketing, James H.. Myers and Gary M. Mullet, 2003, American Marketing Association, Chicago
-. www.wikipedia.com
-. www.socialresearchmethods.net
-. www.changingminds.org