Pragmatic Validity

The pragmatic approach to validation focuses on the usefulness of the measuring instrument as a predictor of some other characteristic or behavior of the individual; it is thus sometimes called predictive validity or criterion-related validity. Pragmatic validity is ascertained by how well the measure predicts the criterion, be it another characteristic or a specific behavior. An example would be the Graduate Management Admissions Test. The fact that this test is required by most of major schools of business attests to it pragmatic validity; it has proved to be useful in predicting how well a student with a particular score on the exam will do in a accredited MBA program. The test score is used to predict the criterion of performance. An example of an attitude scale might be using scores that sales representatives achieved or an instrument designed to assess their job satisfaction to predict who might quit. The attitude score would again be used to predict a behavior - the likelihood of quality. Both of these example illustrate predictive validity in the true sense of word – that is, use of the score to predict some future occurrence.

Another type of pragmatic validity is concurrent validity. Concurrent validity is concerned with the relationship between the predictor variable and criterion variable when both are assessed at the science point in time, for example, a pregnancy test administered to women to ascertain whether they are pregnant provides an example of concurrent validity. The interest here is not in forecasting whether the women will become pregnant in the future but in determining if she is pregnant now.

Concurrent validity is accurate measurement of the current condition or state. Most physical measuring instruments have excellent concurrent validity (e: thermometers, weighting scales, oil dipsticks). Most tests of mental ability also have this type of validity, but to a lesser extent.

Empirical evidence used to test validity -> Compare measure to other indicators, pragmatic (criterion) validity
1. Concurrent validity
Does a measure predict simultaneous criterion?
Validating new measure by comparing to existing measure
E.g., Does new intelligence test correlate with established test

2. Predictive validity
Does a measure predict future criterion?
E.g., SAT scores: Do they predict college GPA?

Pragmatic validity in research looks to a different paradigms than more traditional, positivistic research approaches. It tries to ameliorate problems associated with the rigour-relevance debate, and is applicable in all kinds of research streams. Simply put, pragmatic validity looks at research from a prescriptive-driven perspective.

Validity in prescription-driven research is approached in different ways than descriptive research. The first difference deals with what some researchers call ‘messy situations’ (Brown 1992; Collins, Joseph, and Bielaczuc 2004). A messy situation is a real-life, a highly multivariate one is where independent variables cannot be minimized nor completely accounted for.

The use of the phrase of Pragmatic Validity was first discussed in Worren, Moore & Elliott (2002), who contrasted it with Scientific Validity. This ideas has been taken up in the management literature to a considerable degree.

Cook (1983) actually questions the validity of causal explanations generated in a context-free setting (the goal of positivistic, explanatory research). Causal relationships in pragmatic research are looked at somewhat differently, which is apparent in the wording alone.

In pragmatic science, the goal is to develop knowledge that can be used to improve a situation. This we can call prescriptive knowledge. Prescriptive knowledge, according to van Aken (2004, 2004b, 2005) can take the form of a technological rule. A technological rule is “...a chunk of general knowledge linking an intervention or artifact with an expected outcome or performance in a certain field of application” (van Aken, 2005: p23). This rule can be formulated much the same way as my earlier example of a causal statement; ‘if you perform action X to subject Y, then Z happens’ (Note the cause and effect formulation).

Pragmatic validity is determined strictly by the correlation between the two measures; if the correlation is high, the measure is said to have pragmatic validity. Pragmatic validity is relatively easy to assess. It requires, to be sure, a reasonably valid measure of the criterion with which the scores on the measuring instrument are to be compared. All that the researcher needs to do is to establish the degree of relationship, usually in the form of some kind of correlation coefficient, between the scores on the measuring instrument and the criterion variable. Although easy to assess, pragmatic validity is rarely the most important kind of validity. We are often concerned with “what the measure in fact measures” rather than simply whether it predicts accurately or not.


Source:
-. Managerial Application of Multivariate – Analysis in Marketing, James H.. Myers and Gary M. Mullet, 2003, American Marketing Association, Chicago
-. Marketing Research, Methodological Foundations, 5th edition, The Dryden Press International Edition, author Gilbert A. Churchill, Jr.
-. Wikipedia.com