Ecological validity

In the behavioral sciences, ecological validity is often used to refer to the judgment of whether a given study's variables and conclusions (often collected in lab) are sufficiently relevant to its population (e.g. the "real world" context). Psychological studies are usually conducted in laboratories though the goal of these studies is to understand human behavior in the real-world. Ideally, an experiment would have generalizable results that predict behavior outside of the lab, thus having more ecological validity. Ecological validity can be considered a commentary on the relative strength of a study's implication(s) for policy, society, culture, etc.

This term was originally coined by Egon Brunswik^[1] and held a specific meaning. He regarded ecological validity as the utility of a perceptual cue to predict a property (basically how informative the cue is). For example, high school grades have moderate ecological validity for predicting college grades.^[2] Hammond^[3] argued that the now common use of the term to refer to generality of research results to the "real world" is inappropriate because it robs the original usage of its meaning.

Due to the evolving and broad definition of ecological validity, problematic usage of this term in modern scientific studies occurs because it is often not clearly defined. In fact, in many cases just being specific about what behavior/context you are testing makes addressing ecological validity unnecessary.^[4]

The term "ecological validity" is now widely used by researchers unfamiliar with the origins and technical meaning of the term to be broadly equivalent to mundane realism.^[5] Mundane realism references the extent to which the experimental situation is similar to situations people are likely to encounter outside the laboratory. For example, mock-jury research is designed to study how people might act if they were jurors during a trial, but many mock-jury studies simply provide written transcripts or summaries of trials while in a classroom or office settings. Such experiments do not approximate the actual look, feel, and procedure of a real courtroom trial, and therefore lack mundane realism.

The better-recognized concern is that of external validity: if the results from such a mock-jury study are reproduced in and generalized across trials where these stimulus materials, settings, and other background characteristics vary, then the measurement process may be deemed externally valid. External validity refers to the ability to generalize study findings in other contexts. Ecological validity, the ability to generalize study findings to the real world, is a subcategory of external validity.^[6]  

Another example highlighting the differences between these terms is from an experiment that studied pointing^[7]—a trait originally attributed uniquely to humans—in captive chimpanzees. This study certainly had external validity because when testing if captive chimps will gesture towards food by pointing, the results were reproduced in different trials and under different conditions. Nevertheless, the ecological validity and mundane realism of this study came into question because researchers were attempting to do cognitive research while disrupting these animals’ natural environment.

Because the experimental conditions that are conducive to pointing (i.e. watching humans point) will never be experienced by chimpanzees outside the laboratory, this study is by definition lacking mundane realism. Furthermore, pointing doesn't emerge in wild chimpanzees; therefore, one could argue that because captive chimps were taken out of their natural environment, there is no point in studying a trait that wasn't adaptive through natural selection (holding no ecological validity). However, a counter-argument to this claim is that captive chimpanzees are under the ecological framework merely by existing. It is plausible that captivity is the context that leads to the development of this trait. Furthermore, if this trait can be taught to captive chimpanzees, it is certainly possible that if faced with particular circumstances, wild chimpanzees would also learn to point.

Thus, ecological validity is a sliding scale. Humans have dramatically changed the natural world to the extent that behavior traits that animals adapt in response to human intervention (such as pointing) could technically hold ecological validity. With this in mind, perhaps instead of attempting to broaden the definition of this term, a new term should be identified to define traits that have only emerged due to direct human interference.

• Construct validity
• Content validity
• External validity
• Statistical conclusion validity