Chapter 7 Abstract

 

This chapter begins with an overview of the topic of trustworthiness in measurement. The rest of this chapter and the whole of the next chapter are both devoted to describing the sorts of investigations that need to be carried out to try and establish trust in one’s measurements. The topic of trust is articulated via the reduction in the uncertainty associated with measurement—under the assumption that one can never have complete certainty. Then, the important question is: how can this uncertainty be reduced to a point where it does not interfere with the use of the measurement?

 

The aim of the remainder of this chapter is to describe how to investigate whether “The degree to which test scores [are] … dependable and consistent for an individual test taker” (AERA at al., 2014, p. 222). That is, is the expected noisiness in the measurements of the respondents acceptable—i.e., is the measurement error small enough? Traditionally, measurement texts have cast this as being represented by measurement error and instrument reliability; further, it has been seen as a quality of the instrument separate from validity (i.e., does it measure what it is intended to measure?). However, here reliability and validity are seen here as integral parts of the argument for the trustworthiness of measurement. This chapter covers the issues related with the measurement error and reliability, which affect several components of validity (the latter is the focus of the next chapter).