This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Desimone, L. M. Articles by Le Floch, K. C. Search for Related Content Social Bookmarking                         What's this?

Are We Asking the Right Questions? Using Cognitive Interviews to Improve Surveys in Education Research

Vanderbilt University

Kerstin Carlson Le Floch

American Institutes for Research

Improving the validity and reliability of surveys is a critical part of the response to the call for improved rigor of education research, policy analysis and evaluation. Too often we create inquiry tools without validating our measures against how respondents interpret our questions, and therefore collect data of questionable quality. The purpose of this article is to demonstrate how cognitive interviews can be a useful method for improving the reliability and validity of surveys used in education research. The cognitive interviews described in this article reveal the complex thought processes in which our respondents engaged as they discussed their responses to questions concerning standards, assessments and professional development, and the unanticipated interpretations of seemingly straightforward constructs. We present a number of examples to illustrate complexities that education researchers face in trying to design large-scale surveys to answer critical educational policy questions, and the substantive insights that can be gained from the use of cognitive interviews. The use of cognitive interviews in the development of surveys has been for the most part neglected by the educational research community, and we suggest that more attention be paid to this methodology as a way of improving survey research.

Key Words: cognitive interviews • survey design • validity and reliability

Educational Evaluation and Policy Analysis, Vol. 26, No. 1, 1-22 (2004)
DOI: 10.3102/01623737026001001


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				L. M. Desimone Improving Impact Studies of Teachers' Professional Development: Toward Better Conceptualizations and Measures Educational Researcher, April 1, 2009; 38(3): 181 - 199. [Abstract] [Full Text] [PDF]

				T. C. M. Lam and M. Kolic Effects of Semantic Incompatibility on Rating Response Applied Psychological Measurement, May 1, 2008; 32(3): 248 - 260. [Abstract] [PDF]

				W. R. Penuel, B. J. Fishman, R. Yamaguchi, and L. P. Gallagher What Makes Professional Development Effective? Strategies That Foster Curriculum Implementation American Educational Research Journal, December 1, 2007; 44(4): 921 - 958. [Abstract] [Full Text] [PDF]

				B. Stecher, V.-N. Le, L. Hamilton, G. Ryan, A. Robyn, and J. R. Lockwood Using Structured Classroom Vignettes to Measure Instructional Practices in Mathematics Educational Evaluation and Policy Analysis, January 1, 2006; 28(2): 101 - 130. [Abstract] [PDF]

				L. M. Desimone, T. Smith, D. Baker, and K. Ueno Assessing Barriers to the Reform of U.S. Mathematics Instruction From an International Perspective American Educational Research Journal, January 1, 2005; 42(3): 501 - 535. [Abstract] [PDF]