|
Sign In to gain access to subscriptions and/or personal tools.
|
Measuring Instructional Practice: Can Policymakers Trust Survey Data?
Daniel P. Mayer
Mathematica Policy Research, Inc
Policymakers' new interest in reforming teaching has created a demand for accurate data on instructional practice. Most such data come from teacher surveys, although the accuracy of these surveys in assessing practice is virtually unknown. This exploratory study examines the reliability and validity of using a survey to gauge the percentage of time that algebra teachers use practices that are consistent with professional standards for teaching mathematics. A 13-item composite measuring the frequency with which teachers use these practices had a test-retest reliability of .69 based on two waves of survey data collected four months apart, In addition, there was a .85 correlation between a composite based on survey data and a parallel composite based on classroom observations. Both sets of results suggest that the composite is quite reliable, and the second set of results suggests that it has some validity: This composite fails, however, to capture the quality with which teachers engage in reform practices. In addition, the results indicate that teachers do not reliably report how much time they spend using one practice or another. This suggests that the trend of presenting individual indicators, rather than composite indicators, in state and national reports may be misguided. More research should be done to improve the reliability and validity of both.
Educational Evaluation and Policy Analysis, Vol. 21, No. 1,
29-45 (1999)
DOI: 10.3102/01623737021001029
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
V.-N. Le, J. R. Lockwood, B. M. Stecher, L. S. Hamilton, and J. F. Martinez
A Longitudinal Investigation of the Relationship between Teachers' Self-Reports of Reform-Oriented Instruction and Mathematics and Science Achievement
Educational Evaluation and Policy Analysis,
September 1, 2009;
31(3):
200 - 220.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. M. Desimone
Improving Impact Studies of Teachers' Professional Development: Toward Better Conceptualizations and Measures
Educational Researcher,
April 1, 2009;
38(3):
181 - 199.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
B. Rowan and R. Correnti
Studying Reading Instruction With Teacher Logs: Lessons From the Study of Instructional Improvement
Educational Researcher,
March 1, 2009;
38(2):
120 - 131.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
K. L. Wahlstrom and K. S. Louis
How Teachers Experience Principal Leadership: The Roles of Professional Community, Trust, Efficacy, and Shared Responsibility
Educational Administration Quarterly,
October 1, 2008;
44(4):
458 - 495.
[Abstract]
[PDF]
|
|
|
|
|
|
|
W. R. Penuel, B. J. Fishman, R. Yamaguchi, and L. P. Gallagher
What Makes Professional Development Effective? Strategies That Foster Curriculum Implementation
American Educational Research Journal,
December 1, 2007;
44(4):
921 - 958.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
T. M. Smith, L. M. Desimone, T. L. Zeidner, A. C. Dunn, M. Bhatt, and N. L. Rumyantseva
Inquiry-Oriented Instruction in Science: Who Teaches That Way?
Educational Evaluation and Policy Analysis,
September 1, 2007;
29(3):
169 - 199.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. Karvonen, S. Y. Wakeman, C. Flowers, and D. M. Browder
Measuring the Enacted Curriculum for Students With Significant Cognitive Disabilities: A Preliminary Investigation
Assessment for Effective Intervention,
January 1, 2007;
33(1):
29 - 38.
[Abstract]
[PDF]
|
|
|
|
|
|
|
B. Stecher, V.-N. Le, L. Hamilton, G. Ryan, A. Robyn, and J. R. Lockwood
Using Structured Classroom Vignettes to Measure Instructional Practices in Mathematics
Educational Evaluation and Policy Analysis,
January 1, 2006;
28(2):
101 - 130.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. Milesi and A. Gamoran
Effects of Class Size and Instruction on Kindergarten Achievement
Educational Evaluation and Policy Analysis,
January 1, 2006;
28(4):
287 - 313.
[Abstract]
[PDF]
|
|
|
|
|
|
|
T. M. Smith, L. M. Desimone, and K. Ueno
"Highly Qualified" to Do What? The Relationship Between NCLB Teacher Quality Mandates and the Use of Reform-Oriented Instruction in Middle School Mathematics
Educational Evaluation and Policy Analysis,
January 1, 2005;
27(1):
75 - 109.
[Abstract]
[PDF]
|
|
|
|
|
|
|
L. M. Desimone, T. Smith, D. Baker, and K. Ueno
Assessing Barriers to the Reform of U.S. Mathematics Instruction From an International Perspective
American Educational Research Journal,
January 1, 2005;
42(3):
501 - 535.
[Abstract]
[PDF]
|
|
|
|
|
|
|
L. M. Desimone and K. C. Le Floch
Are We Asking the Right Questions? Using Cognitive Interviews to Improve Surveys in Education Research
Educational Evaluation and Policy Analysis,
January 1, 2004;
26(1):
1 - 22.
[Abstract]
[PDF]
|
|
|
|
|
|
|
T. C. M. Lam, K. E. Green, and C. Bordignon
Effects of Item Grouping and Position of the "Don't Know" Option on Questionnaire Response
Field Methods,
November 1, 2002;
14(4):
418 - 432.
[Abstract]
[PDF]
|
|
|
|
|
|
|
A. C. Porter
Measuring the Content of Instruction: Uses in Research and Practice
Educational Researcher,
October 1, 2002;
31(7):
3 - 14.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. B. Swanson and D. L. Stevenson
Standards-Based Reform in Practice: Evidence on State Policy and Classroom Instruction from the NAEP State Assessments
Educational Evaluation and Policy Analysis,
January 1, 2002;
24(1):
1 - 27.
[Abstract]
[PDF]
|
|
|
|
|
|
|
L. M Desimone, A. C. Porter, M. S. Garet, K. S. Yoon, and B. F. Birman
Effects of Professional Development on Teachers' Instruction: Results from a Three-year Longitudinal Study
Educational Evaluation and Policy Analysis,
January 1, 2002;
24(2):
81 - 112.
[Abstract]
[PDF]
|
|
|
|
|
