Educational Evaluation and Policy Analysis recent issues

District Policy and Teachers' Social Networks

Coburn, C. E., Russell, J. L. — 2008-09-09

Policy makers increasingly include provisions aimed at fostering professional community as part of reform initiatives. Yet little is known about the impact of policy on teachers’ professional relations in schools. Drawing theoretically from social capital theory and methodologically from qualitative social network analysis, this article explores how district policies influence teachers’ social networks in eight elementary schools in two districts involved in the scale-up of mathematics curriculum. It is argued that policy affects whom teachers seek out for discussion of mathematics instruction but that differences in policy provisions lead to variations in the nature and quality of interactions. Furthermore, school leaders mediate district policy, thereby influencing these patterns of interaction. By uncovering the dynamics by which policy influences teachers’ social networks, this article contributes to understandings of the factors that foster the development of social capital. It also uncovers opportunities for intervention for those designing policy initiatives to support implementation of instructional innovations.

Credits, Time, and Attainment: Articulation Policies and Success After Transfer

Roksa, J., Keith, B. — 2008-09-09

While state legislatures are increasingly enacting articulation policies, research to date provides little evidence that these policies enhance students’ likelihood of transfer. Based on a careful historical review of state statutes, the authors propose that articulation policies do not improve transfer rates because that is not their intended purpose; the main goal of articulation policies is to prevent the loss of credits when students transfer within state higher education systems. Subsequently, the authors use the National Education Longitudinal Study to evaluate articulation policies based on an alternative set of outcomes: attainment of a bachelor’s degree, time to degree, and credits required to complete a bachelor’s degree. They discuss the limitations of existing data and propose that future studies be designed to specifically evaluate the transfer or loss of credits applicable toward general education requirements.

Winners and Losers: Changes in Texas University Admissions Post-Hopwood

Long, M. C., Tienda, M. — 2008-09-09

This article evaluates changes in racial and ethnic composition of three Texas universities following the ban on affirmative action imposed by the 1996 Hopwood decision. The authors estimate the extent to which universities practiced affirmative action before the ban and evaluate how officers at these universities responded by changing relative weights accorded to various applicant characteristics. After assessing whether changes in the relative weights favored minority applicants, the degree to which these new policies succeeded in maintaining minority shares at their pre-Hopwood levels is simulated. This article finds that these universities complied with the Hopwood ruling such that direct advantages given to Black and Hispanic applicants disappeared (and in some cases became disadvantages). Although there is some evidence that universities changed the weights they placed on applicant characteristics in ways that aided underrepresented minority applicants, these changes were insufficient to restore Black and Hispanic applicants’ share of admitted students.

Small High Schools on a Larger Scale: The Impact of School Conversions in Chicago

Kahne, J. E., Sporte, S. E., de la Torre, M., Easton, J. Q. — 2008-09-09

This study examines 4 years of small school reform in Chicago, focusing on schools formed by converting large traditional high schools into small autonomous ones. Analyzing systemwide survey and outcome data, the authors assess the assumptions embedded in the reform’s theory of change. They find that these schools are characterized by more collegial and committed teacher contexts and more academically and personally supportive student contexts. There is some evidence of decreased dropout rates and increased graduation rates for the first cohort of students but not for the second cohort. The authors do not find stronger instruction, nor do they find student achievement has improved. They discuss implications for reformers and policy makers who are interested in small schools in particular and high school reform in general.

Accountability Texas-Style: The Progress and Learning of Urban Minority Students in a High-Stakes Testing Context

Heilig, J. V., Darling-Hammond, L. — 2008-06-11

This study examines longitudinal student progress and achievement on the elementary, middle, and high school levels in relation to accountability policy incentives in a large urban district in Texas. Using quantitative analyses supplemented by qualitative interviews, the authors found that high-stakes testing policies that rewarded and punished schools based on average student scores created incentives for schools to "game the system" by excluding students from testing and, ultimately, school. In the elementary grades, low-achieving students were disproportionately excluded from taking the high-stakes Texas Assessment of Academic Skills tests, demonstrating gains not reflected on the low-stakes Stanford Achievement Test–Ninth Edition. Student exclusion at the elementary level occurred through special education and language exemptions and missing scores. Furthermore, gaming strategies reduced educational opportunity for African American and Latino high school students. Sharp increases in 9th-grade student retention and disappearance were associated with increases in 10th-grade test scores and related accountability ratings.

Teacher Effectiveness in First Grade: The Importance of Background Qualifications, Attitudes, and Instructional Practices for Student Learning

Palardy, G. J., Rumberger, R. W. — 2008-06-11

This study uses Early Childhood Longitudinal Study data to investigate the importance of three general aspects of teacher effects—teacher background qualifications, attitudes, and instructional practices—to reading and math achievement gains in first grade. The results indicate that compared with instructional practices, background qualifications have less robust associations with achievement gains. These findings suggest that the No Child Left Behind Act’s "highly qualified teacher" provision, which screens teachers on the basis of their background qualifications, is insufficient for ensuring that classrooms are led by teachers who are effective in raising student achievement. To meet that objective, educational policy needs to be directed toward improving aspects of teaching, such as instructional practices and teacher attitudes.

Evidence of a Differential Effect of Ability Grouping on the Reading Achievement Growth of Language-Minority Hispanics

Robinson, J. P. — 2008-06-11

Ability grouping is sometimes thought to exacerbate inequality by increasing achievement gaps; however, ability grouping may in fact benefit a fast growing and often marginalized student population: children from non-English-speaking home environments. The level-appropriate, small-group instruction received in reading ability groups may be particularly beneficial to these language-minority children, who are not regularly exposed to English at home. Focusing on Hispanics, who make up the majority of language-minority students, the author examined this hypothesis through difference-in-differences estimation techniques in a hierarchical linear model framework. Ability grouping in reading during kindergarten was significantly associated with greater benefits for language-minority Hispanic students relative to other students. However, this benefit faded during the summer and first grade, unless grouping continued in first grade. These findings are robust to alternative specifications and suggest that differentiated instructional strategies upon school entry may be an effective, relatively low cost tool to combat the achievement gap faced by a fast growing segment of students.

Do Teacher Absences Impact Student Achievement? Longitudinal Evidence From One Urban School District

Miller, R. T., Murnane, R. J., Willett, J. B. — 2008-06-11

This article exploits highly detailed data on teacher absences from a large urban school district in the northern United States to shed light on the determinants and effects of teacher absences. The topic is important because both school and district policies can influence teachers’ propensity to be absent. The authors estimate the impact of teacher absences on academic achievement of students matched to elementary school teachers. Models include fixed effects for teachers to control statistically for potential correlation between time-invariant levels of teachers’ skill and effort and their rates of absence. The authors estimate 10 additional days of teacher absence reduce mathematics achievement of fourth-grade students by 3.2% of a standard deviation. They employ an additional instrumental variables strategy to bolster the case for a causal interpretation of results. Instrumental variables results indicate the impact of unexpected teacher absences on student achievement is larger than the impact of anticipated absences.

Does NBPTS Certification Affect the Number of Colleagues a Teacher Helps With Instructional Matters?

2008-02-20

In addition to identifying and developing superior classroom teaching, the National Board for Professional Teaching Standards (NBPTS) certification process is intended to identify and cultivate teachers who are more engaged in their schools. Here the authors ask, "Does NBPTS certification affect the number of colleagues a teacher helps with instructional matters?" If so, this could enhance the influence of NBPTS-certified teachers and their contributions to their professional communities. Using sociometric data within 47 elementary schools from two states, the authors find that NBPTS-certified teachers were nominated more as providing help with instruction than non-NBPTS-certified teachers. From analyses using propensity score weighting, the authors then infer that NBPTS certification affects the number of colleagues a teacher helps with instructional matters. The authors then quantify the robustness of their inference in terms of internal and external validity, finding, for example, that any omitted confounding variable would have to have an impact six times larger than that of their strongest covariate to invalidate their inference. Therefore, the potential value added by NBPTS-certified teachers as help providers has policy and practice implications in an era when teacher leadership has risen to the fore as a critical force for school improvement.

From High School to Higher Education: Curricular Policy and Postsecondary Enrollment in Israel

Addi-Raccah, A., Ayalon, H. — 2008-02-20

Using multilevel models, the authors tested the hypothesis that high schools, through their curricular policies, operate as mechanisms that help members of privileged groups to take better advantage of postsecondary opportunities. The analysis was based on a 7-year follow-up study of 44,666 Israeli students who graduated from 385 high schools in 1991. The main findings were that (a) the curricular experience of students partly mediated between their sociodemographic characteristics and postsecondary enrollment, (b) the curricular arrangements of schools fully mediated the effects of their social composition on their graduates’ postsecondary education, and (c) graduates of socially privileged schools made a better use of their matriculation certificates. This afforded privileged students an additional advantage.

Raising the Bar: Curricular Intensity and Academic Performance

Attewell, P., Domina, T. — 2008-02-20

Using national transcript data, the authors examine inequality in access to an advanced curriculum in high school and assess the consequences of curricular intensity on test scores and college entry. Inequalities in curricular intensity are primarily explained by student socioeconomic status effects that operate within schools rather than between schools. They find significant positive effects of taking a more intense curriculum on 12th-grade test scores and in probabilities of entry to and completion of college. However, the effect sizes of curricular intensity are generally modest, smaller than advocates of curricular upgrading policies have implied.

Early-Grade Retention and Children's Reading and Math Learning in Elementary Years

Hong, G., Yu, B. — 2007-12-11

Many schools have adopted early-grade retention as an intervention strategy for children displaying academic or behavioral problems. Previous analyses of the Early Childhood Longitudinal Study Kindergarten Cohort data have found evidence of negative effects of kindergarten retention on academic learning during the repeated year. Will kindergarten retainees recover their lost ground and excel in the long run? What are the effects of first grade retention? According to the analytic results of this study, the negative effects of kindergarten retention on retainees’ reading and math outcomes at the end of the treatment year substantially fade by fifth grade. Meanwhile, first grade retention shows negative effects that stay almost constant from 1 year after treatment to 3 years later. In general, we find no evidence that early-grade retention brings benefits to the retainees’ reading and math learning toward the end of the elementary years.

An Empirical Investigation of Professional Development Effects on Literacy Instruction Using Daily Logs

Correnti, R. — 2007-12-11

The author examined the effects of professional development (PD) on literacy instruction using 75,689 lessons from 1,945 classrooms in 112 schools participating in the Study of Instructional Improvement. Stratifying over 94 pretreatment covariates, including prior instruction, the results revealed the importance of PD as a lever for changing teacher practice. Teachers receiving intense PD in comprehension offered 10% more comprehension instruction than teachers not receiving intense PD. Similarly, teachers receiving intense PD in writing offered 13% more writing instruction and had students write 12% more text than other teachers. Furthermore, these estimates may be lower bounds, because they were estimated over a single year and because there was demonstrated potential for additional influences of the school context.

When Elementary Schools Stay Open Late: Results From the National Evaluation of the 21st Century Community Learning Centers Program

James-Burdumy, S., Dynarski, M., Deke, J. — 2007-12-11

This article presents evidence from a national evaluation of the effects of 21st Century Community Learning Center afterschool programs. The study was conducted in 12 school districts and 26 after-school centers, at which 2,308 elementary school students who were interested in attending a center were randomly assigned either to the treatment or control group. The findings indicate that the programs affected the type of care and supervision students received after school, with parents less likely to be caring for their child and other adults more likely, but there was no statistically significant effect on the incidence of self-care. Students in the program reported feeling safer after school, but their academic outcomes were not affected and they had more incidents of negative behavior.

The Practical Relevance of Accountability Systems for School Improvement: A Descriptive Analysis of California Schools

Mintrop, H., Trujillo, T. — 2007-12-11

In search for the practical relevance of accountability systems for school improvement, the authors ask whether practitioners traveling between the worlds of system-designated high- and low-performing schools would detect tangible differences in educational quality and organizational effectiveness. In comparing nine exceptionally high and low performing California middle schools, the authors conclude that if such travelers expected to encounter visible signs of an overall higher quality of students’ educational experience at the high-performing schools, they would be disappointed. Rather, they would have to settle on a narrower definition of quality that is more proximate to the effective acquisition of standards-aligned and test-relevant knowledge. High-growth schools tended to generate internal commitment for accountability and consider it an impetus for raising standards.

Acknowledgements

2007-12-11