We noted in a previous post that students in Michigan and Detroit post weaker performances on the National Assessment of Educational Progress (NAEP) than states across the country, particularly Minnesota. For many years, researchers have attempted to identify factors associated with NAEP scores, which would be of considerable interest to stakeholders who want to address Michigan and Detroit’s NAEP performance. Here, we will briefly summarize some of these factors and selected research addressing them.
For several reasons, NAEP scores in mathematics and reading have been of primary interest to researchers. Much of the research on NAEP score predictors, therefore, focuses on performance in these two subject areas.
Given the primacy of demographic factors such as race, ethnicity, and gender in education research, researchers have also asked whether these variables might predict students’ NAEP performance. For example, Vanneman et al. (2009) and Hemphil & Venneman (2011) noted achievement gaps in NAEP mathematics scores between African-American and White students and between White and Hispanic students. A number of peer-reviewed studies also identify race as a factor in NAEP results (Tate, 1997; Fuchs & Reklis, 1994; Thomas & Stockton, 2003). Some studies explore this factor at a greater depth; for example, Card & Rothstein (2007) attribute the race/ethnicity gap (though using SAT, not NAEP scores) to racial segregation of particular geographic areas, while Lubienski (2006) finds that varying test modes for NAEP mathematics appears to have little or no impact on performance.
There is less evidence for the influence of gender on NAEP scores (Abedi & Lord, 2001; Tate, 1997; Hyde & Linn, 2006; Guthrie et al., 2001), though Thomas and Stockton (2003) identify a small positive relationship between female students and NEAP reading scores and McGraw and colleagues (2006) find a negative relationship between female students and NAEP mathematics scores.
The results are also fairly consistent for socioeconomic status (SES). Biddle (1997) and McQuillan (1998) find a negative relationship between poverty and NAEP scores while Abedi & Lord (2001) and Nelson et al. (2003) find a negative relationship between Free lunch/Aid to Families with Dependent Children (AFDC) status and NAEP scores. Byrnes (2003) and Fuchs & Reklis (1994) find a positive association between parental education levels and students’ 12th and 8th grade NAEP math scores, respectively. Using 1996 NAEP data, Lubienski (2002) finds that SES factors such as parent education and number of literary resources in the home do not explain the African-American/White achievement gap discussed above. Inherent in these studies is, of course, the selection and validity of individual-level or school-level (e.g., Title I designated school) definitions of SES (Thomas & Stockton, 2003).
Some researchers have also considered other literacy-related factors and their possible effect on NAEP scores. For instance, Abedi et al. (2001) and Abedi and Lord (2001) find that English Language Learner (ELL) and Limited English Proficiency (LEP) statuses are negatively related to NAEP mathematics performance. Length of stay in the United States appears to be positively associated with NAEP mathematics performance (Abedi et al., 2001). Access to printed reading material (McQuillan, 1998) and access to school and public libraries (Krashen et al., 2012) also appear to be positively associated with NAEP reading scores.
In general, coursework and related preparation seem to be consistent predictors of NAEP scores. Tate (1997), Abedi & Lord (2001), and Abedi et al. (2001) find that advanced mathematics preparation and coursework are positive predictors of NAEP math scores. Guthrie et al. (2001) and Pinnell et al. (1995) find that reading opportunities and reading prosody, respectively, are positively associated with NAEP reading performance. Abedi et al. (2001) find evidence of a positive association between students’ overall grades since 6th grade and NAEP mathematics performance.
Some authors have considered more systemic or institutional factors in their NAEP research, though this research is less consistent and (less?) extensive. Lubienski (2006) finds a positive association between NAEP math scores and (1) collaborative problem-solving instruction, (2) teacher knowledge of National Council of Teachers of Mathematics (NCTM) standards, and (3) certain ‘reform-oriented’ teaching practices such as non-number math strands. Guthrie (2001) finds that balanced reading instruction is positively associated with Grade 4 NAEP Reading Comprehension in Maryland. Grissmer et al. (2000) and Fitzpatrick (2008) find that greater levels of Kindergarten and pre-Kindergarten participation are positively associated with NAEP scores. Carnoy & Loeb (2002) find a positive association between gains in NAEP mathematics results and strength of state accountability (based on high-stakes testing to sanction and reward schools), but no effect on 9th grade retention rates. In a study supported by the American Federation of Teachers (AFT), Nelson et al. (2003) find that charter school attendance, especially in autonomous charter schools in urban areas, are negatively associated with NAEP math and reading test scores. Nevertheless, institutional factors such as these are not definitive in the literature, and their results should be viewed with caution.
Those who are interested in understanding why Michigan and Detroit students lag behind the rest of the nation in NAEP scores might explore some of the variables discussed above. There is not, however, any one variable or combination of variables that appears to serve as a sole and consistent predictor of NAEP performance, and this will pose a challenge for both understanding and devising solutions to the matter.
Bibliography
Abedi, J. & Lord, C. (2001). The language factor in mathematics tests. Applied Measurement in Education 14(3), 219-234.
Abedi, J., Lord, C., & Hofstetter, C. (2001). Impact of selected background variables on students’ NAEP math performance. Center for the Study of Evaluation, University of California, Los Angeles.
Biddle, B.J. (1997). Foolishness, dangerous nonsense, and real correlates of state differences in achievement. Phi Delta Kappan 79(1), 8-13.
Byrnes, J.P. (2003). Factors predictive of mathematics achievement in white, black, and Hispanic 12th graders. Journal of Educational Psychology 95(2), 316-326.
Card, D. & Rothstein, J. (2007). Racial segregation and the black-white test score gap. Journal of Public Economics 91(11) 2158-2184.
Carney, M. & Loeb, S. (2002). Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis 24(4), 205-331.
Fitzpatrick, M.D. (2008). Starting school at four: The effect of universal pre-kindergarten on children’s academic achievement. The B.E. Journal of Economic Analysis & Policy 8(1) 1-38.
Fuchs, V.R. & Reklis, D.M. (1994). Mathematical achievement in eighth grade: Interstate and racial differences. National Bureau of Economic Research. Working Paper No. 4784.
Grissmer, D., Flanagan, A., Kawata, J., & Williamson, S. (2000). Improving Student Achievement: What state NAEP test scores tell us. RAND Corporation.
Guthrie, J.T., Schafer, W.D., & Huang, C.W. (2001). Benefits of opportunity to read and balanced instruction on the NAEP. Journal of Educational Research 94(3), 145-162.
Hemphil, F.C. & Vanneman, A. (2011.) Achievement gaps: How Hispanic and white students in public schools perform in mathematics and reading on the national assessment of educational progress. Statistical Analysis Report. NCES 2011-459. National Center for Education Statistics.
Hyde, J.S. & Linn, M.C. (2006) Gender similarities in mathematics and science. Science-New York Then Washington 314(5799), 599.
Krashen, S., Lee, S., & McQuillan, J. (2012). Is the library important? Multivariate studies at the national and international level. Journal of Language & Literacy Education 8(1), 27-36.
Lubienski, S.P. (2002). A closer look at the black-white mathematics gaps: Interactions of race and SES in NAEP achievement and instructional practices data. Journal of Negro Education 71(4), 269-287.
Lubienski, S.P. (2006). Examining instruction, achievement, and equity with NAEP mathematics data. Education Policy Analysis Archives 14(14), 1-33.
Mcgraw, R., Lubienski, S.P., & Strutchens, M.E. (2006). A closer look at gender in NAEP mathematics achievement and affect data: Intersections with achievement, race/ethnicity, and socioeconomic status. Journal for Research in Mathematics Education 37(2), 129-150.
McQuillan, J. (1998). The literacy crisis: False claims and real solutions. Portsmouth, NH: Heinemann.
Nelson, F.H., Rosenberg, B., & Van Meter, N. (2003). Charter school achievement on the 2003 National Assessment of Educational Progress. American Federation of Teachers, AFL-CIO.
PInnell, G.S., Pilulski, J.J., Wixson, K.K., Campbell, J.R., Gough, P.B., & Beatty, A.S. (1995). Listening to children read aloud: Data from NAEP’s integrated reading performance record (IRPR) at grade 4. National Center for Education Statistics.
Thomas, J. & Stockton, C. (2003). Socioeconomic status, race, gender, & retention: Impact on student achievement. Essays in Education 7.
Tate, W.F. (1997). Race-ethnicity, SES, gender, and language proficiency trends in mathematics achievement: An update. Journal for Research in Mathematics Education 28(6), 652-679.
Vanneman, A., Hamilton, L., Anderson, J.B., & Rahman, T. (2009). Achievement gaps: How black and white students in public schools perform in mathematics and reading on the National Assessment of Education Progress. Statistical Analysis Report. NCES 2009-455. National Center for Education Statistics.