Quasicohort and Period Measures Are Weak Proxies for the New Federal Graduation Rate : Florida Data , 2003 ‐ 2009 Graduation Years

Analysis of newly-released data from the Florida Department of Education suggests that commonly-used proxies for high school graduation are generally weak predictors of the new federal rate.

In February 2012, the Florida Department of Education released data on high school graduation using the new, mandated federal definition of graduation rate (Florida Education Information & Accountability Services [FEIAS], 2012).This new definition requires states to calculate four-year graduation measures using longitudinal data, following ninth-grade students through graduation, continued attendance, attrition, or transfer.Because the report contains historical data from the 2003 graduating class through 2011, this report provides an opportunity to compare the longitudinal rate to proxy measures researchers have been using for more than a decade (see Warren & Halpern-Manners, 2009, for a discussion of methodological issues).Some proxies have been "quasicohort" in using administrative records of enrollment and graduation across several years, with the assumption (or hope) of a strong relationship between reported cross-sectional data and the data from following students longitudinally.One proxy is a period measure synthesizing a hypothetical cohort from cross-sectional data in two successive years.This short paper is an initial comparison of the most commonly-used proxy measures to the new federally-defined rate.

Method Data
As of early 2012, sufficient information from the Common Core of Data (CCD) is available to allow calculation of the proxy measures for the 2003-2009 graduating years, and with 67 countywide school districts, this provides data for 469 county-year combinations.Elements drawn from the CCD include enrollment in eighth through twelfth grades for the years before graduation and regular diplomas reported in the year of graduation.Data from FEIAS (2012) include the reported federal graduation rates for 2003-2009.One small county's 2008 data were excluded (Gilchrist) because of abnormally low diplomas reported, leaving 468 records used in this paper.

Measures
For each county-year combination, the following quasicohort or period measures were calculated from CCD elements: Basic completion rate, 8 th and 9 th grade bases (hereafter BCS-8 and BCS-9; see Haney, Madaus, Abrams, Wheelock, Miao, & Gruia, 2004).This quasicohort measure divides the diplomas in the graduation year (t) by the eighth-or ninth-grade enrollment five or four falls previously (t-5 or t-4).The recognized weakness of BCS-9 is the prevalence of grade retention in 9 th grade.As Warren and Halpern-Manners (2009) note, neither BCS-8 nor BCS-9 adjust for student mobility.
Averaged freshman graduation rate (AFGR; see Seastrom et al., 2006).This quasicohort measure also compares diplomas in graduation year t to an estimate of the starting cohort, and it attempts to adjust for grade retention by using an average of eighth-, ninth-, and tenth-grade enrollments as the denominator, from the fall of years t-5, t-4, and t-3, respectively.This measure does not adjust for mobility.
Cumulative promotion index (CPI; see Swanson, 2004).This measure is a period measure, chaining one-year quasicohort promotion ratios (e.g., tenth-grade enrollment in year t divided by ninth-grade enrollment in year t-1) from ninth grade in year t-1 to diplomas reported for year t.

Summary measures and bivariate relationships
Tables 1 and 2 display summary statistics and a correlation matrix for these measures.The medians and means of BCR-9 and CPI are closer to the federal rate median and mean than AFGR or BCR-8, which are biased upwards against the federal measure.The standard deviation for the federal rate is smaller than the standard deviation for the proxy measures.The correlation matrix shows that AFGR and BCR-9 have the highest correlations with the federal rate, but each of them still have only moderate associations with the federal definition, with an R 2 of .54 and .53,respectively.Another way to explore the relationship between the proxy measures and the federal graduation rate for Florida 2003-2009 is through Figure 1, where each panel shows a different comparison.

Residual analysis
Because the proxies have been commonly used as substitutes for true cohort measures, it is also important to examine the residual errors for each proxy.Table 3 provides summary statistics of the residuals (no decimals displayed for percentages).BCR-9 and CPI have the least bias (-2% for BCR-9 and 2% for CPI).The range for residual errors is greatest for CPI (78%) and lowest for AFGR (42%), and the same is true for standard deviation (9% for CPI as highest, 6% for AFGR as lowest).The skew is positive for each residual set, but the median is very close to the mean in each case. .

Discussion
This paper analyzes the performance of commonly-used proxy measures with the first public release of multiple years of graduation rates using the new federal definition that requires longitudinal tracking.In general, the proxy measures perform poorly, with two clearly-biased measures (AFGR and BCR-8), and with all measures having no better than moderate correlations with the new federal rate.
At least using the data available for Florida from the state department of education and the Common Core of Data, all researchers' attempts to improve on a basic quasicohort rate with administrative data (BCR-9) have resulted in measures that are either more biased, or with lower correlations with the federal rates, or both.Several measures available using population data, to adjust for migration, have more promise, but these measures are not available at the school or district level (Warren & Halpern-Manners, 2009).The conclusion is sobering but perhaps expected: you cannot measure your longitudinal graduation rate until you do so, and one should not accept any substitute, no matter how attractive.

Table 1 .
Summary statistics, graduation measures

Table 3 .
Summary statistics, residual error of graduation measures