Why the College Scorecard Should Complement – Not Replace – the Gainful Employment Standards
The introduction of earnings data by program and by college to the College Scorecard helps students and families make more informed college choices. However, the data is not a replacement for strong accountability rules. Moreover, a recent analysis using the College Scorecard data is likely to confuse – rather than clarify – the relative value of different colleges.
The College Scorecard publishes key data on all types of colleges and universities, including student debts and earnings. The gainful employment rule (GE) – until it was repealed last summer – required career programs to publish key data and meet minimum standards. Specifically, GE requires that programs pass at least one of two debt-to-earnings (DTE) rates that compare a typical program graduates monthly loan payments to how much they earn. Programs that consistently have high DTE rates, indicating that they leave graduates with too much debt, compared to earnings, lose student aid eligibility.
Several people have written on the strengths and weaknesses of the College Scorecard data, including Robert Kelchen, professor of higher education at Seton Hall University and Michael Itzkowitz, the former director of the College Scorecard. Most recently, Andrew Gillen of the Texas Public Policy Foundation published an analysis comparing the two data sources to “help students evaluate earning potential and debt by college major.”
Gillen Analysis Overstates the Impact of Gainful Employment on Some Colleges
To assess the relationship between debt and earnings at all types of programs (not just career programs), Gillen published a “gainful employment equivalent” for programs in the College Scorecard dataset. However, the Gillen methodology overstates the number of programs that would fail the gainful employment rule, particularly at public and nonprofit colleges.
Table 1 summarizes important differences between the two data sources, showing why the numbers are not directly comparable. Most notably, the DTE ratios defined by the gainful employment rule cannot be calculated through the College Scorecard data because the students populations and the time frame for measuring earnings and debt after graduation is different. The College Scorecard also excludes private loan data that would be collected for gainful employment.
Gillen applies only one of the DTE ratios and attempts to address the differences between the data sources with an ad hoc, across-the-board adjustment that reduce DTE by the same amount for every program. However, the College Scorecard data not only produce higher DTE ratios but, at least in some respects, are likely to produce disproportionately higher rates at public and nonprofit colleges. As a result, the data cannot be reliably used to inform student choices or provide policymakers with an accurate picture of how DTE would play out among programs.
Table 1: Differences between Program-level College Scorecard Data and Gainful Employment Debt-to-Earnings (DTE) Rates
| College Scorecard (CS) | Gainful Employment (GE) | Implications | |
|---|---|---|---|
| Treatment of non-borrowers (i.e., graduates who only received Pell grants) | Debt amount only includes borrowers Earnings amount includes both borrowers and non-borrowers who received Title IV aid | Both debt and earnings calculations include all students who received Title IV aid, including non-borrowers | CS data make debt number much higher by excluding non-borrowers, raising debt-to-earnings ratios for institutions with fewer borrowers such as community colleges and some private, non-profit colleges with generous financial aid for lower-income students. It will have a smaller impact on ratios at most programs at for-profit colleges, where students almost always take out some form of loan. |
| Time frame | Debt is measured immediately after graduation Earnings are measured one year after graduation | Debt is measured as the amount of loans borrowed prior to graduation or exit (with some exceptions of the student graduated from multiple programs) | CS substantially overstates DTE compared to GE methodology because it measures earnings when many students are still trying to establish their careers. DTE rates at private, liberal arts colleges will be especially inflated since research shows earnings rise faster after graduation from liberal arts colleges than other kinds of schools. This limitation does not impact programs at other kinds of institutions as much, including for-profits, since earnings growth is typically flatter at non-liberal arts programs. |
| Counting college students who subsequently attend graduate school | Debt amount includes all students regardless of whether they continue to a higher degree Earnings amount excludes students who continue to their education (e.g., graduate school) if they are still enrolled in school after one year | Excludes students who are in in-school deferment for both debt and earnings; however, earnings is measured 3-4 years after graduation so fewer students are excluded because they are still enrolled in graduate school | CS might somewhat understate earnings at selective colleges where many students go on to graduate school and would not have their earnings counted until CS is able to measure earnings further out from graduation. These differences are less likely to impact career and technical focus programs in for-profit sector. |
| Private loans | Does not report data on private student loans | Debt calculation includes private loan amounts reported by institutions | CS data will understate debt amounts and DTE rates, especially at institutions where students commonly take out private loans. This is a major limitation for CS, but it is unclear how much it is like to impact DTE for different kinds of programs. |
| Mean or median? | Debt calculated as both mean or median Earnings only calculated as a median | Debt calculated as median Earnings is the higher of mean or median | CS may understate earnings and overstate DTE rates compared to how they are calculated for GE since mean earnings are generally higher than median (although in many cases the reverse can be true). CS may underestimate GE rates at selective institutions in cases in which a small number of students (e.g., Jeff Bezos of Princeton) make a lot of money after graduation. |
| Universe of programs | Programs at 4-digit CIP level with at least 20 graduates. Pooled two-year cohorts. An example of a 4-digit CIP is 51.06 - Dental Support Services and Allied Professions. | Programs at 6-digit CIP level with at least 30 graduates. Pooled two-year or four-year cohorts years, depending on program size. CIP codes are a taxonomy for fields of study. 6-digit CIP is the most detailed classification available. An example is 51.0603 - Dental Laboratory Technology/Technician. | CS may include a higher number of very small programs that would be excluded from GE. This may skew simple averages of programs at liberal arts colleges with many small programs in the arts and humanities. |
In addition to Table 1’s limitations, which are inherent to differences between the data sources, Gillen’s analysis excludes a few key aspects of the rule. His failure to follow model the rule leads him to overstate of the share of programs that would fail an expanded GE—especially among those at four-year colleges. For example, while failing programs must fail two debt-to-earnings ratios (one based on total earnings and the other based on discretionary income), Gillen models only one.
Table 2: Differences between Gillen Analysis and Gainful Employment Methodology
| College Scorecard (CS) | Gainful Employment (GE) | Implications | |
|---|---|---|---|
| Discretionary earnings debt-to-earning rate | Excluded | Included; programs pass if the typical graduate’s debt payment is less than 20 percent of their discretionary income | High cost, high earnings programs fail even though under GE they would pass the higher DTE threshold allowed for discretionary earnings. This major discrepancy with GE likely inflates DTE for more selective colleges, but impacts programs at for-profit institutions less severely since in many cases these programs fail both DTE metrics. |
| Amortization treatment | Appears the same timeframe for payment is applied to all programs | Students graduating from lengthier programs are assumed to spread lower monthly payments out over a longer period of time, which decreases calculated debt amounts and DTE rates. | Assuming the CS analysis applies the same length of amortization to all programs, this would effectively and set a tougher standard for four-year programs that would benefit from the GE assumption that debt payments are spread out over a longer period of time than shorter programs. This less likely to harm shorter-term career and technical programs, particularly in the for-profit sector. |
Consumer Information Is Not a Substitute for Accountability
Of course, GE not only provides information to students, it also removes access to federal student aid dollars from programs that consistently leave graduates with high debt and low earnings. These accountability standards have driven career programs to offer better value for students.
In repealing the gainful employment rule, the Department of Education argued students could use the College Scorecard to choose better programs. However, as concluded in a recent TICAS report, Consumer Information in Higher Education, passively providing information by itself has not been shown to consistently influence student behavior. Consumer information is a complement—not substitute—for strong federal accountability.
Students and taxpayers need both strong information on program information and accountability systems that will prevent colleges from systematically leaving students with debts they cannot afford. While student debt is a problem at all types of colleges, the data show that programs at for-profit colleges are more likely than other programs to leave students with excessive debt-to-earnings ratios.
[i] Recent analysis tries to adjust debt-to-earnings rates from College Scorecard data with an across the board multiplier. However, applying the same adjustment to all programs is insufficient in replicating Gainful Employment data because earnings rise relatively faster or slower after graduation depending on institution and program type.
[ii] Note that the Department of Education is still working on a methodology for how attribute students to multiple programs as longer-time data become available.
[iii] College Scorecard removed data based on complementary suppression of any data point or combination of data points that disclose information on fewer than 10 students. Complementary suppression removes medians that are calculated based on fewer than 20 students since medians reveal that half of students are below and above a certain number.
