“That Information Isn’t Available from U.S. News”

Last week, I saw the list of “Best High Schools” as reported by U.S. News and World Report.   Once again, U.S. News dubiously listed BASIS Charter Schools in Arizona as the best high schools in our nation.  In fact, six of the ten “best high schools” were BASIS charter schools.  Criticism over the identification of these highly selective charter schools as the “best high schools” continues.  Some educational experts claim that these selective charter schools purposely manipulate which students they admit into these charter schools and which exams to offer in order to make the U.S. News “best high schools” list.

Once a reader begins moving beyond the top 10 or top 25 high schools, he/she will notice that there are non-charter public high schools on the list.  So, why isn’t Clarkstown North one of these schools?  If you read the manual (see page 5), “Identifying Top-Performing Public High Schools for the ‘Best High Schools’ Rankings,” you will find the four step process used by U.S. News to determine the “best high schools.”  Every high school must pass step one before moving onto the next. U.S. News informed us that we did not meet all the criteria for step one.  Clarkstown North missed the performance expectation index for economically disadvantaged students.  For Clarkstown North to advance to the next step, we would have to have had a performance index above 140 to qualify.  U.S. News calculated the performance index for economically disadvantaged students for Clarkstown North as 139.2 (see graph above).

For the past two days, I have attempted to discover how U.S. News obtained a performance index of 139.2 for economically disadvantaged students.  Pages 10-11 of the manual explain how the performance index is calculated.  Then, I looked through the manual to find what state assessments were included to determine the reading and mathematics score.  According to page A-3, Regents examinations were used to calculate performance indexes.  However, which Regents exams were included in these calculations are not listed in this documentation.  Were results from the two required Regents for graduation used (i.e. Algebra I and English Language Arts) or were all Regents results included (English Language Arts, Algebra I, Geometry, Algebra II, and Algebra II non-Common Core)?  I ran a variety of calculations for the economically disadvantaged group of students, and I was never able to derive a performance index of 139.2 as U.S. News reported.  Which exact state assessments were used is still not clear.

Another question I had is how the statistical expectation of 140.0 (i.e. the line of best fit through a multiple regression analysis) was determined for Clarkstown North.  According to the New York State Report Card, Clarkstown North’s performance indexes for economically disadvantaged students was 173 in English Language Arts and 159 for Mathematics, which surpassed the statistical expectation of each (150 and 134, respectively).

When U.S. News was contacted about which state assessments were used and how the performance index for economically disadvantaged students was calculated, Robert Morse (Chief Data Strategist for U.S. News and World Report) responded: “Your question is = will usnews provide you and/or others excel sheets or some other format of the raw ranking data we used to compile NY state. The answer is that isn’t available from USNEWS.”

Carol Burris, John Murphy, and Sean Feeney pointed out a few years ago that U.S. News used the wrong data in its calculations.  Could this have been repeated again?

To know what assessments were used and how the performance index was calculated is a reasonable and simple inquiry.  The inability to respond to this request is disappointing, especially when U.S. News announces through a media blitz about its “Best High Schools” list that becomes either a source of pride or a source of disappointment among local communities.  The inability to answer a simple question is highly troublesome, and the “Best High Schools” list is highly suspect.  Is the U.S. News list any better than that of Niche.com, which rates Clarkstown North as an A+ high school (64th in New York State)?  We may never know the answer to this question.