Quality Counts, but it also depends on what you are counting

For those of you who know me, you know that I am a math nerd.  I will admit it. My mom tells a great story about how I used to cut my toast into fractional pieces when I was little and practiced adding halves and fourths at the table, and she knew then I was destined to be a math teacher. I think she may be exaggerating, but it makes an impressive dinner party story.

My love of mathematics has drawn me to statistics.  I am fascinated by the work psychometricians and data analysis people do with numbers.  A big part of that fascination is knowing how much running one type of test over another or asking certain types of questions can change the outcome of the results. In addition to fascination. It also tends to make me angry with those who take advantage of and twist data for their own purpose.  A favorite quote of mine is from Mark Twain: “There are lies, damned lies, and statistics.”

I will admit, every time I see an educational statistic, I think of Twain.  Especially when they are out of context and being used to serve a political agenda. NAEP, PISA, A/F Report Cards, Value Added Evaluations… the list of data and statistics being collected and twisted in the name of educational reform is, unfortunately, growing.   

The latest “shocking” grade for OK was the 2014 Quality Counts Reports, which puts Oklahoma in the bottom 10 states in the Country in Education.  This report was published by Education Week’s Research Center. By the way, you can buy copies of Oklahoma’s report and even recalculate OK’s grade based on what you think is important.  

Social Media came alive with people sharing the news about how bad Oklahoma did.  News outlets jumped all over it.  Some of the headlines were:

“‘Quality Counts’ report shows need for Vigorous Education Reform in Oklahoma” (NewsOK Editorial) The author of this editorial obviously didn’t even read the report.

Oklahoma Schools get C+ grade in Quality Counts report.” (Tulsa World)

Oklahoma Ranks 10th Nationally in K-12 Achievement” (Channel 9) This one is extra frustrating becuase the headline makes it look like we are doing OK.  10th out of 50 isn’t bad right. But the first sentence of the article points out that we are ” number 10 on the list of states with the worst school systems.”  Not 10th best…10th worst.

We’re Number 44!  Let’s Give our Policy Makers an F.” (Tulsa Kids Magazine).  This one actually talks about what the report actually means.

“Education report: Oklahoma ranks well below national average for academic achievement, school spending” (NewsOK).

In the last one, SDE officals report that “the results of the report were not a surprise given the number of recent education reforms implemented to better prepare students for the national tests” and Superintendent Barresi  says “The grade confirms what we already knew, that we must change our approach to education.”

Let’s take a look at what Ed Week’s Report, titled “District Disruption and Revival: School Systems Reshape to Compete and Improve” took into account.  

“This year’s report focuses on school district governance and operations as its special theme, examining the impact of the increasingly complex fiscal, political, and technological forces that are challenging school districts and prompting efforts to cope with new pressures. Education Week journalists take an in-depth look at the prominent developments—including school choice initiatives, district mergers, and federal policy shifts—transforming the traditional environment for education governance.”

So we were the 10th worst state in the US based on ” fiscal, political, and technological forces that are challenging school districts.”  Barresi might be right.  We must change the approach, because obviously the fiscal, political and technological forces are not working in Oklahoma.

Here is the breakdown for Oklahoma.


Let’s break it down some more.    Our highest score was in Standards, Assessment, and Accountability.  Ed Week’s score card looks like this:


That is not very informational data. It is simply a bunch of yes or no questions based on values and characteristics deemed important by the researchers at Ed Week. Let me point out some of my favorites in there list:

Rewards for good schools…why, yes we provide rewards to high performing school.  Good thing this doesn’t take into account how many of the 229 Reward Schools took advantage of the schools last year  “reward” last year, which was fourteen of them or 6%.

Assistance and Sanctions, sure we’ve got them. We have all kinds of categories and rules for schools that don’t make the grade.  But it’s a good thing Ed Week didn’t take into account how long it takes the state to actually let schools know if they are a Focus, Targeted, or Priority School, which as of December of the 2013-2014 school year was still not released.  So that’s 50% of the school year wasted for the schools that need the most “intervention.”  

But, we have standards for ELA, math, science, and social studies.  They have been around a long time.  I am not sure what the supplementary material is that EdWeek gave us a YES grade for.  But I guess it is out there somewhere.

We received a bunch of NOs in the assessment section, including a NO for not having “vertically equated” scores in 3rd through 8th grade math or English.  That means the score on the 4th grade test means nothing compared to the 3rd.  And the 5th grade score has not relation to the 4th.

Wait…Isn’t that what the Value Added Models rely on?   Apparently, the SDE hasn’t looked at this breakdown yet. More lies, damned lies, and statistics.

Every thing in this report is a YES or NO score. I do not see how this is valuable or useful.

Here is some of the breakdown for teacher quality:(pages 13-14 in Oklahoma’s Highlights) I strongly believe that the vast majority of teachers in Oklahoma are doing their best and are passionate about kids and learning. They are trying to instill a love of learning while providing a safe place for kids.

Requirements for Licensure: (not including alt cert or TFA)

Substantial coursework in subject area(s) taught: Yes 
Test of subject-specific knowledge Yes 
Test of subject-specific pedagogy No
Student-teaching during teacher training Yes*
Other clinical experiences during teacher training Yes* (I can’t find what the asterisk means on the last two, but I would guess it means Universities vary or not for alt

Evaluating teacher performance (2011-12) – Thank goodness for TLE. We got 4 YESes! (sarcasm intended)
Formal evaluations of all teachers’ performance required Yes 
Student achievement is tied to teacher evaluations Yes
Annual basis for teacher evaluations Yes
All evaluators of teachers receive formal training Yes

Data systems to monitor quality (2011) – Don’t worry.. we are working on it.  Next time they check this, we will get the YES.  

State links teachers to student-growth data No
State links teachers and their performance data back to teacher education programs No

Salaries and incentives
Teacher-pay parity – Teacher salaries at least equal to comparable occupations (2010) No (OUCH)
Districts report school-level salaries for teachers (2011-12) No
Pay-for-performance program or pilot rewards teachers for raising student achievement (2011-12) Yes
Differentiated roles for teachers formally recognized by state (2011-12) No (this could happen with programs like the Elementary Math Specialist or Reading Coaches) 
Incentives for teachers taking on differentiated roles (2011-12) No
Financial incentives for teachers to earn national-board certification (2011-12) No (If the State like National Board more, we could have gotten a YES)

*notice the years next to everything.  I don’t think they even use current data.  The 2014 Quality Report used data from 2011-2012. 

The Building Capacity section was pretty pitiful.


The items that would support and build capacity, our state leaders either don’t fund or don’t protect.

I could go on.  But let me end it with this:

Next time someone asks you “Did you see that Oklahoma ranked the 10th worst education state in the Nation?  Our teachers unions, greedy educational establishment , and liberal teachers are ruining everything.”  Ask them if the read the report.  And rest assured that the low grades (and even the high grades) we received in this report have little to nothing to do with the amazing teachers and the job they are doing.  It has little to do with what is actually going on in our schools.

It strongly laundry list of political talking points and reforms agenda items.  

I agree with Barresi.  “The grade confirms what we already knew, that we must change our approach to education.”  Starting with the Office of the State Superintendent and the reforms that are taking Oklahoma schools hostage.