UFT WILL CALL MATRIX BIG VICTORY, BUT...

In 2017, expect to hear the word matrix early and often when UFT leaders come in to schools to sell the new teacher evaluation system as the greatest thing since sliced bread. It might be simpler to figure out than the current John King imposed system the UFT praised back in 2013 but still the devil is in the details on whether or not teachers will be better off under the new 2016 system.

This blog has already commented on the pure incompetence or complicity of the UFT leadership for agreeing in negotiations to more minimum observations than the 2015 state law mandates. Observations are usually either a gotcha game where administrators are looking to set up teachers to see them at their worst or they are something we all have to get over with because it is required so administrators and teachers go through the motions to fulfill mandates. Occasionally, a teacher receives a valid suggestion on how to teach a topic in a better way if someone is working for a decent supervisor.

I am observed these days by a professional assistant principal who generally has a different approach to teaching compared to me so I have benefited by listening to a number of his suggestions since I am new to a progressive environment. I have been lucky that administrators have not used the observation process to attack me throughout my career. However, I have seen it happen to others including my own wife. When administrators are out to get a teacher, the observations are used as a dagger to ruin a teacher's life. In the final analysis, the observation process is completely subjective so perhaps we should be grateful there are other measures involved in rating us.

Keeping this in mind, it is not unreasonable to ask if there is a legitimate growth model that can be used for the Measures of Student Learning portion of teacher ratings. The answer is that none has been developed and researched as of now that is sufficient for high-stakes use on teachers. Student test score results on exams that were never designed to rate teachers do not qualify as an objective measure of teacher performance. This is from the American Statistical Association.  VAM stands for Value Added Models, a type of growth model.

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.  

Well duh!

Enter into this picture UFT President Michael Mulgrew and Department of  Education Chancellor Carmen Farina. Their "cutting-edge" plan is to create brand new assessments to calculate the Measures of Student Learning portion of our ratings in addition to the tests already in use. MOSL Committees will have to decide what is best for teachers and students in every school or else utilize a default measurement from the Chancellor.

Here is what the UFT says in their guide on what the new measurements will be:

MEASURES OF STUDENT LEARNING (MOSL) 
In addition to current MOSL options, other options will be designed to minimize standardized testing in our schools. These will fall into four categories, two existing options and two new options. The new assessments will be developed in 2016-17 for use in 2017-18 and the UFT and the DOE will consider how and where to expand in the future. 

1. Project-Based Learning Assessments are a new category wherein a student’s final assessment is at least partly composed of work the student has developed over time in conjunction with a specific project-based learning unit. These projects and/or units must provide a student with the opportunity to demonstrate standards-based academic growth.

2. Student Learning Inventories (also new) are collections of student work that will include both DOE-developed components as well as classroom artifacts (student work) that capture student growth. 

3. Performance-Based Assessments are assessments the UFT and the DOE have collaboratively developed to learn how well a student understands and completes a specific task. These assessments are already a part of our evaluation system but will be expanded into other grades and subjects starting in 2017-2018. 

4. Progress Monitoring Assessments are third-party assessments that allow teachers to assess academic performance. Examples include Degrees of Reading Power and Teachers College Reading and Writing Project. These assessments are part of our current evaluation system.

Which leads to some reasonable questions:

• Is there any scholarly research to back these assessments up for rating teachers?
• Who is actually going to make up these new and improved assessments?
• Who will judge the validity and reliability of the new tests for rating teachers? 
• What about the possibility for paperwork nightmares for teachers in the new assessments?
• Do all of the assessments count for students or only for teachers?
• Who grades the new assessments?

Don't worry says Mulgrew because his matrix from New York State law will work in your favor.



Let's examine Mulgrew's argument.

• If the principal doesn't like you and rates you ineffective, fear not because the growth of the students on the assessments will give you an overall developing rating as long as the MOSL is at least effective.

• If the students don't show the growth they are supposed to, no need to fret because the principal can still give you that developing rating if the observations are at least effective.

This all sounds favorable. However, if you are not liked by the principal and the kids don't show sufficient growth for at least an effective on the one high-stakes measurement, then you are out of luck and rated ineffective.

Remember, there is only one measure in the new system for the entire MOSL portion of the annual rating. There are no more state and local assessments. We can refer to it as a super-high-stakes assessment.

Instead of being critical, however, let's take the UFT at their word that they have actually come up with a system where it will be more difficult, if not impossible, for a viscous principal to rate a teacher ineffective compared with the current system or the old satisfactory or unsatisfactory system.

I leave it to you to judge if it is better to use unproven, unscientific assessments rather than principals exclusively to give us an annual rating.

You can bet the DOE has already looked at all of this. Their lawyers are one step ahead as they are bringing teachers who are rated developing (still at the sole discretion of the principal) up on incompetence charges in state termination (3012-c) proceedings. If the best a principal can do by writing up terrible observation reports is to give a developing, we can probably expect low rated observations to be the norm for many more teachers while certain administrators figure out how to play with the assessments to ensure teachers they don't like are rated developing or ineffective on their MOSL which will sink the ratings entirely.

This leads back to the question of why on earth the Union would agree to a minimum of four observations per year when state law requires only two? Why not get it over with rather than prolonging the agony by having vindictive administrators impose insane improvement recommendations?

UFT will argue that the burden of proof will still be on the DOE if a tenured teacher rated developing is brought up on incompetence charges. To which we should respond that the burden of proof in the past was always on the DOE in dismissal hearings for every tenured teacher.

However, in the 2013 system and the new and improved 2016 system, after two ineffective ratings, there is a presumption of incompetence and a teacher has to prove he/she is not incompetent in termination proceedings. Good luck with that.

I hope one of our High School Executive Board people will ask for some statistics on how we are doing in the double ineffective incompetence dismissal hearings now that we are in year four of a test based teacher evaluation system.

As for an alternative teacher evaluation system, I think at some point in the future if we ever become a real union again and fight for true teacher empowerment, perhaps a system like the Montgomery County Maryland PAR system would be a good place to start discussions.

Unfortunately, I don't think even this progressive evaluation system would work in NYC because of the trust needed between labor and management for it to succeed. Trust may exist at the top levels between the UFT and DOE but at the school level our system is now all over the place. There are schools with trusting relationships between teachers and administration; there are schools where there is a hostile work environment and everything in-between exists too.

In the end, the evaluation system is still a mess here in NYC. Only the UFT leadership and Mayor Bill de Blasio's administration will try to say otherwise.

To see how much of a mess it still is, let's do a very simple poll with one question for every teacher who has been in the system since 2012:

Would you rather be rated on the matrix or the traditional satisfactory or unsatisfactory system?