Date: Tue, 19 Sep 2000 14:05:50 -0400
From: "Trent, Roger" <Roger.Trent@ode.state.oh.us>
To: 'Allen Wilkinson' <aw@stratos.net>
Cc: "Bowers, Bob" <Bob.Bowers@ode.state.oh.us>,
     "Speakman, Claudia" <Claudia.Speakman@ode.state.oh.us>,
     "Oxender, Jeannette" <Jeannette.Oxender@ode.state.oh.us>,
     "Trent, Margaret" <Margaret.Trent@ode.state.oh.us>,
     "Crandell, Jan" <Jan.Crandell@ode.state.oh.us>,
     "Rochester, John" <John.Rochester@ode.state.oh.us>
Subject: RE: Some Cleve. Hts-Univ. Hts local research on proficiency relevant to the state

Thank you for a copy of your summary and the invitation to comment regarding the questions you pose.  Rather than copy all policy makers who were copied on your memo, I have copied on this reply Department staff who work with the Governor's Office, the General Assembly, the Governor's Commission for Student Success, and the State Board of Education (because one or more of them is likely to get questions from one or more of the individuals copied on your memo).

I appreciate what you are trying to do, but believe you have overlooked THREE extremely critical facts that, to a large extent, can explain much of the differences you have noted.

First, there is the issue of PURPOSE.  The 9^th grade proficiency tests were originally developed as MINIMUM COMPETENCY tests.  The only objectives measured are considered minimal for a 9^th grade student to know or be able to do and the passing score was set as the minimal level expected of a proficient 9^th grade student.

Tests at grades 4 and 6 on the other hand were intended to help us assure that any student needing intervention in a test area received that intervention in the next grade level (or certainly before being required to take the tests for graduation).  The "final proficient score standards" is the score standard many members of education advisory groups, Department staff and the vast majority of State Board members represent the score that will best assure those who need help will get it.

Except for the 4^th grade reading test, all other tests at 4^th and 6^th grade have a small proportion of content that committees of educators believe is "advanced" for the typical student at that grade.  A score standard is set for each of these tests to indicate performance at this level on each test. Although the content of the 4^th grade reading test does not support the advanced standard in any real sense, we also have identified such a score standard on each form of this test -- it is typically one or two raw score points less than a perfect score (meaning there is a kind of ceiling effect on this test since the content does not differentiate performance very well much above the proficient score level).

The second difference between 9^th grade tests and tests at 4^th and 6^th grades is the number of attempts students have to pass.  Students at the lower grades (and at grade 12) have ONLY ONE attempt to pass.  By the end of 9^th grade virtually all students have had at least THREE attempts to pass (and many students have had more).  The report cards show the cumulative percentage who have meet the proficient standards by the end of the grade whether it is one attempt or more.

At 12^th grade a significant factor is the number used as the denominator for calculating percent proficient.  The denominator does NOT include any student who dropped out of school before February of the senior year or anyone who has not passed all 9^th grade tests by January 1st of the senior year.  In other words, any student who arguably might be one of the lowest performing students is systematically eliminated from the population included in the denominator at 12^th grade.

Other factors that relate to differences in performance of districts on the different tests at the different grades include the following (although certainly not an exhaustive list):

-- At 12^th grade the performance standard for districts is 60% meeting proficient standards, as compared to 75% at 4^th and 6^th grades (after one attempt) and the same percentage at 9^th grade (after 3 or more attempts).
 

-- You mentioned different test development firms.  However, these firms were NOT given free rein to develop test questions.  They were given direction in what we call test and item specifications, developed and/or approved by committees of Ohio educators.  And, these specifications are very different at grades 4 and 6 than at grades 9 and 12.  That is, all tests at the lower grades have a significant number of constructed response questions; at grade 9 all tests except writing are totally multiple choice; there is one (or perhaps two) gridded response question in mathematics but the remainder of the test questions in all tests except writing are multiple choice.

-- Then there is the committee that recommends score standards.  The score standard for each test at each grade level was recommended by a different committee of Ohio educators.  Even though similar procedures were used for all tests at any given grade level and all committees at a given grade level were selected using the same criteria, I know of no absolute measure of the
degree to which the rigor of two performance standards differ from one test to another test at the same grade or for the same test from grade to grade.
 

Again, thank you for the opportunity to respond.  I only wish we could have had an opportunity to offer suggestions BEFORE the information was shared widely.

Roger