CHEYENNE – As the Accountability in Education movement in Wyoming continues, the Wyoming Department of Education (WDE) and its partners in the Accountability effort are developing the model used to rate school performance.
Former Casper educator Dr. Mike Flicek, has been working with the WDE to refine the model. He said the current School Performance Rating Model is in draft form, but was presented to legislators and the State Board of Education in May.
The hope is the model will be ready to rate schools following the 2013-14 year. The model can be seen in draft form in its entirety here.
"Accountability has a number of facets and components," said WDE Interim Director Jim Rose. "Thanks largely to the cooperation from the Legislative Service Office (LSO), we have been able to enlist the help and expertise of consultants. Their contributions are one of the principal reasons for the progress we've made.
"Through the leadership of the WDE staff, we have been able to make significant progress on the assessment aspects that are essential to measuring the progress of improvement," Rose continued. "Much of the work must be collaborative in order to fully respond to what is needed. WDE staff have demonstrated extraordinary willingness to forsake, 'turf,' and domain control in order to achieve the progress we are making."
Understanding Who is Involved
Before looking too deep into the model itself, there are some committees whose names will come up more than a few times in this process.
The Wyoming Legislature has enacted the accountability process by passing laws implementing an accountability process as far back as 2010. They have worked through two Legislative Committees – the Joint Education Committee (found here) and the Select Committee on Educational Accountability (found here). Those two committees do the lion's share of the work in ordering studies and developing policy, which they take back to the Legislature as a whole.
These two committees also work through two smaller groups that they rely on for advice and the perspective of educators.
The Advisory Committee to the Select Education Accountability Committee was created in 2011 to assist the Select Committee with its work on statewide education accountability. In 2013, the Legislature specifically charged the Advisory Committee to work with legislative consultants in the study and design of a teacher and school district leader evaluation and accountability system. The Advisory Committee will help to set the weights for indicators that will go into the school performance rating model.
The current members of the Advisory Committee are: Tony Anson (Big Horn 4 Principal); James Bailey (Superintendent, Uinta 1); Janine Bay-Teske (School Board Member, Teton 1); Diana Clapp (Superintendent, Fremont 6); Sue Belish (State Board of Education); Kris Cundall (Principal, Sweetwater 1); Marykay Hill (Governor's Office); Tess Hopkin-Egger (Elementary Teacher, Park 6); Deb Lindsey (WDE); John Metcalfe (Assistant Superintendent for Curriculum, Fremont 1); Jack Patrick (Teacher, Carbon 2) and Bill Schilling (Wyoming Business Alliance).
The Professional Judgment Panel (PJP) is a 27-member panel of teachers, administrators and members from higher education as well as the business community. Their role is to set cut scores for each of the indicators to help categorize schools as exceeding expectations, meeting expectations or below expectations. The PJP will meet in September to set the targets. Members of the PJP were selected by the State Board of Education based on nominations it received in March of 2012 as directed by the Legislature.
The PJP will meet Sept. 16-18 in Casper to work on setting the cut scores to be used in the school performance model.
Indicators for the School Performance Model
Currently, the school performance model is housed in a 15-page document (see PDF attached) and describes all of the indicators, which would contribute to a school's rating. For grades 3-8, the indicators will be: achievement, growth and equity measured by growth. For grades 9-12, the indicators will be achievement, readiness and equity measured by achievement gap.
Perhaps the easiest component of the School Performance Rating Model is achievement. According to the draft report, there will be one achievement score per school and it will be a percent of students who scored proficient or above on achievement tests. The current achievement tests include:
- The Proficiency Assessment for Wyoming Students (PAWS)
- Reading in grades 3- 8
- Math in grades 3- 8
- Science in grades 4 and 8
- Student Assessment of Writing Skills (SAWS) in grades 3, 5, and 7
- The ACT
- Reading test in grade 11
- Mathematics test in grade 11
- Science test in grade 11
- Writing Test in grade 11
Student growth is simply a measure of how a student's test scores grew compared to that of all other students in the state in the same grade which has similar test scores the year before. Other students in the same grade with a similar history of test scores can be thought of as "academic peers." The question we can then answer about my growth in reading or math is, according to Flicek, "how did my growth compare to that of my academic peers?"
Flicek said, "The actual growth score is a Student Growth Percentile (SGP). Those scores range from 1-99. If a student gets a Student Growth Percentile (SGP) of 50, then they did as well as or better than 50 percent of their academic peers."
Each school will have one growth score called the Median Growth Percentile (MGP). The MGP is found by taking each individual student SGP for math and reading for the entire school and ranking them from the highest to the lowest. Then, find the score that has half the individual SGP's above and half of the SGP's below. That number is the MGP for the school.
Because two years of achievement are needed to compute growth, this component will not be calculated for children until they have completed two years of the State Assessment in grade four. Growth will be an important component in the rating of elementary and middle schools, but will not be a part of ratings for schools only holding grades 9-12.
Equity is an easy thing to define where mortgages or car loans are involved, but equity in educational accountability is a little tougher. In this case equity basically means how schools minimize the achievement gaps in its students, from the highest scoring to the lowest.
"It (Equity) tells us how well a school is doing at moving their students from below proficient to proficient," Flicek said. "That is what we are looking for."
There will be two methods to measure equity in Wyoming--one for grades 3-8 and another for high schools. For the elementary schools that have growth scores, equity will be the percentage of students who were below proficiency in the prior school year on the state assessment whose current growth on the state test s strong enough to suggest the student will become proficient within the next year.
"It is really relatively straightforward," Flicek said about the equity measure. "There is a growth score called the adequate growth percentile (AGP). That tells us the growth percentile a student would need this year to be considered on track to become proficient within a year or two. Some of the specific details about this are still in development."
The state will also make use of one consolidated subgroup in its measurements—all students who were not measured as proficient or advanced in a subject by the state test.
There will be another method for figuring out equity, necessitated by the fact there is no growth measure at the high school level. For those schools, an alternative method of equity will be offered. The first component of their equity will come from the current year achievement gap between the consolidated subgroup (all students who weren't proficient according to the state test) and the statewide performance of all schools. The second will be the improvement in that achievement gap from the prior year to the current year. Equity for these schools will be a function of the size of the achievement gap and the amount of improvement (i.e., reduction to the gap) from the past year to the current year.
Readiness is a component made up of four sub indicators and will be measured at all schools offering a diploma. Two of the indicators are called leading indicators and two are lagging indicators.
The first of the leading indicators is tested readiness as measured by the ACT suite of tests (ACT EXPLORE; ACT PLAN and the ACT). Each student at a school who takes one of the ACT suite of tests will be given index points based on their composite score on the test. For example, a student who scores a composite on the ACT of between 1-16 may not receive any index points. A student could receive up to 100 points for an ACT score between 25-35 (The advisory committee to the select committee on educational accountability has yet to offer a final scale of index scores). All of the index points earned by students in the school will be divided against the number of students tested in the school and a tested readiness score will result.
The second leading indicator is the percentage of ninth graders with at least one-fourth of the credits they need to graduate high school.
The first of the lagging indicators is graduation index of a school. Index points will vary from 0, for dropouts, to 100, for four-year graduates. Index points will also be assigned for students who graduate in more than four years. The advisory committee will reach a consensus recommendation on the number of index points assigned for graduation in more than four years.
The final lagging indicator will be Hathaway Scholarship Eligibility Level of all graduates. In other words, a student who is eligible for the Honors level of the Hathaway can earn its school 100 points, while students who are not eligible for the Hathaway will not net any points for their school. The school's average will be the mean of the student points for the graduating class at the school.
The Advisory Committee is still working through how to weigh all of the indicators to come up with one readiness score.
Going Forward with the Model
The next step for the School Performance Model will be to put it into practice and begin to refine the model and the business rules associated with the model. Once the cut scores are set by the PJP in September, the WDE will populate the model with 2012-13 data. Schools will be notified of their categorical ranking, but supports and interventions will not take effect until the 2014-15 school year.
"There will be a trial run after the PJP gets done," Flicek said. "It will be applied to all schools in Wyoming. That data will be analyzed and the schools will get an actual performance level based on next school year's performance (2013-14). We are planning to give schools scores based on the 2012-13 school year as well. We definitely want to see how it works."
Flicek pointed out it is important to note that accountability was never meant as a tool for removing teachers or leaders of schools, rather to help student learning by setting up a system to support their efforts.
"The theory of action is really that we want to identify what schools are doing well and try to replicate it," Flicek said. "Also, we want to identify schools that are struggling and offer support to them so they can get better performance from leaders, teachers and students."