Measuring What Matters: Non-cognitive Assessments in the Jewish Day School
January 28, 2020
Developing Parent and Employee Satisfaction Surveys in Day Schools
January 28, 2020
by RABBI DR. YERACHMIEL GARFIELD and MRS. MELISSA TRUELOVE
Few initiatives in the field of education are as promising as using data to improve learning. Data provides the necessary information to evaluate and advance initiatives in a school setting. In other fields of study, such as in science, medicine and technology, data is a critical element in refining ideas. These areas have progressed because of how data is used to assess the efficacy of new initiatives. While the field of education has made some progress by collecting and analyzing data, the use of data in our discipline is undoubtedly less robust than in other fields. There are a relatively limited number of academic journals and few researchers who attempt to leverage data to prove the efficacy of existing and new practices when compared to the amount of academic research funded in other areas. The number of evidence-based decisions and research driven studies in the field of Jewish Education is even more sparse.
Limitations of Data
Some are hesitant to use data-driven decision making because of well-known failures when data was used to change education. One of the most recognized and disappointing data driven initiatives was the No Child Left Behind Act (NCLB) of 2001 promoted by then President George W. Bush. A main pillar of this legislation was that test scores were a primary measure of success for schools which then determined eligibility for federal funding. The use of data in the form of test scores to measure school achievement held great promise. Tests could arguably provide clear and meaningful data on student achievement via immediate and non-biased feedback reflecting efficacy of each school.
It was just a matter of time before the hopes for a transformation of United States education by using standardized data was shattered. With a real threat of a loss of funding, school leaders learned how to manipulate results, cheat on tests and teach to the exams so that the data was no longer a legitimate measure of success. It also became evident that one data point was insufficient to assess a whole school. The data from the test scores did not provide a sufficient narrative to draw conclusions about the quality of education.
This failed experiment highlighted one of the major limitations of using data to improve education. Data is only useful when its source is reliable, and it measures a tangible phenomenon. In NCLB, the use of test scores was too simplistic a measure for complex organizations such as schools, and the data provided by the school was flawed. Since NCLB, lessons have been learned and new mechanisms for data collection and school evaluation have been employed that expand the measures well beyond the use of test scores so that all schools can effectively use data to improve school learning and culture. While data collection in schools may lead to great improvements, it must be done thoughtfully.
Our Story
Recently, our day school has used data collection to address a common challenge – classroom discipline. The issue addressed, and the data collection process, can be duplicated in most day schools and is a good example of how data can be used to effectively improve schools.
In September 2017, Yeshiva Torat Emet (YTE) hired a new Assistant Principal of General Studies (APGS) to oversee the General Studies program as well as to manage discipline issues for 1st through 4th grades. The APGS was responsible for 140 children spread over 4 grades with each grade being divided into two class sections – one for boys and one for girls. Each class has three teachers for Limudei Kodesh, Ivrit and General Studies.
Together with the Head of School, our APGS developed initial work objectives which included academic program improvements. Early in the year, there was a constant flow of children being sent to her office for classroom behavioral issues. There were so many office referrals for discipline issues that she did not have time to work on curricular improvements. As the year progressed, the discipline issues prevented any meaningful efforts to tackle our academic objectives. The APGS and Head of School tried to tackle the discipline issues but after a few months, there was not much progress. It became clear that the problematic behaviors needed to decrease before the APGS could tackle academic or curricular objectives.
Data Collection
At this frustrating time, the APGS attended a workshop that highlighted the use of data collection to address discipline issues. Immediately, she created a simple and inexpensive data collection system, using a free Google form, to track each office referral in an accessible manner. After only three months of data collection, we had enough data to notice patterns in our areas of challenge and devise strategies to tackle the problem.
The form had five questions:
- Name of Teacher
- Name of Student
- Location of Disturbance.
- Date and Time
- Nature of Disturbance
Data Analysis and Change
The first two questions provided the data to answer the most important questions: which teachers were referring students most often and which students had the most frequent office referrals? This crucial information allowed us to plan how to support the teachers and students who taxed the system the most.
Data Point One – Which teachers had challenges
Specific teachers received coaching and supportive strategies to improve their classroom management techniques. We also shared the data with these teachers which motivated them to bring their referrals in line with other faculty members.
Data Point Two – Which students had challenges
Likewise, with the data for the frequently referred students, the school held important conversations with parents, teachers, and providers on how best to manage behaviors. The data showed that only 6 students in the entire elementary school were responsible for roughly 38% of the office referrals. Focus was put to help identify and address those children’s needs. More appropriate resources and comprehensive behavior plans were put in place, and the data reflected success. By Pesach of that school year, 4 of those 6 students had no more visits to the principal.
Data Point Three – Location of disturbance
The third data point, location of the disturbance, also provided essential information to solve our discipline issues. Not surprisingly, many office referrals came from the recess playground. We changed how and when to staff these environments to address these challenges.
Data Point Four – Time and date of the disturbance
Information about the time and date of the disturbance yielded significant direction. We found (again not surprising), that a majority of students were sent to the office in the afternoons, specifically between 1:30 PM and 3:30 PM. We took a preventative approach, with the administration making more frequent classroom visits, proactively engaging students during afternoon break, and walking the halls during that critical time. We also shared this information with the teaching staff so they could understand why and when to reinforce expectations which prevented negative behaviors from happening in the first place.
The date of the disturbance also yielded interesting results. When the number of referrals were analyzed by month, we found that December and March had significantly higher rates of office referrals. The following school year, we strongly encouraged teachers to reinforce rules and expectations during this time period. We modified the curriculum and added engaging curricular components to help support students before the anticipated Winter and Pesach breaks. For example, in the month of March, the third grade class did a special biography project which added excitement and gave students the opportunity to make more choices in their learning.
Data Point Five – Nature of disturbance
The fifth data point, the nature of the referral, also gave us direction. By analyzing the nature of the referrals, we were able to provide targeted professional development for faculty to address the most frequent challenges. Over the year, the administration shared strategies focused on the challenging behaviors. We also provided targeted professional development around relationship building and how to incorporate social-emotional learning in the classroom. This focus helped students mediate and resolve conflicts more effectively which lessened behavioral issues in the classroom and the playground.
Results
All in all, by using data, along with key observations and teacher feedback, the school reduced office referrals by 60% from one school year to the next. In the 2017-2018 school year, there were 527 separate office referrals, an average of 3.19 per day. That number dropped dramatically in the 2018-2019 school year to 207, an average of 1.27 students per day. We are now able to focus on other aspects of school improvement. We continue to review office referral data to maintain and further expand our classroom behavior initiatives.
Limitations
While the decrease in referrals and our newfound ability to focus on instruction is certainly worthy of attention, it is important to keep the limitation of this data in mind. NCLB taught us that hard data and statistics alone do not provide actionable information for effective change. There must always be thoughtful data analysis. Only through proper and thorough data analysis, can decisions be made that will produce positive results.
Alongside data points, one must consider school context and culture. This information does not come from spreadsheets but from engaging with and being aware of all stakeholders. For example, a child is referred to the office for multiple outbursts by a single teacher. It is important to determine what is the trigger for that behavior which will not be directly expressed in the data at all. An administrator may find after conversation with parents that a home life issues, such as a death in the family, is the root cause of the problem. Or the administrator may discover that there were five absent teachers, and an uptick of 13 office referrals in one day. The data is the beginning of the conversation and serves as an essential piece of the complex puzzle that is school discipline.
Conclusion
Data can deliver great results within our schools. While data collection and usage must be done cautiously, it should be used to improve our schools both on micro level and across all day schools in a broader sense as well. Yeshiva Day schools have yet to embrace the power of data to effect change. In our school, we have found high impact opportunities to use data and have tasted how it can improve our schools.
Rabbi Dr. Yerachmiel Garfield (ygarfield@ytehouston.org) and Mrs. Melissa Truelove (mtruelove@ytehouston.org) serve on the administration of the rapidly growing Yeshiva Torat Emet in Houston, Texas. They welcome your thoughts and would be happy to share their Google form with other educators.