Assessing the Assessment: How to Know if our Assessments are Effective
January 28, 2020
I Screen, You Screen, We All Screen for…Data?
January 28, 2020
by RABBI TZVI DAUM
A little over ten years ago, Dr. Shawn Zelig Aster penned an article in “The Commentator” (a Yeshiva College student newspaper) titled “The Illiteracy Epidemic — Is there a Scandal of Orthodox Indifference?” The thrust of the article was to ask the question of how was it possible that after twelve years of Yeshiva day school education, many students arrived in Israel for their gap year without being able to translate a posuk of Chumash correctly? The article (which no longer appears online), was linked to on Lookjed and generated a large response from members of the field of Jewish education (see https://bit.ly/34pafn5).
My personal response at the time was to lament the lack of standards and direction in Jewish education. It has been my experience that teachers are generally not given any guidance about which skills students are expected to have mastered, and which need to be reinforced or taught.
I wrote back then:
“The lack of guidance is appalling. The average day school teacher, when hired, is told “teach x perakim of Sefer y”. There is no guidance as how to go about doing this. Which vocabulary words should be stressed? What skills should be developed? Which Rashi’s should be covered? How do the skills in fourth grade build on what was taught in third grade? There is hardly any system to speak of. This obviously not only affects teachers but more importantly it affects their students who don’t have any real methodology to work with and have no idea of what is expected of them.”
I continued with a call to create a Facebook group that would focus on creating standards for Jewish education, but that never went too far.
I should point out that this lack of standards is not the only reason why students might not have mastered being able to read a posuk of Chumash, but I think it is a major one. Other factors include a lack of accountability for both students and teachers throughout the years and perhaps the amount of time schools spend teaching Chumash.
The lack of accountability for students and teachers, although not my initial motivation, is related to the lack of standards. Over twenty years ago, Paul Black and Dylan William wrote an article entitled “Inside the Black Box: Raising Standards Through Classroom Assessment” (https://bit.ly/2raqFS7), where they argued that assessment was needed to track student learning. They describe what occurs in the classroom as the “Black Box” of education, meaning, principals and teachers often know that some kind of learning is taking place in the classroom, but nobody is quite sure of what it is.
The development of standards for Jewish education would guide teachers, help scaffold the teaching so that fourth grade builds upon third, and create a framework for accountability. This is where the need for assessments comes in – to measure outcomes against these standards and provide the data that schools need to improve their curriculum and instruction.
These ideas are standard in secular education. We know that curriculum development starts with the creation of standards and continues with the development of both formative and cumulative assessments, where the results need to be analyzed and used to further improve the curriculum and the educational experience of students. Today, we all have a greater understanding of the importance of data. Businesses, sports and everything else in between is driven by data that objectively measures and informs decision making. Yet, this was sorely lacking in the field of Jewish education.
In the summer of 2015, at the annual summer “Think Tank” conference conducted by CoJDS, a request was made for CoJDS to get involved in providing schools with standardized assessments and the ability to make data-driven decisions. To answer that call, we initiated an effort to develop a standardized assessment for fifth grade students.
We began by reaching out to seventeen day schools across the country and sent them five detailed surveys regarding which subjects and educational standards they felt were imperative for students to know upon leaving fifth grade. Through this process, we collected hundreds of data points pertaining to the subjects of Chumash Skills, Chumash Knowledge, and Jewish Laws and Practice (Dinim and Yediot Klaliot). To the surprise of many, there was actually overhwelming agreement among the various day schools regarding which skills and content students should master when leaving grade five; the average agreement rate per standard was over 94%.
From these standards we developed JSAT Level 5, a standardized assessment for students leaving grade five, to help schools develop objective data about student achievement and to use this data to improve educational outcomes. To date, over four thousand students have taken the JSAT Level 5 exam and educators have used the rich data we provide to analyze and improve their educational programs.
This fall we have launched JSAT Level 8 which replaces the BJE High School Placement Exam. The standards for this exam were developed in partnership with over fifty day schools across North America. JSAT Level 8 will be taken by over two thousand students in more than fifty schools this school year.
We are hopeful that the standards for the JSAT Level 5 and Level 8 can be used by schools as they develop curriculum and send their teachers to the classroom. We hope the data gleaned from this exam will help crack open the “black box” and allow Jewish day school leaders to understand what is being taught in the classroom. In addition, we hope to wipe out “The Illiteracy Epidemic” and ultimately help students come to Israel after twelve years of Jewish education being able to fully translate and comprehend a posuk of Chumash on their own.
Rabbi Tzvi Daum is the Director of Curricular Initiatives at CoJDS and the developer of the JSAT exam. Rabbi Daum has spent over sixteen years as a classroom teacher at the Jewish Foundation School in Staten Island, NY. He holds a Master’s Degree in Education with a specialty in Educational Technology from California State University at Bakersfield.