The Journey to Data-Driven Instruction

by RABBI SHMUEL CHAIT

In every professional realm, businesses are increasingly moving toward real-time data to help their effectiveness. Due to its dramatic results, colleges are offering degrees on using data effectively. The world of education is also seeing the vast benefits of using real time, ongoing and precise information. For example, many have noticed that the end of the year state test is not the most valuable information for determining future instruction. Among other issues, they find that the scores return too late to determine future placement in later grades.

Many schools are carefully crafting methods of collecting exact student data, administering assessments to the students strategically, and giving teachers timely, useful information that influences future instruction. By using data-driven instruction (DDI) and  having a strong understanding of what each student has learned, these schools are aiding teachers in addressing the strengths and educational gaps in their students, and determining adjustments to professional development and curriculum. DDI is changing the paradigm from knowing “what was taught” to focusing on, “what did the students learn?” While the transferability and application to Torah based schools is still being determined, the possible benefits of such practices are overwhelming. 

Each school has their unique journey moving towards DDI. I will describe the journey of one such school, Yeshiva Toras Chaim Toras Emes (YTCTE), ,which encompasses over 1,000 students, aged 3-21, on its two campuses. YTCTE identified the learning of Chumash and kriah as the primary goals of our lower elementary kodesh curriculum. We then did some deep introspection to analyze if each student was reaching the pre-established, grade-level benchmarks.

Why DDI?

During this self-analysis, we discovered that despite the teachers knowing the scope and skills that each student was expected to learn, it was not clear what skills were taught to the mastery level and what was simply introduced. We also discovered that even though rebbeim and moros were teaching to the school standards there was no system  in place to see if the students were absorbing the skills. Obviously, each teacher had their methods to assess student knowledge. However, since these assessments were developed and deployed by each teacher individually, there was no method to check student progress uniformly.

One way this manifested was in the referrals to the resource room. A disproportionate number of students were sent for support in general studies as opposed to limudei kodesh. To many, this was troublesome. A student’s deficiency in some area of cognition or ability to succeed in such area is usually just as relevant to limudei kodesh as it is to the general studies curriculum. Was this discrepancy due to Chumash teachers having better teaching skills or resources at their disposal? Perhaps it was because they didn’t know exact grade-level expectations and had ineffective ways of testing for these skills. Moreover, while parents might resist their children being pulled from class for support, if we were able to show non-judgmental, standards-based data to parents concerning their child’s struggles, they would be more accepting of the advised intervention. 

Another problem  YTCTE had was that recent rapid growth of the school brought an influx of new limudei kodesh staff, and it became challenging for the administration to determine if parallel classes were all on the same level. The administration knew what each class covered in Chumash and the report card grades for each student in the class. Yet, they were unable to pinpoint the level of progress for each student objectively. One cannot manage what one cannot measure. As an educational institution, we felt it was our responsibility is to ensure student learning, and as such, it was incumbent on us to find the means to measure that.

“Data-driven decision-making is about gathering data to understand if a school is meeting its purpose and vision,” says Victoria Bernhardt, author of Data Analysis for Comprehensive Schoolwide Improvement. “If we do not have a target, we could make decisions that essentially lead to ‘random acts of improvement.’” Noted educator Rabbi Hersh Fried, in an interview printed in the fall 2015 issue of Jewish Action, said, “One of the problems in our communities is that we have no statistics. This allows anybody with an agenda to randomly declare ‘the biggest problem plaguing our community.’  They also claim to know the cause of that problem and to demand that everyone support their solution. We would be much further along as a community if we had some data on the goings-on in our communities and schools.”

YTCTE’s Journey 

In the book, Driven by Data 2.0, Paul Bambrick-Santoyo outlines a process of using data effectively by identifying four steps: assessment, analysis, action, and culture.  We are using this four-step process to implement DDI in the identified priority areas of Chumash and kriah.

Assessment: By creating thorough and easy-to-administer evaluations, we can find meaningful data. 

Chumash: We are in the initial stages of a schoolwide computerized Chumash assessment. We created grade-level tests with teacher input, assessing the critical skills required for that age bracket. Rigorous, multiple-choice answers were written and uploaded onto a digital platform. Every student at YTCTE took this 30-minute exam. 

Kriah: YTCTE has been tracking data on students’ correct syllables per minute (SPM) and accuracy. Initially, we used a thorough but labor-intensive 12-minute exam. In year two we amended this process by having a team of kriah experts administer a two-minute exam that tests a student’s basic kriah competency. 

Analysis: By breaking down the results, one can begin to determine the causes and remedies.

Chumash: Results from the computerized assessment automatically populated a spreadsheet, giving us a centralized location where staff can analyze the results. Important data included percentages of students who answered a given question incorrectly. We also looked for schoolwide trends in the data. For example, we noticed that the volume of information we expected students to know by certain grade levels was beyond the ability of a large segment of students. The data further showed that weaknesses in the prefix/suffix skills negatively impacted the students’ knowledge of the shoroshim. Team-wide discussions are now taking place about how to better enforce these critical skills.

Kriah: The results of the SPM (syllables read per minute) testing are entered in a spreadsheet, divided by grade and teacher. An analysis is done to find class-wide challenges and individual problems. Follow-up meetings are held to plan for individual intervention when necessary.

Action: After data is gleaned and sifted through, a plan of enhanced learning is developed and applied. 

Chumash: We analyzed the most often recurring words that would best promote future learning. These observations led to team-wide discussions on how to better reinforce these critical skills. For individual students, it has made an impact on the criteria necessary for other interventions and class placement. 

Kriah: Students with lagging skills are targeted for remediation, and teachers are coached on how to target class-wide challenges. The data also showed that the younger grades are stronger readers than the older ones, confirming our investment in extra kriah support two years ago. Since we don’t have comprehensive data about older students’ abilities from prior years, conversations are ongoing to figure out why the older students are not as developed and what can be done to help them.

Culture: Creating and maintaining an environment where DDI can thrive.  

Likely, our greatest accomplishment is the staff embracing DDI. Every limudei kodesh teacher is now looking for quantifiable measures to chart their students’ progress and struggles. Staff conversations on understanding student learning and improving outcomes for all students are becoming part of the culture. Even greater teacher concern for students’ knowledge gaps is apparent.   

Our Next Steps of Implementation:

Chumash: In the future, we are planning a test to be administered once per trimester to determine student progress in real-time. These periodic, formative assessments will provide increased clarity to kodesh teachers as to what the class-wide challenges are and what individual students have not yet mastered. We feel that this will help students learn and retain what they have gained. A study published in the Journal of Experimental Psychology: Applied (2017) found that

 “frequent classroom quizzing with feedback improves student learning and retention, and multiple-choice quizzing is as effective as short-answer quizzing for this purpose.”

We hope that our frequent testing will allow our students to realize this benefit.

We also are looking to modify the digital platform, especially for the younger grades. Due to our use of a digital platform, students enjoy taking the exam, but in the future, we are considering including game-based testing to lower their anxiety even more.   

Kriah: Many advise that even after a student has learned to read, they should be periodically tested to ensure that they have not regressed in the critical measures of kriah. As such, all students will be reassessed before the end of the school year. Ideally, we would like to have data-driven conversation meetings within 48 hours of the student’s exam. 

The possibilities are endless: 

In conversations with data experts from the business world, we have learned that the value of data increases exponentially as the volume of information increases. So too, the use and value of DDI will dramatically increase as one amasses more data over a more extended period. This long-term view will enable us to examine the effectiveness of different programs and establish more explicit expectations. As DDI becomes standard in our Chumash and kriah efforts, we hope to apply it to other essential parts of the curriculum. We hope that this data can also provide direction to our professional development activities. 

Every Step Counts: 

As you take your journey to DDI you may realize that it’s a longer road then you expected. Keep in mind the words of Shlomo in Mishlei, (ד: יט)

 דֶּ֣רֶךְ רְ֖שָׁעִים כָּֽאֲפֵלָ֑ה לֹ֥א יָ֜דְע֗וּ בַּמֶּ֥ה יִכָּשֵֽׁלוּ “The way of the wicked is like complete darkness; they do not know on what they stumble.”

It’s worthwhile to note that the Shlomo does not say, “The way of the righteous is perfect,” or “The way of the wicked is corrupt.” It seems the downfall of the wicked is that they are in the dark. Keeping track of what they’ve done may not give the righteous all the answers, but at least it gives them the information they need to identify what to get better at. Even as we progress further down the road, with more complete student data, we will still make mistakes, but at least we will be well informed and doing all we can to help our students. 

Rabbi Shmuel Chait, MAEL, is a sixth grade rebbi and educational specialist in Yeshiva Toras Chaim Toras Emes in Miami. He is currently enrolled in the CoJDS PTI program. He received smicha from Yeshiva Chofetz Chaim and is a graduate of Bellevue University. He can be reached at RabbiChait@gmail.com.

• Print
• Email