Script: Effective Observation Training
Note: This extended script includes all of the audio from the video plus any content that is only conveyed visually or is not fully-explained in the narration.
Agenda: Effective Observation Training
The third New Jersey Principal Evaluation for Professional Learning (NJPEPL) observation instrument video will explore effective observation training. This will be followed by scoring guidance and guidance for using the New Jersey Principal Evaluation for Professional Learning (NJPEPL) Observation Instrument for administrators other than principals, assistant principals and vice principals.
The Importance of Accurate and Consistent Measurement
When completing any type of measurement, including performance evaluations, it is important that everyone using the measuring tool is familiar with it and how to use it properly. It is also important that each individual user is using the tool in a consistent way that yields the same results for the same evidence and that these measurements are accurate.
For example, if several students are asked to record the temperature of a beaker of water, they must understand a few important things.
- Is the reading to be taken in Celsius or Fahrenheit?
- How many degrees do the hash marks on the scale represent?
- In what part of the water should the thermometer be placed?
- How long should the thermometer be submerged in the water?
- At what angle should the thermometer be held when taking a reading?
- And which end of the thermometer goes in the water?
Failure to understand any of these fundamentals can lead to inaccuracy in measurement and some students drawing different conclusions from the same evidence.The observation and evaluation of educators is substantially more complex that finding the temperature of a beaker of water; agreeing to a set of fundamental principles prior embarking on this important process is critical.
Effective Observation Training
Effective observation training ensures that superintendents and principals agree on what evidence constitutes what level of performance according to the definitions provided by the observation instrument.
There are 3 main points to consider in effective observation training:
- Understand how the observation instrument describes excellent school leadership in each domain of practice,
- Understand the distinction between each performance level in each domain, and
- Identify the type and quality of evidence that signifies each performance level in each domain.
We will be addressing each of these points in a series of activities for domains one and three in the next section of the video.
Activity: Understand the Domain
Domain 1. Mission, Vision, and Core Values
Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student. Firstly, how does the NJPEPL observation instrument describe excellent school leadership in each domain of practice? We will use an activity for Domain 1: Mission, vision and core values to provide a template for understanding each of the domains.
The descriptor for Domain 1 reads effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.
- Talk: In small groups, discuss with others how this practice is important for excellent leadership (5 minutes).
- Write: Write a brief summary of your thoughts about the necessity of this practice for leading a great school (2 minutes).
- Share: Share 1 to 3 phrases that summarize your thoughts on the importance of this practice with your colleagues (3 minutes).
- Agree: Finally, generate a summary statement that captures the main agreements related to the necessity of this practice. This summary will help establish a shared understanding for this domain for all administrators using the instrument (3 minutes).
Please pause the video while you complete this activity and adjust the suggested time allotted as needed.
Activity: Distinguish Between Performance Levels
Domain 1. Mission, Vision, and Core Values
Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.
Now we have established a shared understanding of the domain language and the importance of the standard it relates to, we will explore performance levels.
- First, on your own, review the observation instrument language for Domain 1 (2 minutes).
- Highlight or circle the key phrases that distinguish one performance level from the next, annotating as needed (5 minutes).
- Then in your group, discuss and agree on the key similarities and differences between each level of practice (3 minutes).
Pause the video while you complete this activity.
Activity: Type and Quality of Evidence
Critical Attributes
Domain 1. Mission, Vision, and Core Values
Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.
With a shared understanding of the domain language including the distinction between performance levels, we can discuss what type and quality of evidence might signify performance at different levels. Given that the principal is performing at the highest level in domain 1, what types and quality of evidence could you use to justify your assessment of this performance? It may be helpful to review the critical attributes associated with this domain to stimulate your thinking.
Guiding Question
Domain 1. Mission, Vision, and Core Values. Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.
[Text on Screen: Guiding Question. For a principal operating at the highest level in this domain, what types and quality of evidence could you use to justify your assessment of this performance?]
To complete this activity:
- [Think]: Jot down some ideas and note whether the evidence is indirect or direct (3 minutes).
- [Share]: Share your ideas with the group (3 minutes).
- [Identify]: Note areas where the group agrees and where there may be open questions (5 minutes).
Pause the video to complete this activity.
Activity: Identify Performance Level Based on Evidence
Part 1
Gather Evidence
Now you have a better sense of what evidence might look like for the highest level of performance in this domain, the next activity provides an opportunity to apply this understanding.
Part one of this activity involves independently gathering evidence of practice by watching the first 5.5 minutes of the video Shaping a Vision of Academic Success for All Students. You can [also] access this video using the link in the resource section below.
While watching the video, list both direct and indirect evidence, and whether this evidence represents actions or outcomes. Once you have gathered the evidence, assign a score to this according to the instrument language for Domain 1. The score must be between 1 and 4 but may contain one decimal place: 3.5, for example.
You may choose to watch the video more than once to ensure you have fully captured the evidence. Please pause the presentation while you watch the video.
[Text on Screen]
- Watch the first 5.5 minutes of Shaping a Vision of Academic Success for All Students.
- Individually list the evidence:
- Label by type (direct or indirect)
- Label whether it is a principal action or an outcome of an action.
- Review the evidence and assign a score to this principal according to the instrument language for Domain 1. The score must be between 1 and 4 but may contain one decimal place, e.g. 3.5.
Part 2:
Agree on Performance Level
Once you have collected the evidence from the video and determined a provisional score, you can now begin to agree on what level of performance is justified by the type and quality of evidence collected.
First, in your small group:
- Discuss your provisional score and how this was justified by the evidence,
- Identify the relative importance of different pieces of evidence and how this impacted your evaluation, and
- Agree on a final score for the principal based on the evidence available.
In the larger group:
- Share your small group’s final score and provide a brief justification for the score. Include any additional information that would help support your decision on a final score for this domain.
- Note any evidence that might be used for domains other than Domain 1.
Please pause the video while you complete this discussion.
Activity: Understand the Domain
Domain 3. Professional Capacity of School Personnel
Effective educational leaders develop the professional capacity and practice of school personnel to promote each student’s academic success and well-being. We are going to repeat the preceding calibration process for Domain 3 of the NJPEPL Observation Instrument, Professional Capacity of School Personnel. Please modify the process as needed based on your experience with calibrating on Domain 1 in the video.
The descriptor for Domain 3 reads Effective educational leaders develop the professional capacity and practice of school personnel to promote each student’s academic success and well-being.
- [Talk] In small groups, discuss how this practice is important for excellent leadership (5 minutes).
- [Write] Write a brief summary of your thoughts about the necessity of this practice for leading a great school (2 minutes).
- [Share] Share 1 to 3 phrases that summarize your thoughts on the importance of this practice with your colleagues (3 minutes).
- [Agree] Finally, generate a summary statement that captures the main agreements related to the necessity of this practice. This summary establishes a shared understanding for this domain for all administrators using the instrument (3 minutes).
Please pause the video while you complete this activity and adjust the suggested time allotted as needed.
Activity: Distinguish Between Performance Levels
Domain 3. Professional Capacity of School Personnel
Effective educational leaders develop the professional capacity and practice of school personnel to promote each student's academic success and well-being.
As before, to distinguish between performance levels for this domain, take the following steps, adjusting time as needed and pausing the video when you are ready to begin the activity.
- Review the observation instrument language for Domain 3 (2 minutes).
- Highlight or circle the key phrases that distinguish one performance level from the next, annotating as needed (5 minutes).
- In your group, discuss and agree on the key similarities and differences between each level of practice (3 minutes).
Activity: Type and Quality of Evidence
Critical Attributes
Domain 3. Professional Capacity of School Personnel
Effective educational leaders develop the professional capacity and practice of school personnel to promote each student's academic success and well-being.
Now you are ready to develop a set of guidelines for the type and quality of evidence which indicates performance at the highest level in this domain. Here are the critical attributes associated with Domain 3 to assist your thinking.
Guiding Question
Domain 3. Professional Capacity of School Personnel
Effective educational leaders develop the professional capacity and practice of school personnel to promote each student's academic success and well-being.
[Text on Screen] Guiding Question. For a principal operating at the highest level in this domain, what types and quality of evidence could you use to justify your assessment of this performance?]
For this activity:
- [Think:] Jot down some ideas and note whether the evidence is indirect or direct (3 minutes),
- [Share:] Share your ideas with the group (3 minutes), and
- [Identify:] Note areas where the group agrees and where there may be open questions (5 minutes).
Pause the video to complete this activity.
Activity: Identify Performance Level Based on Evidence
Part 1
Gather Evidence
The first 5 minutes of the video Improving Instruction will provide a chance to practice collecting evidence for Domain 3 and rating the principal’s performance accordingly.
Independently, list both direct and indirect evidence, and whether the evidence represents actions or outcomes. Once you have gathered the evidence, rate the performance of the principal using the 1 to 4 scale. You may use one decimal place: 3.5, for example.
[Text on Screen]
- Watch the first 5 minutesof Improving Instruction. (access in resources section below)
- Individually list the evidence:
- Label by type (direct or indirect)
- Label whether it is a principal action or an outcome of an action.
- Review the evidence and assign a score to this principal according to the instrument language for Domain 3. The score must be between 1 and 4 but may contain one decimal place, e.g. 3.5.
You may choose to watch the video more than once to ensure you have fully captured the evidence. Please pause the presentation while you watch the video.
Part 2
Agree on Performance Level
Part 2 of the activity involves coming to an agreement on the performance level shown in the video based on the type and quality of evidence shown.
In your small group, discuss your provisional score and how this was justified by the evidence. Identify the relative importance of different pieces of evidence and how this impacted your evaluation, and agree on a final score for the principal based on the evidence available.
In the larger group, share your small group’s final score and provide a brief justification for the score. Include any additional information that would help support your decision on a final score for this domain. Note any evidence that might be used for domains other than Domain 3.
Please pause the video while you complete this discussion.
[Text on Screen]
In your small group:
- Discuss your provisional score and how this was justified by the evidence,
- Identify the relative importance of different pieces of evidence and how this impacted your evaluation, and
- Agree on a final score for the principal based on the evidence available.
In the larger group:
- Share your small group’s final score and provide a brief justification for the score. Include any additional information that would help support your decision on a final score for this domain.
- Note any evidence that might be used for domains other than domain 3.
Recommendations for Further Training
The training process provided up to this point in the video provides a template that district leadership teams may use to complete initial training on the other five domains of the NJPEPL observation instrument.
Leadership teams should commit to developing a set of evidence collection guidelines and model documents and activities that can be referred to throughout the evaluation cycle
Leadership teams should also re-engage in training and calibration processes several times during each evaluation cycle, particularly during the first observation window, and make adjustments accordingly.
[Text on Screen]
- Complete initial training for each of the remaining five domains for the NJPEPL Observation Instrument
- Develop local evidence collection guidelines and model documents/activities
- Re-engage in training and calibration processes several times during each evaluation cycle and make adjustments accordingly.