New Jersey Department of Education

NJPEPL Observation Instrument Training Modules

NJPEPL Observation Instrument Module 3

Effective Observation Training

Agenda - Effective Observation Training

The third NJPEPL observation instrument video will explore Effective Observation Training.
[This will be followed by scoring guidance and guidance for using the NJPEPL Observation Instrument for administrators other than principals, assistant principals and vice principals.]

The Importance of Accurate and Consistent Measurement

When completing any type of measurement, including performance evaluations, it is important that everyone using the measuring tool is familiar with it and how to use it properly.  It is also important that each individual user is using the tool in a consistent way that yields the same results for the same evidence and that these measurements are accurate.
For example, if several students are asked to record the temperature of a beaker of water, they must understand a few important things.

• Is the reading to be taken in Celsius or Fahrenheit? 
• How many degrees do the hash marks on the scale represent?
• In what part of the water should the thermometer be placed? 
• How long should the thermometer be submerged in the water? 
• At what angle should the thermometer be held when taking a reading? 
• And which end of the thermometer goes in the water?

Failure to understand any of these fundamentals can lead to inaccuracy in measurement and some students drawing different conclusions from the same evidence.
The observation and evaluation of educators is substantially more complex that finding the temperature of a beaker of water; agreeing to a set of fundamental principles prior embarking on this important process is critical.

Effective Observation Training

Effective observation training ensures that superintendents and principals agree on what evidence constitutes what level of performance according to the definitions provided by the observation instrument.
There are 3 main points to consider in effective observation training:

• Understand how the observation instrument describes excellent school leadership in each domain of practice,
• Understand the distinction between each performance level in each domain, and
• Identify the type and quality of evidence that signifies each performance level in each domain. 

We will be addressing each of these points in a series of activities for domains one and three in the next section of the video.

Activity: Understand the Domain

Domain 1. Mission, Vision, and Core Values.

[Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.]
Firstly, how does the NJPEPL observation instrument describe excellent school leadership in each domain of practice?  We will use an activity for Domain 1: Mission, vision and core values to provide a template for understanding each of the domains.

The descriptor for Domain 1 reads “effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.”

• [Talk] In small groups, discuss with others how this practice is important for excellent leadership (5 minutes).
• [Write] Write a brief summary of your thoughts about the necessity of this practice for leading a great school (2 minutes).
• [Share] Share 1 to 3 phrases that summarize your thoughts on the importance of this practice with your colleagues (3 minutes).
• [Agree] Finally, generate a summary statement that captures the main agreements related to the necessity of this practice.  This summary will help establish a shared understanding for this domain for all administrators using the instrument (3 minutes).

Please pause the video while you complete this activity and adjust the suggested time allotted as needed.

Activity: Distinguish Between Performance Levels

Domain 1. Mission, Vision, and Core Values. Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.

4	3	2	1
The principal consistently plans, implements and supports actions that develop, advocate and enact a shared mission, vision, and set of core values of high quality education for each student’s success. Shared ownership by staff and students leads to significant improvements in or sustained excellence of the educational environment.	The principal plans, implements and supports actions that develop, advocate and enact a shared mission, vision, and set of core values of high quality education for each student’s success resulting in a positive impact on the educational environment.	The principal is inconsistent in planning, implementing and supporting actions associated with mission, vision and core values or these actions have limited or no positive impact on the educational environment.	The principal’s actions or attitude regarding the development and enactment of mission, vision, and core values have a negative impact on the educational environment.

Now we have established a shared understanding of the domain language and the importance of the standard it relates to, we will explore performance levels.

• First, on your own, review the observation instrument language for Domain 1 (2 minutes).
• Highlight or circle the key phrases that distinguish one performance level from the next, annotating as needed (5 minutes).
• Then in your group, discuss and agree on the key similarities and differences between each level of practice (3 minutes).

Pause the video while you complete this activity.

Activity: Type and Quality of Evidence

Critical Attributes

Domain 1. Mission, Vision, and Core Values. Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.

4

The principal consistently plans, implements and supports actions that develop, advocate and enact a shared mission, vision, and set of core values of high quality education for each student’s success. Shared ownership by staff and students leads to significant improvements in or sustained excellence of the educational environment.

Critical Attributes

Collaborate to develop an educational mission for the school to promote the academic success and well-being of each student. Using relevant data, develop and promote a vision for the school on the successful learning and development of each child. Articulate, advocate, and cultivate core values that define the school’s culture and stress the imperative of child-centered education; high expectations and student support; equity, inclusiveness, and social justice; openness, caring, and trust; and continuous improvement. Strategically develop, implement, and evaluate actions to achieve the vision for the school. Review the school’s mission and vision and adjust them to changing expectations and opportunities for the school, and changing needs and situations of students.

With a shared understanding of the domain language including the distinction between performance levels, we can discuss what type and quality of evidence might signify performance at different levels.
Given that the principal is performing at the highest level in domain 1, what types and quality of evidence could you use to justify your assessment of this performance?” It may be helpful to review the critical attributes associated with this domain to stimulate your thinking.

Guiding Question

Domain 1. Mission, Vision, and Core Values. Effective educational leaders develop, advocate, and enact a shared mission, vision, and set of core values of high-quality education and academic success for, and well-being of, each student.

4

The principal consistently plans, implements and supports actions that develop, advocate and enact a shared mission, vision, and set of core values of high quality education for each student’s success. Shared ownership by staff and students leads to significant improvements in or sustained excellence of the educational environment.

[Text on Screen: Guiding Question. For a principal operating at the highest level in this domain, what types and quality of evidence could you use to justify your assessment of this performance?]
To complete this activity:

• [Think]: Jot down some ideas and note whether the evidence is indirect or direct (3 minutes).
• [Share]: Share your ideas with the group (3 minutes).
• [Identify]: Note areas where the group agrees and where there may be open questions (5 minutes).

Pause the video to complete this activity.

Activity: Identify Performance Level Based on Evidence

Part 1: Gather Evidence

Now you have a better sense of what evidence might look like for the highest level of performance in this domain, the next activity provides an opportunity to apply this understanding.

Part one of this activity involves independently gathering evidence of practice by watching the first 5.5 minutes of the video “Shaping a Vision of Academic Success for All Students.” You can [also] access this video using the link in the resource section below.

While watching the video, list both direct and indirect evidence, and whether this evidence represents actions or outcomes. Once you have gathered the evidence, assign a score to this according to the instrument language for Domain 1. The score must be between 1 and 4 but may contain one decimal place: 3.5, for example.

You may choose to watch the video more than once to ensure you have fully captured the evidence. Please pause the presentation while you watch the video.

[Text on Screen]

• Watch the first 5.5 minutes of “Shaping a Vision of Academic Success for All Students.”
• Individually list the evidence:
• Label by type (direct or indirect)
• Label whether it is a principal action or an outcome of an action.
• Review the evidence and assign a score to this principal according to the instrument language for Domain 1. The score must be between 1 and 4 but may contain one decimal place, e.g. 3.5.

Part 2: Agree on Performance Level

Once you have collected the evidence from the video and determined a provisional score, you can now begin to agree on what level of performance is justified by the type and quality of evidence collected.
First, in your small group:

• Discuss your provisional score and how this was justified by the evidence,
• Identify the relative importance of different pieces of evidence and how this impacted your evaluation, and
• Agree on a final score for the principal based on the evidence available.

In the larger group:

• Share your small group’s final score and provide a brief justification for the score. Include any additional information that would help support your decision on a final score for this domain.
• Note any evidence that might be used for domains other than Domain 1.

Please pause the video while you complete this discussion.

Activity: Understand the Domain

Domain 3. Professional Capacity of School Personnel.

Effective educational leaders develop the professional capacity and practice of school personnel to promote each student’s academic success and well-being.
We are going to repeat the preceding calibration process for Domain 3 of the NJPEPL Observation Instrument, Professional Capacity of School Personnel. Please modify the process as needed based on your experience with calibrating on Domain 1 in the video.
The descriptor for Domain 3 reads “Effective educational leaders develop the professional capacity and practice of school personnel to promote each student’s academic success and well-being.”

• [Talk] In small groups, discuss how this practice is important for excellent leadership (5 minutes).
• [Write] Write a brief summary of your thoughts about the necessity of this practice for leading a great school (2 minutes).
• [Share] Share 1 to 3 phrases that summarize your thoughts on the importance of this practice with your colleagues (3 minutes).
• [Agree] Finally, generate a summary statement that captures the main agreements related to the necessity of this practice.  This summary establishes a shared understanding for this domain for all administrators using the instrument (3 minutes).

Please pause the video while you complete this activity and adjust the suggested time allotted as needed. 

Activity: Distinguish Between Performance Levels

Domain 3. Professional Capacity of School Personnel. Effective educational leaders develop the professional capacity and practice of school personnel to promote each student's academic success and well-being.

4	3	2	1
The principal consistently plans, implements and supports actions to develop the professional capacity and practice of school personnel to promote each student’s success. Shared ownership by staff leads to significant improvements in or sustained high levels of staff performance.	The principal plans, implements and supports actions that develop the professional capacity and practice of school personnel to promote the success of each student resulting in a positive impact on staff performance.	The principal is inconsistent in planning, implementing and supporting actions that develop the professional capacity and practice of school personnel or these actions have limited or no positive impact on staff performance.	The principal’s actions or attitudes regarding the development of the professional capacity and practice of school personnel have a negative impact on staff performance.

As before, to distinguish between performance levels for this domain, take the following steps, adjusting time as needed and pausing the video when you are ready to begin the activity.

• Review the observation instrument language for Domain 3 (2 minutes).
• Highlight or circle the key phrases that distinguish one performance level from the next, annotating as needed (5 minutes).
• In your group, discuss and agree on the key similarities and differences between each level of practice (3 minutes).

Activity: Type and Quality of Evidence

Critical Attributes

Domain 3. Professional Capacity of School Personnel. Effective educational leaders develop the professional capacity and practice of school personnel to promote each student's academic success and well-being.

4

The principal consistently plans, implements and supports actions to develop the professional capacity and practice of school personnel to promote each student’s success. Shared ownership by staff leads to significant improvements in or sustained high levels of staff performance.

Critical Attributes

Plan for and manage staff turnover and succession, providing opportunities for effective induction and mentoring of new personnel. Develop teachers’ and staff members’ professional knowledge, skills, and practice through differentiated opportunities for learning and growth. Deliver actionable feedback about instruction and other professional practice through valid, research-anchored systems of supervision and evaluation to support the development of teachers’ and staff members’ knowledge, skills and practice. Develop the capacity, opportunities, and support for teacher leadership and leadership from other members of the school community. Develop workplace conditions for teachers and other professional staff that promote effective professional collaboration, development, practice, and student learning. Tend to their own learning and effectiveness through reflection, study, and improvement, maintaining a healthy work-life balance

Now you are ready to develop a set of guidelines for the type and quality of evidence which indicates performance at the highest level in this domain.
Here are the critical attributes associated with Domain 3 to assist your thinking.

Guiding Question

Domain 3. Professional Capacity of School Personnel. Effective educational leaders develop the professional capacity and practice of school personnel to promote each student's academic success and well-being.

4

The principal consistently plans, implements and supports actions to develop the professional capacity and practice of school personnel to promote each student’s success. Shared ownership by staff leads to significant improvements in or sustained high levels of staff performance.

[Text on Screen] Guiding Question. For a principal operating at the highest level in this domain, what types and quality of evidence could you use to justify your assessment of this performance?]
For this activity:

• [Think:] Jot down some ideas and note whether the evidence is indirect or direct (3 minutes),
• [Share:] Share your ideas with the group (3 minutes), and
• [Identify:] Note areas where the group agrees and where there may be open questions (5 minutes).

Pause the video to complete this activity.

Activity: Identify Performance Level Based on Evidence

Part 1: Gather Evidence

The first 5 minutes of the video “Improving Instruction,” will provide a chance to practice collecting evidence for Domain 3 and rating the principal’s performance accordingly.
Independently, list both direct and indirect evidence, and whether the evidence represents actions or outcomes. Once you have gathered the evidence, rate the performance of the principal using the 1 to 4 scale.  You may use one decimal place: 3.5, for example.

[Text on Screen]

• Watch the first 5 minutes “Improving Instruction.” (access in resources section below)
• Individually list the evidence:
• Label by type (direct or indirect)
• Label whether it is a principal action or an outcome of an action.
• Review the evidence and assign a score to this principal according to the instrument language for Domain 3. The score must be between 1 and 4 but may contain one decimal place, e.g. 3.5.

You may choose to watch the video more than once to ensure you have fully captured the evidence. Please pause the presentation while you watch the video.

Part 2: Agree on Performance Level

Part 2 of the activity involves coming to an agreement on the performance level shown in the video based on the type and quality of evidence shown.

In your small group, discuss your provisional score and how this was justified by the evidence. Identify the relative importance of different pieces of evidence and how this impacted your evaluation, and agree on a final score for the principal based on the evidence available.

In the larger group, share your small group’s final score and provide a brief justification for the score.  Include any additional information that would help support your decision on a final score for this domain. Note any evidence that might be used for domains other than Domain 3.

Please pause the video while you complete this discussion.

[Text on Screen]

In your small group:

1. Discuss your provisional score and how this was justified by the evidence,
2. Identify the relative importance of different pieces of evidence and how this impacted your evaluation, and
3. Agree on a final score for the principal based on the evidence available.

In the larger group:

1. Share your small group’s final score and provide a brief justification for the score.  Include any additional information that would help support your decision on a final score for this domain.
2. Note any evidence that might be used for domains other than domain 3.

Recommendations for Further Training

The training process provided up to this point in the video provides a template that district leadership teams may use to complete initial training on the other five domains of the NJPEPL observation instrument.
Leadership teams should commit to developing a set of evidence collection guidelines and model documents and activities that can be referred to throughout the evaluation cycle

Leadership teams should also re-engage in training and calibration processes several times during each evaluation cycle, particularly during the first observation window, and make adjustments accordingly.

[Text on Screen]

• Complete initial training for each of the remaining five domains for the NJPEPL Observation Instrument
• Develop local evidence collection guidelines and model documents/activities
• Re-engage in training and calibration processes several times during each evaluation cycle and make adjustments accordingly.