Results

To assess the effectiveness of our programs, the NYC Leadership Academy analyzes quantitative and qualitative data, focusing on evidence of student learning participant satisfaction and enrollment rates in our coaching programs. The following are a sampling of key evaluation results to date:

Impact on Student Learning Outcomes

The NYC Department of Education (NYC DOE) Office of Accountability compared elementary and middle schools led by graduates of the Aspiring Principals Program (APP) with those led by similarly tenured peers (those with three years of consecutive service in the same school), statistically controlling for predicted student performance.

These analyses found that:

• Elementary and middle schools [1]* led by APP graduates for three consecutive years [2]** achieved almost twice the gains in students’ reading proficiency ratings (6.4% vs. 3.7% gains) than comparison schools.  This finding is statistically significant (R2=.28, F (1, 106)=4.208, p<.05), meaning that there is less than a 5% probability that this difference is due to chance alone; the 0.5% difference in Math gains reported below is not statistically significant, meaning that this difference is likely due to chance alone.
• 

The NYC DOE Office of Accountability also ran analyses comparing high schools [3]*** led by APP graduates with similarly tenured peers, controlling for school- and student-level variables through sampling.

These analyses found that:

• High schools led by APP graduates for three consecutive years posted 9.6% higher gains in the percent of students earning 10+ credits over time than comparison schools.  This finding is statistically significant (R2=.19, t(22) = 2.289, p < .05); there is less than a 5% probability that this difference occurred due to chance alone.

• High schools led by APP graduates for three consecutive years posted 6% higher weighted Math Regents pass rates and 12% higher weighted Science Regents pass rates within the schools’ subgroup of students with the lowest scores on the corresponding Grade 8 subject tests than comparison schools.   These findings are not statistically significant, meaning that the difference observed is likely due to chance alone.
• High schools led by APP graduates narrowed the achievement gap by moving 11.2% more students with an Individualized Education Program (IEP), 11.4% more students classified as Limited English Proficient (LEP) and 8.4% more African-American students towards graduation than comparison schools, as measured by the percent of students earning 10+ credits in 2006/2007.   Each of these findings is statistically significant at p<.001, meaning that there is less than a 1% probability that this difference occurred due to chance alone. 

Evidence of Effect on the NYC Public School System:

• Aspiring Principals Program (APP) graduates lead over 108,000 NYC public school students.
• The Aspiring Principals Program (APP) has placed 90% of its graduates in a variety of leadership roles within the NYC DOE. with 69% serving as principals. 
• APP graduates post a higher principal retention rate for third-year principals as compared to NYC DOE principals system-wide.
• 20% of APP graduates have opened new, small schools.
• 69% of APP graduates lead schools eligible for Title I funding.

APP maintains a high degree of excellence, as evidenced by:

• Implementing a highly rigorous admissions process that results in a selectivity rate of less than 23% each year.

Program satisfaction rates are high, as evidenced by end-of-year survey responses. For example:

• In 2007-2008 participant survey results, respondents rated the impact coaching***** [4] had on their effectiveness as a school leader high with a mean score of 3.7.

*Elementary and middle schools were matched on principal tenure (all principals with similar tenure were included in the analyses) and the analyses statistically controlled for student performance by using a predicted student performance measure created by the NYC DOE, Office of Accountability (n=107).

**Researchers suggest it takes three years to turn around an underperforming elementary school, six years to turn around a high school and eight years to turn around an entire school district (Fullen, Michael. Afterword. Evaluation Inquiry: Using Evaluation to Promote Student Success. By Beverly Parsons. Thousand Oaks: Corwin Press, 2001. 141-142). Based on this understanding of when to measure leadership-related gains, only APP graduates who had served as principal in the same school for three consecutive years are included in our analyses. However, to be precise, the student test results used for these analyses were collected after the graduate had been in the principalship for only 2½ years.

***High schools were matched on a 1:1 basis based on peer index, principal tenure and grade/no grade (on the progress report) status (n=24).

****Ninety-four percent of all eligible principals placed in existing schools who were coached by Leadership Academy coaches in 2006/2007 returned for coaching support in the 2007/2008 school year and ninety-one percent of eligible principals who opened new, small schools and chose to work with a Leadership Academy coach in 2006/2007 returned for coaching support for the 2007/2008 school year.

*****In the January 2008 survey, principals of established schools rated the impact coaching has had on their effectiveness as a school leader high with a mean score of 3.7 (on a 4-point scale) and principals of new, small schools rated the impact coaching has had on their effectiveness as a school leader with a mean score of 3.6 (on a 4-point scale).