Sunday, October 29, 2017

Measuring Impact with the Digital Practice Assessment (DPA)

Note: This post is directly related to my work at the International Center for Leadership in Education

Efficacy has been on my mind a great deal as of late, and as a result, it has been reflected in my writing.  When I think back to the successful digital transformation and implementation of innovative practices at my former school when I was a principal the key driver for us was the ability to show, not just talk about, evidence of improvement.  By combining both quantitative and qualitative measures, we were able to articulate the why, how, and what, as well as the detailed process that went into each respective change effort.  The “secret sauce” in all of this was the strategic use of digital tools to proactively share the details of our efforts and resulting impact.  

Image credit:

During my tenure as a principal, I was always in search of tools and processes to help measure the impact of the changes we were implementing.  Unfortunately, nothing existed.  As I work with schools and districts on a weekly basis, I am often asked how they can determine the impact and effectiveness of the many innovative initiatives they have in place. Practices such as BYOD, 1:1, blended learning, personalized learning, classroom and school redesign, branding, makerspaces, professional learning, etc.  This need served as a call to action of sorts and catalyzed my current work.  As Senior Fellow with the International Center for Leadership in Education (ICLE), I have worked with a fantastic team to develop services and tools to help districts, schools, and organizations across the world transform teaching, learning, and leadership.  One of these tools is the Digital Practice Assessment (DPA). 

The DPA creates the context for our work with leaders and teachers, providing authentic baseline data to support personalized professional learning. It begins by examining the strategies in place at each school or district that support student learning with technology in the areas of rigor, relevance, relationships, engagement, and overall culture. The process then moves to understanding the current leadership practices in place to successfully implement technology and innovative practices, aligned to the 7 Pillars of Digital Leadership & Learning (Student Learning, Learning Spaces & Environment, Professional Growth, Communication, Public Relations, Branding, and Opportunity). 

Through this proven model, our consultants can help schools and districts identify opportunities to begin their transformation or take their digital and innovation goals to the next level, leveraging the knowledge, experience, and practice of ICLE’s thought leadership. The DPA process consists of a combination of a self-reflection questionnaire rubric, on-site observations, and online inventories comprised of data and evidence collection. We then leverage evidence-based rubrics to observe leadership and instructional practices while collecting artifacts to provide evidence of effective digital learning and innovative professional practice. Once collected and analyzed, a detailed summary report outlining areas of success, focus opportunities, and recommended next steps will guide the professional learning partnership with ICLE, supporting the development of a strategic professional learning and implementation plan. 

Below is a summary of the DPA process:

Step 1: The Pillars of Digital Leadership Questionnaire is completed by the district or school. This 18 question rubric asks school leaders to reflect on their perceptions for where their school falls on a continuum from not yet started to well developed. During this reflective process, it is expected that school leadership teams collect and document aligned evidence for each item.  This information is completed and archived in the Professional Learning Portal (PLP), a free digital platform developed by ICLE to support schools in data collection,  reflection and goal setting, to grow and improve. The baseline evidence shared is in the context of digital leadership and learning (including examples of data, lesson plans, unit plans, student work, PLC minutes, rigorous digital performance tasks, walk-through forms, assessments, sample observations/evaluations, portfolios, PD plans, social media accounts, pictures, videos, press releases, media coverage, partnerships, etc). 

Step 2: On-site observations and interviews are conducted by consultants to validate perceptions and evidence collected for the seven Pillars of Digital Leadership Questionnaire, as well as targeted classroom observations of student learning, aligned to rigor, relevance and engagement. Additional data is collected and archived in the PLP during classroom observations. The idea is to engage school leaders in dialogue about their culture, student learning and digital integration, no matter where they are with their digital transformation. 

Step 3: The data and evidence are tightly aligned to ICLE’s research-based rubrics to provide a detailed view of where a district or school is with their digital transformation.The data and artifacts are analyzed, leading to a summary report that details the current state of practice at each school or in the district. 

Step 4: The DPA report is shared and discussed with the school leadership team. In partnership with ICLE, observations about the evidence collected are shared and discussed. During the strategic planning process, discussions focus on areas of strength and improvements to develop a tailored and personalized implementation plan.

Step 5: On-going professional learning is implemented and progress monitoring through the online Pillars of Digital Leadership Questionnaire is documented to determine the efficacy of the digital transformation.

The DPA process has been created to support districts and schools looking for ways to measure and articulate the impact of technology and innovation on practice.  While data is valuable, it moves beyond this as the only metric for success by actually taking a lens to an array of strategies and practices that combine to create a thriving learning culture.  

The DPA doesn’t just look at technology and innovation. It also provides insight on all elements of school culture and student learning.  In addition to being informed by a broad body of research and driven by evidence, the DPA process is also aligned to the following:

We don’t know where we are and how effective change is until steps are taken to look critically at practice. We hope that through the DPA process we can help you develop, refine, measure, and then share amazing examples that illustrate how efficacy has been attained.  

If you are looking for a method of determining where you are and where you want your district or school to be in the digital age, please contact Matt Thouin at ICLE (  He can arrange for an interactive and detailed look at the DPA rubrics and process as well as the PLP platform from the convenience of your home or office.  We look forward to supporting you on your journey toward systemwide digital transformation. If you have any questions for me, please leave them in the comments below.

Copyright © by International Center for Leadership in Education, a division of Houghton Mifflin Harcourt. All rights reserved. 

1 comment:

  1. This seems to be work that will be ever-changing. I like where ICLE's thoughts to analyze implementation are headed. With technology always changing, this will be daunting, but important work. Keep it up.