Return on Instruction (ROI)

"When integrating technology there needs to be a Return on Instruction (ROI) that results in evidence of improved student learning outcomes." - Eric Sheninger

For educational technology to be fully embraced as a powerful teaching and learning tool there must be a focus on substance over assumptions and generalizations.  There is a great deal of evidence to make educators reflect upon their use of technology. The most glaring was the OECD Report that came out last fall. Here is an excerpt:

"Schools have yet to take advantage of the potential of technology in the classroom to tackle the digital divide and give every student the skills they need in today’s connected world, according to the first OECD PISA assessment of digital skills. Even countries which have invested heavily in information and communication technologies (ICT) for education have seen no noticeable improvement in their performances in PISA results for reading, mathematics, or science."

This week I came across a post by Larry Ferlazzo that asked educators to provide their response to why EdTech has over-promised and under-delivered. Before even reading this post, I already began to develop some of my own answers based on my work and observations of schools all over the world. This response stood out from the second part of Larry’s piece:

"Good teaching is not about where or what to click.  Good teaching is about building quality relationships with students, helping students make connections to the real world, building students individual cognitive networks, and having our students enjoy learning for the sake of learning.  Technology will never solve all the ills of education! Nor should it! So what is the biggest problem in EdTech?  The biggest problem is that we have been teaching teachers and students how to use technology without giving them the why of technology.  We have mistakenly believed that giving teachers and students new software or a new box will help fix education."

Image credit: http://artisanauctions.com/

I agree that part of the problem has been a lack of focus on why technology should be integrated.  As the OECD Report alluded to, the problem isn’t the technology per say, but the lack of quality professional learning to support educators with effective implementation.  There needs to be a greater focus on instructional design, digital pedagogical techniques, and the development of better assessments aligned to higher standards.  I am proud to say that this is the foundation of our digital work at the International Center for Leadership in Education (ICLE). It is important to note that this dilemma is not only specific to technology, but innovation as well.  There has to be a concerted focus on the why, how, and evidence of results. 

In addition to professional learning, we also have to be more critical of what we see and hear when it comes to educational technology.  For technology to be taken seriously as a tool to support and enhance teaching and learning then we must no longer accept assumptions and generalizations as to what it actually does. I for one want students empowered to own their learning, create artifacts, to demonstrate conceptual mastery, use their voice, be responsible in online spaces, and connect with the world in authentic ways.   From an educator perspective I also want teachers and administrators to utilize technology and innovative practices to improve teaching, learning, and leadership.  However, the principal in me also needs to balance this with clear results. This is a reality for every teacher and administrator that cannot be ignored.  It is important to show how students apply what they have learned in relevant ways aligned to the highest levels of knowledge taxonomy. Telling just doesn’t cut it anymore.

The next step is to begin to connect this to results that prove beyond assumptions and generalizations that technology is playing a role to positively impact teaching and learning. It is important to remember that if teaching, learning, and leadership don’t change, technology and innovation will never have the type of impact that is expected. Consider these four areas of evidence:

• Data:  Now let me start off by saying that this is only one indicator of success. The key is to be able to align various data sources to technology use or initiatives. Standardized test scores have the greatest ability to illustrate to stakeholders how technology is positively impacting learning and achievement. Please take a look at this study by the University of Buffalo. It shows how Lockport City School students in a 1:1 iPad environment experienced significant achievement gains.  Read the entire piece as it explains why achievement increased.  Other data sources include graduation rates, acceptances to four-year colleges, attendance rates, discipline referrals, and levels of authentic student engagement. In terms of engagement make sure that it is actually leading to learning. Understand though that not all data is good data and that we should not be obsessed with this. However, saying it does not have any importance is unrealistic.  
• Observations/Evaluations: To really see if teaching, learning, and leadership are changing, administrators have to get in classrooms more. As principal my teachers had a combination of five of these each year (3 unannounced observations, mid-year evaluation, end of year evaluation). In addition to this, my entire leadership team and I conducted non-evaluative walk-throughs each day. We can't forget that building leaders can use just as much support as teachers. Administrators are in desperate need of more quality feedback in relation to their role in digital implementations.
• Artifacts: Examples of digital lessons, projects, assessments (formative, summative, rubrics, etc.) curriculum, and student work that aligns with higher standards. Blog posts were a great way for me to showcase examples of these artifacts. Here is an example of a teacher using Instagram and the standards-aligned rubric. My teachers aligned artifacts to their observations to support not only what happened during the observed lesson, but also what happened before and after.  All of these artifacts were aligned to standards found in the McREL tool we used.  By the end of the year, all observation comments and artifacts populated into each teacher’s end of year evaluation giving me a body of evidence that clearly showed whether teaching and learning were actually changing. Each teacher wound up with a portfolio.
• Portfolios: Educators (teachers and administrators) and students can demonstrate evidence of growth and improvement over time in relation to learning goals. Everyone seems to talk about portfolios quite often, but I rarely see examples aligned to student and professional standards. 

Technology can and will have an impact on learning if and only if there is a focus on substance.  We must move past our infatuation with apps, tools, taglines, catchy sound bites, and broad claims that are not supported by either research or evidence of improvement. All educators should be able to answer the following question - How do you know that technology is impacting student learning and professional practice? Within this response should be examples of substance.