In March 2022, The Learning Accelerator (TLA) launched the first of three Strategy Lab: Virtual & Hybrid cohorts to tackle the challenge of developing more effective, engaging, and equitable virtual or hybrid learning environments. Working through our Real-Time Redesign process, district teams started to conduct needs assessments to understand and identify a problem of practice related to their model in order to develop and pilot measurable solutions.
However, the districts were soon met with a challenge in this process: few models exist to help districts define quality in virtual and hybrid contexts. To meet this need, we designed two research-based measurement tools to facilitate meaningful conversations about quality and help teams identify potential problems of practice in context. We know from our own research — as well as past work with districts — that leaders need to build shared understanding and common language to make systemic improvements. These tools serve as a catalyst for conversations that can lead to improvement, provide a concrete means to identify quality, and play a central role in helping districts identify the underlying causes and factors affecting their students’ experiences within virtual or hybrid settings.
Tool #1: Individual Self-Assessment
Using the Key Factors that Help Drive Virtual and Remote Learning Quality as a framework, we designed a self-assessment survey that asks each respondent to rate either their level of confidence or the likelihood that certain drivers exist in context. For example, within the Pedagogy section, respondents indicate how confident they feel that students experience elements of mastery-based learning such as “giving and receiving feedback from peers.”
Since Strategy Lab teams consist of both district and building leaders, the self-assessment allowed each respondent to answer individually and anonymously before coming together to review a summary of the results. We structured the activity in this way for three reasons:
-
Allowing each individual to first respond anonymously mitigated potential power dynamics that could influence responses;
-
Discussion of the data summary presented an opportunity to build shared understanding as team members analyzed trends; and
-
The survey allowed us to collect pre-data that can later be analyzed to measure systems’ progress as a result of participating in Strategy Lab.
Tool #2: Team Assessment
Whereas the self-assessment captures individual perceptions, the team assessment identifies the prevalence of the drivers in context. This tool uses the same survey items as the self-assessment, but rather than ask individuals to rate their confidence in each item, it prompts teams to use a modified version of the Stoplight protocol from DataWise to indicate the degree to which each driver exists in context — consistently, in pockets, or not at all. This team assessment also prompts teams to add evidence to support their observations.
For example, one district indicated that they consistently “prioritize building relationships with students.” In addition to describing evidence such as having an advisory structure and regular individual meetings with students, the district also noted that 81% of their students responded favorably to a culture survey question asking whether they had a positive relationship with an adult.
Districts worked through the team assessment during coaching sessions with TLA’s Strategy Lab team. These sessions supported the district teams while they completed the assessment and served as an opportunity for us to gain valuable feedback about the tools. We modified some of the survey statements in response to requests for clarity, and explicit recommendations have allowed us to better address equity and accessibility concerns. For example, one team suggested that a quality driver should include communicating with families in an accessible manner regardless of visual or hearing impairment in addition to using their preferred language.
Next Steps
These two tools have already started to foster meaningful, evidence-based dialogue within the district teams and help them build context for their problem of practice. In addition, the team assessment will serve as a dashboard to help district teams monitor their progress throughout the Strategy Lab process.
In the coming weeks, we will continue to collect feedback from Strategy Lab participants and develop a rubric to evaluate their virtual and hybrid pilots. As districts reflect on the current school year and transition to the next, these resources (available below) can help them make concrete improvements so that every student experiences a more equitable, engaging, and effective learning environment, whether in-person or online.
