Metacognition in Maths Teaching: SSIF Evaluation

The SSIF project's funding application leveraged the high ranking of metacognition and self-regulation as effective teaching strategies within the EEF Teachers’ Toolkit. While the Toolkit doesn't offer definitive guarantees, it provides strong evidence-based information on potentially beneficial approaches. The project aimed to enhance primary mathematics teaching and improve pupil outcomes by focusing on metacognitive learning and self-regulation. The project's theory of change was that improved teacher practices in these areas would lead to enhanced pupil achievement and progress, particularly relevant for children experiencing high school mobility. However, initial challenges included the tight timeframe imposed by the DfE, with coaching starting mid-year (after February half-term) when school timetables and lesson plans were already established. A further challenge was determining whether Lead Teachers should follow their students to the next grade level in the subsequent school year.

The project involved ten schools. Initially, plans were for two full-time Lead Practitioners (LPs), each working with five schools weekly, but the project ultimately used three part-time, experienced LPs. These LPs provided coaching and ran termly network meetings for Lead Teachers (LTs). This allowed for sharing of best practices and additional teacher professional development (CPD). A practitioners’ re-launch conference and an end-of-project conference were also held to broaden participation and knowledge sharing. The coaching schedule involved advertising for LPs in Term 1 (September-December) with a fixed-term contract from January 2018 to April 2019. The Lead Teachers were selected by individual schools, allowing for autonomy. While the project focused on mathematics, not all LTs were subject leads in maths, showing flexibility in implementation. All of the LTs who successfully completed the coaching were women, highlighting an unexpected outcome of the project, suggesting there's potential for future research exploring gender implications in educational leadership and school improvement. The project also made use of additional tools, including LORIC (initially met with skepticism by LPs) and Basecamp (whose utility was debated by LTs).

The project's evaluative approach was based on a theory of change diagram developed for each school. This diagram charted anticipated causal relationships between project activities and desired outcomes. The diagrams were revised mid-project (end of Term 2) to reflect evolving school contexts. The overall evaluation method assessed the theories of change underpinning the project's design and implementation, both holistically and at an individual school level. The metacognitive pedagogical approach utilized drew on existing research, such as Vygotsky and Bruner's work highlighting the importance of language and communication in intellectual development. Furthermore, the project incorporated Dialogic Teaching methods which emphasize an active teacher role in facilitating thinking skills. The project also acknowledges Leat and Higgins' work on Powerful Pedagogic Strategies, such as Odd One Out and Mysteries, highlighting the importance of integrating thinking skills within subject matter and schemes of work. The coaching methodology incorporated elements of modelling, joint planning, co-teaching and debriefing, reflecting a contextualized, specialist coaching approach.

The project employed a contextualized specialist coaching model. While the Lead Practitioners (LPs) were experienced teachers, they weren't initially experts in metacognition. Their initial task was to gain a strong understanding of metacognition and its classroom application in mathematics. They sought training, attended meetings, and engaged in extensive reading. The coaching approach was highly contextualized to the individual challenges, year groups, and experience levels within each school. The LPs adapted their delivery and pedagogical approaches continually, based on gathered data and reflection. The coaching involved modelling, joint planning, co-teaching, and debriefing with the Lead Teachers (LTs). This wasn't a 'clean' coaching model; it integrated mentoring, guidance, and feedback, aligning with the 'specialist coaching' approach described by CUREE (2005). The coaching also drew on the 'Thinking through...' approach developed at Newcastle University, which emphasized subject-based infusion of thinking skills and Powerful Pedagogic Strategies.

The project initially planned for two full-time LPs, but ultimately employed three experienced part-time LPs. This proved beneficial, as each LP brought unique and complementary skills; the project manager believed the outcome was 'more than the sum of its parts.' The LPs themselves agreed that this structure functioned well. The LPs did not receive formal induction training; instead they were 'self-led' in developing their metacognition expertise. One LP noted that their learning was more effective due to the collaborative nature of their work. Their initial limited knowledge in metacognition prompted extensive self-directed learning, including meetings with the evaluation team's metacognition specialist. The timely release of the EEF's metacognition booklets provided credibility to the LPs’ work with both schools and LTs. The LPs also developed strong collaborative working relationships, utilizing a WhatsApp group for communication and support. They acknowledge that their work was at times hampered by the project's ethos of working 'with' the schools, rather than dictating to them, resulting in occasional difficulties when schools did not fully engage. The LPs also reported improvements in their own skills, notably in coaching, communication, and conflict resolution.

Schools were given autonomy to select their Lead Teachers (LTs). Although the project focused on mathematics, not all selected LTs were subject leads in the field. All the successfully completed teacher professional development (CPD) LTs were women. The LTs highly valued the network meetings, appreciating the time out of class and opportunity to network and share ideas. They viewed this aspect as crucial to the project’s success. The availability of the EEF metacognition report in April 2018 was seen as a valuable resource by some LTs, however, this indicates a need for more comprehensive dissemination of relevant materials to all teachers. The use of LORIC, a tool intended to facilitate group projects, was initially viewed skeptically by the LPs and somewhat inconsistently adopted by the LTs. Basecamp, another tool, was deemed a 'not bad resource' by LTs, although many struggled with utilizing it consistently. Initially, LTs expressed nervousness about rolling out the program to their schools, needing substantial support from the LPs. However, they gradually gained confidence in their ability to implement the new pedagogical approach with LP support. Overall, the LTs found the project valuable, noting significantly enhanced teaching, boosted confidence, and exploration of career development opportunities.

The project's evaluation used a theory of change approach, creating flowcharts for each school illustrating the anticipated causal links between project activities and outcomes. These diagrams were iteratively refined, acknowledging that the theory of change itself would evolve over time due to changing school contexts (e.g., student turnover, leadership changes, Ofsted inspections). The final evaluations assessed the extent to which desired outcomes were achieved, considering the influence of project implementation and context. This holistic approach, evaluating both individual schools and the overall project, aligns with Laing and Todd's (2015) definition of a theory of change as explicitly articulating how a project intends to achieve outcomes through actions, while considering context. The reliance on a theory of change framework for evaluation was appropriate given the project’s inherent implicit theory of change.

A significant challenge was the lack of baseline data for comparison, resulting in an assessment reliant on anecdotal evidence. The limited observations conducted by the Quality Assurance Consultant (only three schools observed twice) further hindered the evaluation of pedagogical practice changes and pupil engagement over time. This lack of robust quantitative data meant that the reported impact was largely anecdotal, or 'soft' data. Inconsistent data collection methods across the ten schools, due to current DfE guidance, also hampered the ability to attribute any progress directly to the project's interventions. One LP commented on the difficulty of definitively stating that any progress made was a direct result of the project and its interventions. Despite these limitations, anecdotal evidence from teachers and students pointed towards notable positive changes.

Despite the challenges in assessment, several positive impacts were observed. Lead Teachers (LTs) reported increased teacher confidence and enhanced teaching, supported by teacher confidence audits. Pupils demonstrated increased resilience, more confident mathematical discussions, and shifts in their perceptions of their mathematical abilities. Some pupils expressed newfound confidence in their abilities, stating that they were ‘really good at this now’ or ‘really good at maths,’ marking a significant change in self-perception. LTs also observed that pupils, particularly those who had experienced the intervention for a longer duration, were now using the metacognitive language introduced within the project. At the whole-school level, the project fostered increased peer classroom observations, both internally and externally. This, along with increased teacher reflection, was seen as a direct positive outcome by the LPs, with the uptake exceeding expectations. The use of cluster observations was recognized as one of the project's most significant and successful elements. The LPs also reported personal benefits, including improved coaching and communication skills, and increased confidence in navigating difficult situations. The project’s success in developing leadership is evidenced by the fact that an LT who had initially disliked her leadership role gained renewed confidence and a desire to return to leadership after participating in the project.

The project yielded positive results at the pupil, teacher, and whole-school levels. Lead Teachers (LTs) reported significant increases in teacher confidence, evidenced by the LPs' confidence audits. Pupils demonstrated improved self-regulation, increased resilience, and enhanced mathematical communication skills. LTs observed noticeable differences in metacognitive language and problem-solving skills among pupils who had participated in the project for longer periods, showcasing the importance of sustained implementation. At the whole-school level, there was a marked increase in peer classroom observations, both internally and externally, demonstrating a shift toward a more collaborative and reflective teacher professional development (CPD) culture. This increased collaboration was viewed as a direct outcome of the project's emphasis on sharing pedagogical approaches. The positive impact on teacher and pupil confidence suggests the potential for high impact with minimal additional workload and resources. The project manager specifically noted the unexpectedly positive outcome of increased female leadership within the schools as a result of the project.

To ensure the project's legacy, a resource pack and a professionally produced video are planned. The resource pack, available electronically and in hard copy, contains materials used throughout the project, including tools to assess impact (staff and pupil questionnaires), a metacognition guide for stakeholders, and links to relevant research and websites. The pack aims to support ongoing teacher professional development (CPD) and training of new staff. LTs positively received the resource pack at the final network meeting, confirming its value for implementing metacognition within their schools. The video showcases metacognitive strategies used in mathematics lessons across key stages. It will explain metacognition, cover the SSIF project itself, and present examples of metacognitive mathematics sessions on topics such as activating prior knowledge, scaffolding, modelling, and reflective evaluation. It will also include interviews with headteachers, SLEs, LTs, and pupils. An animated version of the revised EEF model will also be included.

The project's greatest legacy is expected to be the fostering of collaborative work among professionals. This collaborative approach is likely to enhance teacher collective efficacy, as identified by Donohoo (2017), contributing to a coaching culture within the participating schools. The increased collaboration will potentially result in better outcomes for students. The project's success in developing this collaborative approach is expected to extend beyond the initial participant schools. An end-of-project conference, open to all schools in the area, will further disseminate the project's findings and situate them within a broader context of metacognition. This conference will include keynotes and workshops led by LPs, academics, EEF staff, and others. The project's success in enhancing teacher professional development (CPD) and promoting improved pedagogical approaches focused on metacognition and self-regulation has the potential to lead to lasting school improvement across the participating schools and beyond.