Episode 9: Learning from Phonics Research with Anna Comas-Quinn

In the ninth episode of The Sounds-Write Podcast, Sounds-Write's Head of Research and Development, Dr. Anna Comas-Quinn, talks about the trials and tribulations of conducting research in the field of phonics. She talks through some of the research that Sounds-Write has conducted, what we can learn from that and why context is essential when evaluating schools' results.  Enjoy!

Some helpful links:
Condensed version of our longitudinal study
Systematic synthetic phonics: case studies from Sounds-Write practitioners
Sounds-Write’s Success Stories
Learning to read is vital: here’s why
Episode 8: Literacy Intervention with Jacinda Vaughan
Pre-register for the Sounds-Write Practitioners' Portal!
Sounds-Write's Facebook
Sounds-Write's Instagram
Sign up to our mailing list

Transcript

Laura:  00:02
Hello and welcome to the Sounds-Write podcast. I'm the host, Laura, and in today's episode I'm speaking with Anna Comas-Quinn. Anna comes from a background in academia and has a doctorate in education. These are expertise that she now brings to Sounds-Write as Head of Research and Development. In this episode, we discuss the challenges involved in conducting research in the field of education, why context matters when evaluating schools' results, and how teachers can judge whether their phonics programme is working well for them. I hope you enjoy the episode. Hello, Anna, welcome to the podcast.

Anna:  00:38
Hi, Laura, and thanks for having me.

Laura:  00:41
No worries. So, could you start off by telling us a bit about your professional background and what it is that you do at Sounds-Write?

Anna:  00:48
Yes, of course, Laura. I joined Sounds-Write about a year ago, after 25 years as an academic. My job in academia in higher education was designing and teaching undergraduate and postgraduate courses, and I carried out research in the fields of online and open education, and teacher professional development. At Sounds-Write, my role is to evaluate what we offer. So, I spend a lot of my time collecting evidence to understand the experiences of Sounds-Write practitioners during and after their training, also to identify what additional support and resources we can develop to maximise the impact of Sounds-Write, and also to prove and to showcase the effectiveness of Sounds-Write.

Laura:  01:34
Lovely. So, the first thing we were going to talk about is that when Sounds-Write was first developed - and we've heard a whole episode before on what happened when Sounds-Write was developed and all of that process - but a study was conducted to evaluate the effectiveness of the Sounds-Write programme in those early years of Sounds-Write. Could you talk a bit about the process of setting up that study?

Anna:  02:00
Yes, of course. Well, from the very beginning, there was a real keen interest in collecting data to show the effectiveness of Sounds-Write. So the programme was designed in the early 2000s and it was first piloted in 2002 at St Thomas Aquinas, which is a school in Bletchley in Buckinghamshire in England. The results were really, really good, really impressive. And then Sounds-Write started to train teachers from the following year. Over the next few years, Sounds-Write persuaded many schools to test their children at the end of every academic year and to share the data with Sounds-Write. And the idea here was to track the literacy development of children who were being taught using Sounds-Write, and to do that from Reception to the end of Year 2. And to explain a little bit, in England, children start Reception on the September after their fourth birthday. So, what we were looking at here was literacy development between the ages of four and seven. What they chose to use for that study was a normed spelling test. Now, a test that is normed or norm-referenced means that it tells the teacher, it can tell the teacher, whether a student's performance is better or worse than that of a typical student in a comparable group. But I guess the big question that comes to mind is, why did they choose a spelling test and not a reading test, if what we're trying to find out is how well a student can read? And there were very good reasons for that choice. The first reason is that Sounds-Write teaches reading and spelling together because the code is reversible. So many Sounds-Write lessons include a step of writing the word and then reading back what has been written. And research shows us that writing really helps with getting that code into the long-term memory. The second reason is that spelling requires retrieval memory. It's harder than reading. Reading only requires recognition memory. So a student cannot spell accurately a word that they cannot read. So if we are able to determine a student's spelling age, we can then be fairly confident that their reading age will be equal or higher than their spelling age. And finally, there was a very pragmatic reason. Reading tests are very time consuming to administer because each child has to be tested individually, whereas with a spelling test, you can administer it to the whole class, and it also generates a written record of results for each child, and then that can be marked by the teacher and shared with the researcher.

Laura:  04:33
Wow, that all makes a lot of sense. I hadn't thought of that before, why you would use a spelling test rather than a reading test. So, the study itself, we refer to it as a longitudinal study. What does that mean?

Anna:  04:49
Yes, a longitudinal study means that the same children would be tested at different points over time. So, in this case, they were tested at the end of Reception, at the end of Year 1, and at the end of Year 2. And the data was collected over a period from 2003 to 2009, as more and more schools were added to the study. And by 2009, it was felt that there was enough data, enough data had been collected, and the results could be analysed. So, in total, we had, included in the study, 1607 children. They came from 76 classrooms in 24 schools. And the schools were spread over four locations across England. So there were some schools in Greater Manchester, in the North. There were some schools in Milton Keynes, some in Bedfordshire, and some in Kent. Now, it's also important to think about, you can't include every single child in the test, and there's good reasons for that. So we actually tested about just over 2000 children in Reception, but not all of them could be included in the final analysis, and some of them had to be excluded for the following reasons. Either they had moved to another school after Reception, or maybe they weren't there on the day of the test in Year 1 or Year 2, or maybe they had been taught in Year 1 or Year 2 by a teacher who hadn't been trained in Sounds-Write. So that left us, from those over 2000 children, the 1607 children included in the study. The overall result was that 91% of the children who had been taught using Sounds-Write for three years had literacy skills that were age appropriate or above, and that meant that they were ready to move on to Year 3 with literacy skills that allowed them to participate in learning and access the rest of the curriculum. The measurement that was used in the study was a comparison between the spelling age and the chronological age of the child. And the results showed that by the end of Year 2, there was only that 9% of children in the study that had spelling ages that were more than six months behind their chronological age. And what was even more remarkable was that the average spelling age for these 1607 children in Year 2 was more than eleven months higher than their chronological age, and that is an average. So for some children it was a few months higher, but for some children it was several years higher. It's worth saying as well that the school in which Sounds-Write was originally tested, which was also part of this study, still uses Sounds-Write today. And their reading results are incredible. They are actually in the top 10% of all schools in England for reading.

Laura:  07:41
Wow. Brilliant. We've got a kind of condensed version that you've put together, actually, of that study, the data from that study. So, yeah, I'll link that in the show notes so everyone can go and have a look at that if you're interested. So what are the limitations of a study of this sort? I know it must be really difficult to organise and carry out such a huge study over such a matter of years. So what are some of the ethical and logistical challenges that come up when you're carrying out this sort of research in the field of education?

Anna:  08:16
Yeah, the logistics are quite challenging. It's difficult to get that kind of numbers that were achieved in that initial study. A lot of published research on phonics teaching is based on very small groups, so it took many years of persuading teachers to use the spelling test and to share the data with us. And also it helped that the design of the study was very simple. Even then, as I've explained, we lost almost a fifth of the children that were tested in Reception. We couldn't include them because they hadn't been following the programme for the three years. You could also set up a study to compare Sounds-Write with other phonics programmes, for example. But from our point of view, that would be unethical really, because we truly believe that Sounds-Write is the most effective way of teaching reading and spelling. So why would you purposefully keep children from being taught using Sounds-Write? Another challenge when comparing the performance of different schools is that it's very difficult to control all the variables. For example, in this initial study, we found that in every single one of the 76 classrooms included in the data, the average spelling age achieved in the test was higher than the average chronological age of the students. However, these ranged from just a few months higher to more than two years higher in one school. So we looked at what the schools that had performed better were doing and this is what we found. They were teaching Sounds-Write with fidelity, 30 minutes every day, and they had designated one person to lead the implementation of Sounds-Write. So the study gave us this insight into the ideal conditions in which Sounds-Write will have maximum impact. We couldn't control those variables at the time of the study, we didn't know about it. But we can now strongly recommend those three things to schools. Teach Sounds-Write with fidelity, 30 minutes a day, and have a phonics lead to take responsibility for implementing Sounds-Write across the school.

Laura:  10:21
Brilliant. Those are some really kind of actionable points to take away from that study. I think that's great.

Anna:  10:28
Yes.

Laura:  10:29
So in May of 2022, more recently, we released a book of case studies. It's called Systematic Synthetic Phonics: Case Studies from Sounds-Write Practitioners. This was put together by you and a couple of other colleagues at Sounds-Write, and it's a collection of case studies from Sounds-Write practitioners, as the title would suggest. Could you tell us a bit about what it was like to put that together?

Anna:  10:53
Yes, I just loved working on that book. It was such a great experience. We invited Sounds-Write practitioners to write about their own experiences of using Sounds-Write in their schools and their settings and to provide some evidence for the impact that Sounds-Write was having. And we received many proposals and then we had to make sure that in our final selection for the book, we were offering a good variety of contexts in which Sounds-Write is used. So that included various settings such as schools, the work of speech therapists, reading clinics, and also across several countries and continents, schools in small towns, city locations, etc. We provided authors with a very simple template to ensure that the case studies would be concise and clearly structured, because we know that teachers are time poor, so we wanted to make it easier for them to find the relevant information. So authors had to provide some context on their setting. They had to explain how they had implemented Sounds-Write, then present their data and show how they had evaluating the success of Sounds-Write. And then finally we asked them to offer some recommendations for others who were considering adopting Sounds-Write. And then each one of the three editors worked with individual authors to help them tell their story and to ensure that the resulting case study was as strong as possible, as clear as possible, and the best it could be.

Laura:  12:20
Brilliant. So the book of case studies is peer reviewed. For us mere mortals who are not in the field of education and research, what does that mean in this context?

Anna:  12:31
Well, peer review in academic context is a process in which some suitably qualified scholars will be asked to read an article or a book chapter and make a judgement on whether the content should be published or not, and also make suggestions on how to improve the quality of the text, both in terms of content, but also presentation, language, clarity, etc. In the academic publishing world, what is used is called blind peer review, and that is where the reviewers don't know who the author is and the author doesn't know who the reviewers are. But it was a bit different for our book of case studies. Once the authors had prepared their draft chapter, we asked their peers, and these would have been then other education professionals and Sounds-Write practitioners, to act as reviewers. But we used an open peer review model, and that means that everybody involved, the authors and the reviewers, knew each other's names, even though they didn't know each other personally.

Laura:  13:28
Great. Something else that's quite notable about the book of case studies is that it was written during the pandemic, and as we know, schools were hit really hard by COVID and the lockdowns had a huge effect on children's education. So what was the rationale behind including that COVID-19 data in the book, and what were some of the challenges that kind of sprung from that?

Anna:  13:56
Yeah, it's true that we had just started planning the book of case studies when the pandemic began, and it was difficult to know at the time how long it would take for things to go back to normal, whatever that means. So we decided to just go ahead with the book and asked the authors to provide context for the data that they were presenting to explain how their teaching had been affected by the lockdowns. And there was lots of lockdowns in the UK and Australia, for example. And it's true a considerable amount of learning was missed due to the lockdowns, but also illness, isolation rules. And of course, this had a huge impact in the early years of school, when children begin to learn to read and spell, and also in children's readiness for learning, because many of the youngest children had missed on socialisation and preschool experiences as well.

Laura:  14:48
Yeah, and I think that brings us on nicely to my next question, actually, which is about kind of the contexts and the settings that were included in this book. So we were lucky enough to have Professor Pamela Snow write the foreword to it and she commented on the diversity of contexts that were included in the case studies. And I know you've talked quite a bit before about how important it is to take into account those contexts when you're looking at schools’ data. So could you tell us more about why you think it's so important to show data from a variety of contexts and settings and bear those settings in mind as well?

Anna:  15:29
Absolutely. Sounds-Write is used across the world in different educational systems, geographical locations, levels of disadvantage, age groups. It's very difficult to compare the performance of a programme across such a variety of settings. You need context, you need a rich description of the conditions in which the programme is being used, because they are so different. So, for example, in our book of case studies, we have a primary school in Western Australia that is also an Intensive English School and it supports students who have just arrived in the country as refugees. So three quarters of the students don't have English as their first language and about half of the students only stay in the school for one or two years and then they move back to their local school. So literacy is a big priority in this school and they do a great job, but in very different circumstances, from, for example, a school in an affluent suburb in a major city with a stable population of middle class children. So that's why it's so important to get a full picture of the conditions in which Sounds-Write is being used, and why just looking at results can only give us a partial, and sometimes quite misleading, picture. The book of case studies was a really great way of evaluating how Sounds-Write is being used and letting us know that it is effective. But we've also been putting together more informal short case studies that tell the stories of schools who have implemented Sounds-Write, and these can be found on our website. There's a section called Success Stories where we keep adding them.

Laura:  17:07
Brilliant. So what do you think that teachers can kind of take away from all of that? How do different contexts affect teachers on a day-to-day and individual level, do you think?

Anna:  17:18
Yes, teachers do a very difficult job, but they are professionals and as such, they are capable of exercising their professional judgement. So they will look and analyse the situation in their classroom and their context to understand what's missing, what are the gaps in knowledge, the misconceptions that need to be addressed. And formative assessment is embedded in Sounds-Write lessons and will give the teacher that information. So teachers can do that, can analyse what is happening in their classroom, they can make decisions then on what to do with that cohort of students in front of them. We might have a cohort one year that moves through the units really fast and then the next cohort the following year just needs lots of practice and lots of repetition and the pace has to be slower, so the teacher needs to make those decisions. And obviously also as a professional, the teacher will begin to develop their own practice further to make better decisions, to achieve better outcomes for their students. So that's why we like to describe Sounds-Write as an approach, why we are reluctant to tell teachers what to do every week, because teaching has to be responsive. No two schools are the same. No two cohorts are the same. So each teacher needs to apply Sounds-Write to their own class, to their own circumstances at that particular time.

Laura:  18:41
Yeah, absolutely. This is something that comes up quite regularly in our team discussions and a lot of other phonic schemes provide a timetable, pretty much your entire academic year timetabled out for teachers, kind of already showing you you're going to teach this lesson on this day, six months from now. But we think at Sounds-Write that that ignores basically the fundamental role of teachers, which is to make the best decisions for the students that are in front of them.

Anna:  19:12
Exactly.

Laura:  19:14
So, aside from formative assessment and testing, what you just talked about, how can teachers make sure that the phonics programme they've selected is working for them? And what are some of the things that they should kind of be on the lookout for?

Anna:  19:29
Yeah, let's start with this. Reading and spelling are essential life skills and without them, children's life chances are seriously curtailed. For example, poor literacy will limit a person's employment options, their earning power, their ability to access culture. We know, for example, that prisoners have lower levels of literacy than the general population. And with a big difference, I mean, there was a study carried out in Texas in the year 2000 that found that approximately 80% of prisoners in Texas were functionally illiterate. Why am I saying this? Because a phonics programme has to start from the premise that every child can learn to read and spell and needs to learn to read and spell, because the consequences of not ensuring that a child can read are terrible.

Laura:  20:22
Yeah, of course. We put a blog post up recently on this, actually, on our website, and there's huge consequences for children who aren't taught to read and write properly, both throughout their education and obviously in later life as well. So we can link that in the show notes for anyone interested.

Anna:  20:41
Yeah, please do, because that is such an important point. So when you consider the suitability of a phonics programme, you have to keep some things in mind. You want a phonics programme that doesn't put a cap on children's potential by grouping them into ability groups or streaming them according to their previous performance. We know that this only leads to bigger achievement gaps. You want to choose a phonics programme that ensures that all children are taught the same content because they all need to get to the same place. And, of course, you will add the necessary scaffolds so that all children can access what's being taught and there will be additional interventions to provide more practice and more repetition for those that need it. And I just wanted to mention, on YouTube, there's a little bit of an episode of the Simpsons in which Bart is in a remedial class and they're going to learn the letter A. And Bart says to the teacher, 'So, let me get this straight. We are behind the rest of our class and we are going to catch up to them by going slower than they are?' And it's exactly... It's just crazy, isn't it? So how do you know if the phonics programme that you're using is working? Well, by keeping a very close eye on children's progress, by looking at the data. And by that, I don't mean doing lots of testing and spending lots of time arranging the results to show them to your school leaders. At Sounds-Write, we favour formative assessment over formal testing. Formative assessment takes place in every lesson, it's just part of the way in which we teach. Every lesson is an opportunity to check the students' understanding and to modify the teaching to achieve the desired learning. So watch out for phonics programmes that provide extensive assessment materials and lots of teaching time then has to be spent in testing. Often very little is done with the results of those tests to really find out what the gaps in students' knowledge are. Teachers need to think really hard about the purpose of all this testing and whether the time might be better spent in teaching and practice. Some children will require much more practice than others before things click into place and the new knowledge goes into their long-term memory. They need more teaching, more repetitions, more practice, and the sooner those interventions are put in place, the better. And arranging the interventions, delivering them, with properly trained staff and giving those children the additional time that they need, is the only way of ensuring that they can catch up.

Laura:  23:22
Yeah. Our most recent episode of the Sounds-Write podcast actually was on this topic. Jacinda talked through all things intervention and people responded really well to that episode. Actually, lots of people reached out saying that they'd found it really useful. So, if you're interested in hearing more about running effective interventions. Yeah, it was the most recent episode of the podcast.

Anna:  23:46
Yes. And I really recommend that people listen to it because it's such an important part of the programme. So, how can teachers make sure that the phonics programme they're using is working for them? There's one final thing, and that is fidelity to the programme. You can't know whether a programme is effective if you're not applying it properly. Once you've decided on a phonics programme, you should be all in. Mixing in other things can confuse the students because the approaches might be contradictory or you might be adding unnecessarily to their cognitive load. Sounds-Write has been designed very carefully, so trust the programme and you'll see the results. Whatever phonics programme you're using, as a teacher, I think you already know whether you're getting the results. And remember what we said earlier. The result we should be aiming for is that every child learns to read and spell. If you're not getting the results, I think there are only two possible explanations. Either the programme is not being applied properly and consistently across the school, and then there's a job to be done there in finding out what's happening and how to fix it, or you're just simply using the wrong programme.

Laura:  24:57
Yeah. And it makes me think, actually, of, you guys were recently at a conference in the States and our colleague Jill was kind of, people were coming up to her and saying, okay, well, why should we change our reading and writing scheme? And she was saying, well, is what you're doing working, you know, you're the one who knows, you're the teacher, you're in the classroom every day, you know whether it's working or not. 

Anna:  25:24
Exactly, yeah. 

Laura:  25:25
Well, this has been brilliant. I think often we kind of read research, we see data and we don't really know what goes into collecting all of that. So it's been amazing to hear some of that straight out of the horse's mouth. And I hope that this has kind of given our listeners a little insight into how Sounds-Write works behind the scenes and how we make sure that the programme is effective and grounded in research. So, yeah, thank you so much, Anna, for this.

Anna:  25:55
Thank you, Laura. It's been a pleasure.

Laura:  25:57
Brilliant. And we'll see you next time. Thank you for listening.