We are thrilled to share that the Abdul Latif Jameel Poverty Action Lab (JPAL) at the Massachusetts Institute of Technology (MIT) has selected BookNook for its second round of groundbreaking social science research projects. The full press announcement is here.
With our selection comes both a financial grant and technical assistance to help us work with one of our strongest implementation partners, the Boys and Girls Clubs of Greater Houston, to plan a rigorous randomized control trial research (RCT) project for the 2020-21 school year.
As a dual bottom line company, we are committed to ensuring that our programs don’t just delight students and help make teachers’ lives easier–we want to see that they are driving significant reading improvement for students.
While we are excited to have seen such a large effect size in the quasi-experimental design study we released in the spring, RCT is the ‘gold standard’ in social science research and will help us to better understand how students are benefiting from their time using BookNook.
We are grateful to the JPAL team at MIT for selecting us, and to BGC Greater Houston for signing on as our partner for this research–and most of all for our students, whose effort and engagement with our platform are what make all of this possible!
Read the release from MIT here.
Last fall we engaged Dr. Jing Liu, Ph. D, a Postdoctoral Research Associate, at the Brown University School of Education to complete an independent study on the effectiveness of BookNook used with students at five of our partner elementary schools.
When we compared BookNook students to their peers, the BookNook students made significantly more improvements in reading over the course of the school year. One school even gained almost 13 percentage points on their standardized test. What does this mean for educators?
The results of the study indicate that BookNook is an effective tool in supporting the mastery of literacy for elementary students
“When we looked at all BookNook students together across the schools, there was a positive effect with a magnitude of 1.8 standard deviations.”
In simple terms this means BookNook is taking kids above and beyond the average. When implemented effectively, BookNook fosters measurable success in reading outcomes among early readers.
The modern education system is all about added value: added value of a program, an intervention, a teacher. The results of Dr. Liu’s study indicate that BookNook is an effective tool in supporting the mastery of literacy for elementary students. Our students, who began significantly behind their peers, made sizable strides toward literacy mastery.
Download the report to learn why BookNook is quickly becoming a favorite guided reading solution for schools and non-profit programs!
First let’s look at the individual schools. Of the five schools included in our study, three showed positive effects and were statistically significant! Jing’s results show BookNook helped children from School II gain almost four full points on their standardized tests, equivalent to 13 percentage points. Meanwhile, our two KRR schools showed a similar growth of almost twelve points, equivalent to 8.6 percentage points of their standardized test.
When we looked at all BookNook students together across the schools, there was a positive effect with a magnitude of 1.8 standard deviations. This means, on average, BookNook helped students gain a letter grade in their ELA standardized assessments.
While two schools did not show the same positive gains, we are concerned about their results, but not discouraged. In School I, the BookNook results did not show statistical significance, or even close to the .05 threshold, meanwhile, the other interventions used showed a negative influence that was statistically significant. This would indicate to us that there might have been some implementation issues that should have been addressed, but the results should not discourage us from promoting our product. There is a similar case for School III. Perhaps even more exciting than our individual school results are the results of the combined schools test.
In statistics, increasing the population of a study increases the weight of the results. If you think about an experiment, the more times you do the experiment and get the same result, the more you can trust that the independent and dependent variables are related. Therefore, Dr. Liu combined the smaller populations of the five schools to simulate the results of one large population. Because the schools used different assessments, he standardized them to create comparable results and they show that BookNook improved test scores across the schools by almost two whole standard deviations! But what does that actually mean in real terms, well…
The mean score is a fancy way of saying the average. Standard deviations are ways of relating all the observations that create an average to each other. The classic example is to think of the bell curve (a normal distribution). The majority of observations fall under the bell, and the odd balls, the very high or very low observations fall further away. One standard deviation away from the average (mean) captures the majority of all observations (68% to be exact), two standard deviations captures 95% of all observations, three captures 99%.
Let’s look at an example. If the average score on a literacy test is 75 and I know that 68% of all test scores fell between a 70% and an 80%, that would mean one standard deviation from the mean was five percentage points. If a BookNook student took that test, they would score 1.8 standard deviations from the mean, a score of 84%. As Dr. Liu says, 1.8 standard deviations is a “gigantic effect size,” and a phenomenal indicator that we are improving students’ literacy skills. In even simpler terms, these results indicate we are helping kids get results well beyond the average of their peers.
Now we recognize that this coefficient is not statistically significant at the .05 degree most statisticians would like for the full stamp of approval, but we are statistically significant to the .1 degree. In the words of my old professor Dr. Rebecca Maynard, yes 90% is not the 95% convention, but it would be pretty foolish not to think those findings have impacts worth pursuing.
The final point we want to highlight from this study are the students. In our study Dr. Liu found that BookNook students on average “are academically much weaker than their peers.” Our mission at BookNook is to serve any student who is struggling in their literacy development, but our primary goal is to help those with the greatest need. These results suggest we are doing just that!
These results demonstrate an incredibly promising future for BookNook. As we improve our platform by incorporating other best practices as well as improve the training and development processes of reading guides, we believe these results will continue to improve. This is incredibly exciting news for the team because it means our platform is making real changes in our students.
Why did we want a study?
At BookNook, we believe that through our platform we can provide educators a tool that amplifies their skills and meets students at their individual level. To make sure we are providing a tool that is truly effective we want to incorporate and reflect evidence-based practices as much as possible. By funding a third-party evaluation of our own platform, we want to demonstrate that we are true to these values and transparent about the model we promote.
Making sense of statistical jargon.
Unfortunately for researchers (and educators), education does not happen in a vacuum. Children are all unique and they do not live in petri dishes. It is therefore very hard to do that traditional science experiment where we can control everything except our one variable, BookNook.
Instead, researchers must use lots of fun statistical techniques to isolate the real influence of interventions in our kids’ lives. In his report, Jing Liu used a common method of the OLS regression model to calculate the influence BookNook had on the students in the study compared to other variables. Put most basically, a regression model takes all the independent variables represented by a child, like sex, race, socio-economic background, attendance, and BookNook; and compares their influence on the dependent variable, for us a test score.
The equation might look like this:
Student’s test score = influence of age + influence of sex + influence of BookNook + influence of attendance
Within these models there are two very important qualifiers to each independent variable: effect size and statistical significance. Effect size is the magnitude of influence one independent variable has compared to others. Through effect size, the model is trying to tell a story, explain what exactly are the factors within an outcome and which are the most important. From our study we wanted to determine first, if BookNook had a positive or negative influence on student reading scores and second, to what degree.
In science jargon, statistical significance is the degree to which we believe the results did not happen by chance, the degree to which we believe it is safe to reject the hypothesis that our independent variable would have no effect on our dependent variable. In plain English, statistical significance is the test of, do we believe that our results are accurate in explaining an outcome? Within a statistical model, every output gets tested for statistical significance, so every effect size has their own test for significance.
You will see statistical significance represented as a percentage and it will never, ever be 0%. This is because in the statistical world, there is always the possibility that if we ran this experiment, random chance could explain our results. The convention within the statistical community is that you promote results (known as a p-value) of .05 or less, meaning we are 95% sure that what we see is not due to chance. If we got a p-value of .4, that means we are 60% sure that what we are seeing not due to chance; that result holds too much uncertainty and we would say ‘our finding is not statistically significant.’ The closer your p-value gets to 0.00, the more and more confident you can be.
Putting these two things together, in an evaluation we are looking for positive effect sizes and that those effect sizes are statistically significant.
Creating a Literacy-Rich Environment
By Samantha Burke
About the author: Samantha Burke is a credentialed reading specialist that has worked with children in the San Francisco Bay Area for ten years. She enjoys creating innovative learning opportunities for students and resources for their parents through her tutoring company, Valley of the Moon Learning.
Literacy-rich environments provide opportunities for students to interact with print and literacy tools in a meaningful way. By creating a supportive environment that is literacy-rich, students have more opportunities to practice literacy skills including: phonemic awareness, phonics, fluency, vocabulary, comprehension, listening and speaking in authentic ways. The way a classroom is set up, the materials that are accessible and the way interaction with materials is modeled will influence how students engage with literacy in it. Here are some simple things you can do to help promote a literacy-rich environment for your students. Happy Reading!
Have different types of age-appropriate reading materials available:
You never know what students might really take a liking to. Try incorporating a variety of books, magazines, newspapers, comic books and menus into your projects, games, play areas or classroom library.
Play games that promote reading:
Bananagrams, Spot It, Sequence Letters, Boggle, Hedbanz, Story Cubes, Sight Word Bingo, Tall Tales: The Game of Infinite Storytelling, and Reading Rockets has a list of games you can make yourself here.
Create cozy places to read:
Comfortable places to sit or lay with good lighting. You can add pillows, stuffed animals and even small lamps in your classroom to make it extra cozy.
Switch books out seasonally:
By changing up book options, children will be more interested in looking through books. They will like seeing what’s new, what’s the same and what’s gone. The library and sites like Thriftbooks are great resources to help spice up your classroom library.
Incorporate reading opportunities into your classroom environment:
Label things around your classroom and school – better yet, have students help you do the labeling. For younger students you can add words and symbols to blocks, label items in dramatic play area, as well as add notepads or dry-erase boards to play area.
Introduce students to as much vocabulary as you can and give them opportunities to use their vocabulary:
Talk to them about experiences, ask them about their opinions, model using language that will help them better communicate with others.
Look for reading opportunities wherever you are with your students:
Walking around school, sitting in the library, on a field trip, waiting outside for dismissal, waiting in the lunch line.
I hope you find these tips helpful in creating a literacy-rich environment in your classroom. If you have any additional ideas please feel free to share in the comment area below!
Get a demo
About 10 miles south of the Wisconsin/Illinois border, Carrie Modra is hard at work with her students. As a Woodland School District 50 educator for 8 years in her role as a Speech-Language Pathologist, Carrie is an integral part of the daily ins and outs of the school day at Woodland Elementary East. In fact, she loves it so much that she has gone back to school to get her second master’s degree in School Leadership and works directly with her Principal mentor because of her passion for education.
For the completion of her master’s degree, Carrie needed to choose a topic for her internship project. After discussions with her principal about school goals and needs, Carrie focused on third grade reading. She was very excited to involve parents in the project, so she utilized the Woodland PTA to recruit volunteers for the BookNook program. This allowed for teachers to recommend students to BookNook and have volunteers guide the reading as an additional weekly reading support. Recruiting volunteers can be difficult, but Carrie was hoping to learn a lot through this project, and it felt like a great opportunity.
This post is the second part in a series we are doing on Barriers and lessons learned on implementing Technology in Education. Check out part one here.
Overcoming Barriers: Emerging New Best Practices
As daunting as the barriers may seem, current EdTech research is finding several ways schools can remove or at least address them in order to achieve technology integration.
Involve Teachers in Decision-Making
First, developing a shared vision and technology integration plan can help overcome the leadership barrier (Hew and Brush). “Probably the most important issue to consider when formulating a shared vision regarding technology integration,” they write, “is to address the specific relationship between technology and particular curriculum content areas because a commitment to the curriculum is a critical scaffold for technology integration.” Notably, they recommend that “the vision should not be created by just the school leaders. Instead, “teachers, in particular, should be involved in the decision-making because teacher participation has been found to be one of the ingredients for successful wide-scale integration of technology in a school district.”
Strategic Placement of Resources
If there’s a scarcity of resources, schools can “introduce technology into one or two subject areas at a time to ensure that teachers and students in those areas have adequate technology” (Hew and Brush). Technology integration almost never happens all at once, and it’s often the rush to scale-up that leads to bigger problems.
Changing Attitudes and Beliefs
One clear benefit to teachers is a school leader who provides ongoing professional development, encourages experimentation and improvement, and grants freedom to take risks and make mistakes. It is “more important to focus on the features of professional development rather than its types” (Hew and Brush).
The most effective professional development does the following:
- Focuses on content (practical skills, specific knowledge)
- Gives teachers opportunities for “hands-on work”
- Is highly consistent with teachers’ needs
A New Vision of Assessment
Schools must find a balance between “considering how technology can be used to meet the current demands of standards-based accountability” and altogether reconsidering assessment approaches once technology is integrated into the curriculum. Teacher’s can’t abandon standardized testing entirely, but they shouldn’t have to invent whole new assessments to include technology. Computer adaptive testing, or tailored testing, is just one example of how the technology itself can help educators re-imagine assessment possibilities.
A Mentoring Approach
Theodore J. Kopcha advocates for a model of integration that uses “mentoring and communities of practice to support teachers as they develop skills, pedagogy and beliefs needed to integrate technology in a student-centered manner.” Mentoring, Kopcha writes, “has been found to overcome many of the common barriers to technology integration.”
For instance, mentors provide teachers with “just-in-time support while they integrate technology into lessons they are actually teaching.” Good mentoring begins with knowing teachers’ needs and setting goals collaboratively throughout the entire process of integration.
Technology as Part of School Policy
Veteran teachers Nancy Frey and Douglas Fisher integrated technology into language arts classes, then began creating a technology policy to monitor its uses. Soon, the authors realized that most “technology policies focused on prohibition” rather than teaching students to use it constructively. So they created a school-wide courtesy policy that included expectations of technology courtesy, thereby fully integrating technology expectations into the whole school’s standards of behavior.
For as many barriers as schools face when integrating technology, there are equally numerous, as well as creative, solutions. All require deep collaboration, clear planning, and ongoing professional development and assessment. Technology integration is never quick or easy, but it can be successfully planned and implemented.
See the Source List for this article Here