WATCH – Are Assessments Involving Pictures Valid in Global Contexts? Evidence From Preschoolers in Mombasa, Kenya

Rebecca Zhu, Postdoctoral Researcher from Schools2030 Research Partner, University of California, Berkeley, presents initial findings from the research project in a webinar and blog.

03 October 2022
By Sarah James

Many childhood cognitive assessment tools rely on picture stimuli. As these assessments spread globally, researchers make an implicit assumption: that children across early environmental contexts understand pictures in the same way. What if this assumption does not hold? Differences in early picture comprehension may drastically change the validity of assessment scores.

Our research investigates whether children in some early environmental contexts might perform more accurately on a vocabulary task involving objects than on the same task involving pictures. We worked with children, ages 2 to 7 years (average age of 4.5 years) in their first year of formal schooling at twelve participating Schools2030 schools in five sub-counties of Mombasa, Kenya. 96 children participated in a vocabulary task involving objects, and another 96 children participated in a vocabulary task involving pictures.

Local researchers presented each child with a Swahili vocabulary task involving twelve words that belonged to three different categories: nouns (i.e., “toothbrush” / “mswaki”, “hat” / “kofia”, “crayon” / “krayoni”, “fork” / “uma”), number words (i.e., “one” / “moja”, “two” / “mbili”, “three” / “tatu”, “four” / “nne”), and color words (i.e., “red” / “nyenkundu”, “blue” / “samawati”, “yellow” / “manjano”, “green” / “kijani”). Local researchers asked the child to point to the target word (e.g., “Show me the hat”; “Nionyeshe kofia”) from a larger array. These three kinds of words were chosen to create variability in difficulty levels: children living in Mombasa typically learn common nouns fairly early in development, number words slightly later in development, and color words only when they enter the formal school system.

Overall, we found that children had an “object advantage” – this is, they performed more accurately in the object vocabulary task than the picture vocabulary task. Moreover, we found that this overall accuracy difference between the object and picture tasks was driven by the nouns and number words (i.e., the words that children had some familiarity with). However, on the color words (i.e., the words that children were largely unfamiliar with), children performed similarly on both the object and picture tasks. These results suggests that a meaningful difference between objects and pictures, rather than other extraneous variables (e.g., differences in motivation between the two groups of children) drove children’s differing performance in the object task and the picture task. Overall, our findings suggest that assessments involving pictures may underestimate children’s cognitive capacities in some contexts.

How universal is this performance difference between object tasks and picture tasks? In current follow-up research, we investigate whether young children in Berkeley, California show the same “object advantage” as their counterparts in Mombasa. Moreover, most children in Mombasa grow up in fairly urban environments; how might children in a more rural area of Kenya, such as Lamu, perform on these kinds of object tasks and picture tasks?

Ultimately, we hope that this research will contribute to the development of more appropriate assessments to improve early childhood educational trajectories. This line of research is still in progress, and any questions or comments from readers would be highly appreciated!

Schools2030 is a ten-year participatory learning improvement programme based in 1,000 government schools across ten countries. Schools2030 supports teachers and students to design and implement education micro-innovations. These low-cost and scaleable innovations will inform and transform education systems to improve holistic learning outcomes for the most marginalised learners worldwide. Join the movement on TwitterFacebook, and LinkedIn.