Creating maths wizards with AR gaming
Figuring out numbers may become easier for people with learning disabilities, thanks to teens with cerebral palsy who are co-designing an augmented reality maths game.
Students and teachers at Central Auckland Specialist School (CASS) have been working with app developers to create a fun game that helps people with learning disabilities understand how to count.
The idea for an augmented reality (AR) app was first thought up by Nasser Giacaman (at top centre in the above photo), a software engineering education specialist at the University of Auckland. “I initially had the idea of combining AR with mathematics, as my university students and I were already exploring AR with geometry.”
Nasser was keen to use this technology to make maths more accessible so he approached Gail Ledger and Gillian Frankcom-Burgess at the university, who were researching how to teach mathematics to students with cerebral palsy.
Gail and Gillian already had a partnership with CASS through their research and both thought that an AR app would work well for the students there. The trio then contacted Melanie Langlotz, CEO of Geo AR Games, for developing the prototype app and creating the game's storyline.
At CASS, five senior students (aged 18 to 21 years) – William, Oscar, Vincent, Quinn and Miracle – were the main testers for the game. They played a vital role in making sure that the game actually was attention-grabbing, motivating and memorable for players like them.
“I’m excited about testing the game,” said Vincent.
William (left), Oscar (third from left), Vincent (third from right), Quinn (right) and Miracle (out of shot) helped ensure the cards were readable.
The result is The Wizard World of Numbers, a game that puts the player into the role of an apprentice wizard in a magical world of spells, potions and keys.
The player goes on a journey that requires counting magical objects, measuring potion ingredients, or finding a certain number of keys in order to reach the final number spell test for each learned number as they progress through each of the 10 levels.
The app comes with printed game cards and image recognition software that make up the augmented reality component. For example, when a player focuses the device’s camera on a card with the number 6, the app displays six animated treasure chests that ‘float’ in front of the card, with a voice counting from one to six as each treasure chest moves in sequence.
There are also musical chimes that increase in pitch as counting aids for those with low vision or visual processing impairments. According to the teachers at CASS, “repetition with variety" is central to how students with cerebral palsy learn. As a result, the animations, voice, music and sounds change often and the game integrates multiple ways of learning to count.
In the feedback sessions, Melanie asked the students whether they found the cards easy to read, and whether they had noticed the chimes playing in a counting scene, as well as other questions. The students responded by nodding, blinking or pointing to pictures and words to indicate ‘yes’.
Melanie (right) and a teacher (left) demonstrating the app with Miracle (seated)
At the end of each level, the spell test involves focusing the camera on an empty ‘magic card’, which triggers the game to display a random number. The game then asks the player whether that number is the same as the learned number, to which they respond by pressing either of the ‘yes’ or ‘no’ buttons displayed in the game.
If the player is able to recognise the learned number multiple times, they win one of ten rewards – ranging from a cauldron to a wand.
When asked about whether they liked the types of rewards offered, Miracle said, “No, it would be better to have a dragon as a reward”, and the other students agreed with him. Since a dragon would be too large to visually represent in the ‘rewards room’, Melanie suggested that she and her team would instead put a dragon in a future version of the game.
Each reward is a step towards the player proving themselves as an apprentice wizard and brings them closer to freeing the Great Wizard, who is locked away in a castle.
Melanie asked the students if they wanted to free the Great Wizard, and if they thought the game might help others like them learn to count. All five said “yes” to both questions, with Quinn giving a strong thumbs-up for the game.
The prototype app will be updated as the team continues get feedback from users and develop ideas for gamifying larger mathematical challenges such as addition and subtraction.
Download the prototype app and try it yourself!
Apple Store | Google Play
About the project
The Wizard World of Numbers is led by the University of Auckland in partnership with Geo AR Gaming and Central Auckland Specialist School.
For more information, visit the Wizard World of Numbers website
View all stories