It’s a pandemic-era feeling we’re all familiar with — you’re listening to a colleague on Zoom or attending an e-learning course…when your mind starts to wander. How many emails do I have to send once this is over? What shall I have for dinner tonight? Can I squeeze in a quick workout before that? The list goes on.
“These days, we’re spending more and more time online,” says Ooi Wei Tsang, an associate professor at NUS Computing. “And it’s very easy to get distracted.”
The digital age has shrunk our attention spans — multitasking equates with efficiency, notifications and news updates pop up on our screens constantly, and TikTok videos that last seconds seem more appealing than Facebook posts that ramble on for paragraphs.
As teachers, Ooi and his colleague associate professor Zhao Shengdong are especially concerned with how shortened attention spans affect learning, in particular online learning. “Even before the pandemic, there were already online video lectures, for example on YouTube,” says Ooi. “Now even more so.”
“As you watch a video, your mind may wander for a few seconds. This is the type of mind wandering we’re interested in detecting,” he explains. “Because once we are able to recognise this, we can say at the end: ‘These are the things you missed, would you like to go over it again?’”
But current methods for detecting how attention spans fluctuate aren’t great, says Zhao. They typically involve self-reported surveys, where participants are asked to reflect on their state of attention at periodic intervals.
“The obvious limitation with this is that you’re only able to achieve coarse granularity because you can’t constantly ask people to self-report every second or so,” he says. “Therefore you can’t detect very small attention fluctuations.”
Additionally, existing methods only provide discrete data at distinct timepoints. But what the researchers wanted was a way to monitor attention states continuously. “If we can do this, then we can come up with more interesting intervention methods to allow students to learn better when they watch videos online,” explains Zhao.
Measuring moment-to-moment attention fluctuations could be useful in other scenarios as well, he says. It might, for instance, help detect when a long-haul driver’s attention is fading and send him a warning ‘wake-up’ alert.
Hit the spacebar
To find a suitable monitoring method, the two researchers and their students trawled through the literature. Their research threw up something promising: a model called gradual-onset continuous performance task, or gradCPT.
Although gradCPT has been a well-established technique used in psychological studies for many years, the researchers were the first ones to apply it to the field of computer science. “That’s the good thing about cross-domain research,” says Zhao. “It gives you new ideas.”
The approach is “a set of methods that are able to tell your attention fluctuation in a very small time granularity,” he explains. More specifically, in intervals as small as 0.8 seconds.
GradCPT, also known as a sustained attention task, works by getting participants to watch images — usually just two types — flash sequentially across a screen. Participants are instructed to hit the space bar when image A is seen, for example, but remain unresponsive to image B. “It’s a bit like playing a game,” says Ooi.
In the experiments he and Zhao ran, alongside undergrads Yan Zihan and Shardul Sapkota, as well as then-PhD student Zhang Shan, they showed 18 participants a series of images comprising either mountain or city scenes. Participants were told to strike the space bar only when they saw a city scene (which appeared in 9 out of 10 instances).
At the same time, participants were hooked up to an electroencephalogram (EEG), a device placed on their heads to measure brain waves. With this data, the researchers could identify how brain wave patterns corresponded to attentive versus non-attentive states, as reflected by the gradCPT.
“This gave us the ground truth which we then used as input for the machine learning model, to train the model to recognise when the mind is wandering,” explains Ooi.
Brain waves are telling
Next, the researchers had to validate their model to see how well it could predict attention states. They recruited 24 new participants, placed EEGs on them, and showed them a 22-minute long video on the history of psychology. At ten random intervals, the video was stopped and participants had to indicate if they were paying attention or not just before the pause occurred.
The findings from the validation exercise were highly encouraging. By comparing the participants’ yes/no replies with the EEG readings, the researchers determined that their model was able to detect moment-to-moment attention fluctuations with an accuracy of nearly 73.5% (read the paper detailing their findings here).
“Looking at the literature, the accuracy we got is quite good,” says Ooi. Especially given how noisy brain signals can be, he says. Tiny movements, such as eye blinking, can generate spikes in the EEG pattern and distort the results.
“If you don’t clean up this noise, then the algorithm isn’t able to achieve good accuracy,” Ooi explains. But the process was certainly a challenging one, he admits.
Apart from reducing noise, there are other ways to improve the model’s accuracy too, says Zhao. But these involve tradeoffs. For instance, because brain wave patterns can differ between people, you could create a personalised EEG pattern for each individual to better predict their attention states (rather than using an aggregate derived from a pool of people), but this would involve a longer calibration period.
Or you might choose to use a more sensitive EEG — Ooi and Zhao used a device that takes readings from the forehead and earlobe, but there are others that can take measurements using multiple electrodes placed on the skull. “That would definitely help but that would increase the cost of the device and can be more invasive,” says Zhao. By comparison, the EEG they employed was portable, low-cost, and easy-to-use.
“We see our method as a kind of foundation that will allow researchers or practitioners to measure attention fluctuations more precisely, and come up with better intervention techniques,” he says.
For now, the researchers are working on sensors that can measure people’s attention states in a more relaxed environment or while they are on the go. Sitting as still as possible, which is what the participants had to do in the current study, isn’t feasible in the real world, they acknowledge. Ooi envisions future sensors might even be integrated into clothing, which would make things much more convenient — both for researchers and those being studied.
“Attention is actually a very, very important thing,” says Zhao. “Knowing people’s attention states holds a lot of potential in terms of understanding what’s important, and less important, to them. And whether or not the material you provide them is engaging or not.”
“In the future, you’ll definitely see that this is a hot topic for research,” he adds. “More and more people are now looking into how you detect attention and other intellectual states, and how you can use those states to provide a service that is useful to people.”
Paper: Moment-to-Moment Continuous Attention Fluctuation Monitoring through Consumer-Grade EEG Device
