Six NUS Computing graduates win Outstanding Computing Project Prize

5 Aug 2021 – Six NUS Computing graduates recently won the Outstanding Computing Project Prize, a new prize that was instituted by NUS Computing in 2020.

Four Computer Science graduates, Liu Zhaoyu, Monika Manuela Hengki, Ronald Santoso and Sean Ng Peng Nam, along with Business Analytics graduate Kong Yan San and Computer Engineering graduate Ang Jie Liang, each won the Outstanding Computing Project Prize.

The prize is awarded by the school to recognise undergraduates who have produced excellent Final Year Projects (FYPs), or projects under the Undergraduate Research Opportunities Programme (UROP). FYPs are year-long projects that are usually undertaken at the final year of an undergraduate programme.

A cash prize of $500 or $1000 is awarded to the individual or team respectively.

Detecting small and fast-moving objects in videos

Computer Science graduate Liu Zhaoyu was awarded the Outstanding Computing Project Prize for his FYP on developing an accurate and efficient ball-tracking model network for tennis games broadcasted on television.

Detecting a moving ball in broadcast sports videos is a fundamental and important task for the evaluation of players’ performance and analytics of game strategies. However, this proves challenging because a ball, like the tennis ball for example, is small, moves fast, and is often obstructed by objects such as nets or walls.

“An accurate and efficient ball tracking model can help obtain information such as bouncing and hitting points, ball speed, and a player’s hitting habit, in a cost-effective way without the usage of professional equipment like high-speed cameras and radar sensors. Moreover, such detailed information can be used by data-driven analytics technologies on strategy reasoning to improve winning chances and player performances,” explained Liu, who also has a keen interest in sports.

“Besides, highlighting the ball positions in a broadcast sports video could enhance the audience’s perception and entertainment too.”

The network model that Liu developed detects the location of the tennis ball at each timestamp from tennis videos, and determines critical moments such as bouncing points.

This is done by converting a pre-trained single-image object detection network into a sequential-based spatial-temporal ball detector through transfer learning and utilised temporal correlation on scaled-up feature maps.

The network model takes several consecutive frames of a video as input, then uses a pre-trained convolutional neural network-based (CNN) feature extractor to extract a scaled-up feature map from each frame. A pixel-wise dense classification is then performed on the combined feature maps to obtain a detection heatmap for locating the ball in the video.

“Through our experiment, we prove that our network is not only easy to train because it has few trainable parameters, but also has better performances and generalisability compared with existing methods. The accuracy of our ball detection model and bouncing points identification method can reach 97.84% and 90.49% respectively, which outperforms existing methods by big margins,” said Liu, whose FYP was supervised by Professor Dong Jin Song.

Liu’s ball tracking network model can also be generalised for many other sports, such as table tennis, badminton, and soccer, through transfer learning. The model can also be used for online sports videos, such as college games or non-professional games.

Having played various sports such as basketball and badminton, he thought it would be “cool” to combine his interest, and computer vision technology together, in such a project.

“I feel very honoured to be awarded the Outstanding Computing Project Prize. This prize is an affirmation of my efforts throughout the whole year. And it will also continue to encourage me as I begin my PhD journey,” said Liu, who is due to begin his PhD studies in Computer Science at NUS Computing in August this year.

“I would like to thank my FYP supervisor Professor Dong and my FYP tutor Jiang Kan for their great help and support in this project,” he added.

Developing an empathetic medical chatbot

For her FYP, Computer Science graduate Monika Manuela Hengki developed a medical adherence chatbot that is designed to be more empathetic to patients.

Medication adherence refers to whether or not patients follow the recommended medicine prescription by healthcare professionals.

Research showed that empathetic chatbots were perceived by patients to be more likable, trustworthy, caring. When medical adherence chatbots provide empathetic responses, the patients are more likely to feel as if the chatbot understood their feelings. Consequently, this would make patients more open towards suggestions given by the chatbot, and adopt more positive behavioural changes in their lives, explained Hengki.

To make her chatbot more empathetic, Hengki used the technique of reflective listening.

“Reflective listening is one of the core methods of motivational interviewing, and a standard counselling technique. To employ reflective listening, we use paraphrases to convey an understanding of someone’s issue, and demonstrate empathy,” explained Monika, who was supervised by Associate Professor Ooi Wei Tsang for her FYP.

To create such a chatbot, Hengki needed to use machine learning algorithms that would understand words and phrases, and perform the task of reflective listening.

One way to do this was to use Natural Language Processing (NLP), which helps computers process, understand, and manipulate natural language data such as speech or text.

“I formulated this into an NLP task of paraphrase generation and abstractive summarisation. Under the hood, the API uses PEGASUS, a Transformer-based NLP model, trained on paraphrase and summary data. Although the API was initially developed for a dental flossing context, it is flexible enough to be pivoted to be used in other chatbots in wellness and healthcare context, like mental health,” said Hengki.

“I also developed a new metric – ParaMetric – to measure the quality of a paraphrase. ParaMetric is an aggregate score of three measures on literature-backed criteria of what makes a good paraphrase. From our user study, ParaMetric performs better than other standard NLP metrics such as BLEU, METEOR, or ROUGE, in terms of correspondence percentage to human judgments,” Hengki explained.

Her project is part of Project Curantis, which was started by A/P Ooi with the aim of using AI for wellness and healthcare, with a focus on medication adherence.

“I believe this project has provided a valuable contribution to the community that is using AI for wellness and healthcare,” said Hengki.

She also experienced a steep learning curve in the initial stages of her FYP, but remained undaunted.

“I barely had any knowledge in NLP before taking up this FYP. In the first semester, I was drowning in a sea of jargon when reading various NLP research papers. Thankfully, my interest in the project kept me going,” said Hengki, who took it upon herself to sign up for a Stanford University online course on NLP and deep learning over the holidays to better understand the topic.

“I think my project stood out as it uniquely combines technology — Natural Language Processing (NLP) — and psychology, in the form of motivational interviewing, to tackle an interesting and relevant problem of making medication adherence chatbots be more empathetic,” she added. “This was totally unexpected, and I felt very grateful for the award.”

Making learning Mandarin fun

Having struggled with learning Mandarin when he was younger, Computer Science graduate Ronald Santoso was eager to find out how technology could benefit children who have a difficult time learning Mandarin too.

So for his FYP, Santoso developed a mobile and web application called SLIONS Kids that could make learning Mandarin for students in lower primary more enjoyable.

“There are increasingly many English-speaking families these days, and children are often not exposed to much Mandarin outside of their classes in school. This is a pity because younger children pick up language faster, and an early and frequent exposure to Mandarin will allow children to be bilingual in both Mandarin and English. Since young children use gadgets frequently, developing a mobile application that children can use at home will encourage them to learn Mandarin outside class too,” explained Santoso, whose FYP was supervised by Associate Professor Wang Ye.

He spoke with about ten Mandarin teachers from a local primary school to understand the students’ struggles with learning Mandarin and how best he could motivate their learning of the language through an app.

More importantly, he wanted the app to be different from most of the existing language applications, which were primarily designed for adult learners.

As a result, Santoso came up with an app that is fun and interactive. SLIONS Kids uses Automatic Speech Recognition (ASR) technology to provide immediate feedback to students when it comes to the pronunciation of Chinese words, and features fun Mandarin songs that users can listen to.

Within the app, students earn points in the form of ‘stars’ by listening to more songs and listening to passages being read aloud. They go on to earn medals if they sing along and read the passages aloud.

As the primary school teachers were also keen to incorporate the use of tech in their teaching, Santoso also developed a web app for teachers to monitor students’ usage of the app. This enables the teachers to come up with lessons that are better tailored to their students’ learning habits and interests.

The team is currently in talks with the Ministry of Education in Singapore to explore ways to incorporate the app in the syllabus.

“As someone who is passionate about education, this project is my first step into the field of education technology,” Santoso said. “I am greatly honoured to be awarded the prize and I would like to share this joyful occasion with my project supervisor, Prof Wang Ye, and the members of the Sound and Music Computing Lab who have worked closely with me for this project, especially Dr Xiao Chang and Ms Jin Minjia. Working with a solid team and an encouraging supervisor really pushes me to give my best for this project. I would not have been able to complete this project without their support.”

Improving congestion control algorithms to boost Internet speed

Computer Science graduate Sean Ng’s FYP involved improving a congestion control algorithm to solve an Internet traffic problem.

The Transmission Control Protocol (TCP) is a network protocol that forms the foundation of the Internet. It enables computers and devices to successfully send and deliver data and messages over a network.

In 2016, Google developed the Bottleneck Bandwidth and Round-trip propagation time (BBR), a TCP congestion control algorithm, to improve Internet traffic.

Such congestion control algorithms can be found inside any computer or phone connected to a network, and they control how fast a data is sent.

The BBR algorithm has higher bandwidths and lower latencies (i.e. a shorter amount of time needed for data to be transmitted) for Internet traffic, and is used to power sites such as Google, YouTube, and Spotify.

To determine the health and reliability of a network connection, one has to look at the Round Trip Time (RTT) of a network. RTT refers to the time it takes for a network request to be sent to a location from its starting point and back to its starting point again.

The RTT unfairness problem implies that in some scenarios, traffic from Internet users who are geographically far away might have a disproportionately larger share of the bandwidth, while those who are nearer might experience slower Internet speed.

To solve this, Ng’s FYP proposes two methods. The first method involves regulating the buffer size of the network bottleneck. The second method involves modifying the original algorithm to aggressively raise the number of data packets it can transmit, when it has a good chance of successfully transmitting data in response to aggression of other competing TCP flows.

“Most of the project involved practical experiments. I had to modify the kernel to get accurate data,” added Ng, who was supervised by Associate Professor Ben Leong for his FYP.

In computer science, the kernel refers to the computer program at the core of a computer’s operating system.

“The congestion control algorithm research area is not an overcrowded research area, but it is important enough that the field is worth significant interest,” he said.

Improving classroom engagement and learning using technology

When Business Analytics graduate Kong Yan San saw the issues teachers and instructors faced while transitioning to online learning during the pandemic, she decided to investigate how technology could be used to help teachers regulate learning in online classrooms as part of her FYP.

“Teachers and students who have limited experience in teaching and learning online were suddenly forced to immerse themselves in an exclusively virtual classroom. Managing virtual classrooms becomes exponentially difficult, and teachers are less able to sense their students’ engagement levels,” explained Kong, who was supervised by Assistant Professor Desmond Ong for her FYP.

Since student engagement has been shown to be positively related to learning, Kong wanted to enable teachers to monitor the engagement levels within an online classroom without having to manually scroll through the Zoom gallery to see if everyone was paying attention.

“If teachers are aware of students’ disengagement, they can effectively intervene and recapture the attention of the student, resulting in better learning outcomes. While monitoring student learning is already difficult in-person, it is made more challenging in an online classroom setting like Zoom, which has become more prevalent during the Covid-19 pandemic,” She said. “Teachers only have small Zoom windows through which to view their students, and additionally have to deal with other non-teaching related technological issues such as observing the chat box, making sure they are not muted, screen-sharing and dealing with connectivity issues. All of this makes it much more challenging for teachers on Zoom to get a grasp of their students’ engagement levels.”

First, Kong set about collecting a new dataset to develop a machine learning technique that would help teachers monitor students’ engagement levels,

She gathered actual Zoom lesson videos of teachers and students, which reflected various online learning conditions such as the presence of a Zoom background, varying levels of camera quality, and the usage of external monitors, which did not exist in previous datasets.

After identifying and labelling all the videos with the different online learning conditions, she put the dataset through a machine learning model to train the algorithms, so that it could make classifications or predictions about classroom engagement levels.

“I implemented the long short-term memory (LSTM) machine learning model that could be used to help detect and predict the engagement levels of students from the videos and I ran various experiments with different features (e.g. OpenFace, OpenPose, and VGG16) extracted from the model to test which was the best feature for predicting classroom engagement levels.”

Kong said she was surprised to receive the Outstanding Computing Project Prize for her FYP, as she felt that her machine learning models did not produce the results she wanted to see.

“I am very grateful for the award as it is a motivation for me to keep improving this project with my project supervisor, even after my graduation,” she said. “Even though I was very new to Affective Computing and Computer Vision, Prof Desmond Ong was really patient in guiding me through it all as we figured things out together. His trust in my work and ideas also motivated me a lot throughout this experience!” she added.

“Online learning is not going anywhere, even once the pandemic ends. I believe my research would have a long-lasting impact on education, as well as in other scenarios, such as ensuring safe driving or ads monitoring,” Kong said.

Understanding how database systems perform on Persistent Memory

In computing, a database system is an organised collection of stored data and is an important component in big data infrastructures of many enterprises.

Traditional data storage devices used are typically capable of only meeting at most two of the three performance requirements: latency, capacity, and persistency.

Latency refers to the amount of time needed for data to be captured, transmitted across devices and retrieved, while persistency refers to the characteristic of a state that outlives the process that created it.

There are trade-offs between using volatile and non-volatile storage devices to satisfy performance requirements. Volatile storage refers to a type of computer memory that requires power to retain the stored data – otherwise, anything stored will be removed when the system shuts down. Non-volatile storage devices do not require power to retain stored information (e.g. hard drives).

One such volatile storage device is the Dynamic Random Access Memory (DRAM), a type of semiconductor memory commonly used in personal computers.

However, recently, Persistent Memory (PM) – a high-performance byte-addressable memory device – is becoming increasingly attractive to the database community.

Unlike traditional data storage devices, PM is capable of high storage capacity, has low access latency of DRAM and the persistence of hard drives. As PM is a relatively new technology, existing research only focuses on the performance of PM in individual database operators and data structures, such as hash tables.

For his Final Year Project, Computer Engineering graduate Ang Jie Liang set out to find out how database systems – particularly online analytical processing (OLAP) systems – perform on PM.

In his paper, he presented a micro-architectural analysis of six Online Analytical Processing (OLAP) systems running on two types of memory devices: One on a new PM product, the Intel Optane DC Persistent Memory Modules (DCPMM), and another running on DRAM.

He also implemented a hybrid cache buffer pool on an open-source database system to showcase the potential gain in performance from integrating PM within database internals.

Ang then analysed the performance results from thousands of data points and derived certain observations about how modern database systems run on DCPMM. He discovered that micro-architectural behaviour of OLAP systems on DCPMM were significantly different from those on DRAM and data cache stalls were dominant when systems are running on DCPMM. The computer’s performance was significantly affected by three factors: intermediate writing of data, random access, and multi-threaded performance.

“The extensive experimental evaluations show that a clear understanding of device characteristics of PM is crucial to effectively integrate it into OLAP systems. I hope that my work revealed important characteristics of DCPMM in OLAP workloads, which can shed light on future OLAP system design and research,” explained Ang.

“This is a relatively new technology which does not have much existing research done yet. My project is the first to run an in-depth microarchitectural analysis of OLAP systems running on DCPMM. I believe the findings and insights are impactful for future research and integrations,” said Ang.

Ang’s project was done in collaboration with Internet company ByteDance, and was supervised by Associate Professor He Bingsheng.

“I faced various challenges when working with DCPMM and designing the experimental setup. It took me a few weeks to understand the hardware and configuration details of DCPMM in order to effectively design and automate extensive experiments and analyse the micro-architectural behaviours of six different OLAP systems from both hardware and software perspectives,” said Ang.

“I was very grateful and thrilled when I learnt that I won the award as I spent a lot of my time and effort working on this project in my final year, and am proud of the recognition my project received. I would like to thank my ByteDance colleagues – Qian, Jiong, Hieu and Jefferson – and Prof He for their helpful feedback and support,” added Ang, who completed his FYP while he was a software engineer intern at ByteDance.