9 December 2020 – Associate Professor He Bingsheng and his collaborators have won the IEEE Transactions on Parallel and Distributed Systems 2019 Best Paper award.
Their paper, ‘Exploiting GPUs for Efficient Gradient Boosting Decision Tree Training’, describes the development of ThunderGBM, a new, accelerated version of a Gradient Boosting Decision Tree (GBDT) algorithm.
In recent years, GBDT, a machine learning technique where multiple decision trees are trained and used to predict unseen data, has been widely used in areas such as advertising systems, spam filtering, and medical data analysis.
Despite its simplicity and effectiveness, GBDTs are very time-consuming to train. ThunderGBM was developed to improve GBDT training performance through the use of Graphics Processing Units (GPUs).
“We designed a series of techniques such as compression, approximation, and reuse of intermediate results to achieve acceleration of the training process. Our proposed optimisations and designs achieve a speedup of 40 times, with better scalability than current state-of-the-art approaches,” said A/P He.
However, developing ThunderGBM was a long and challenging journey for the research team.
“It is difficult to develop a GPU implementation of GBDTs, due to the memory-bound nature of the algorithm. We tried various approaches that didn’t lead to any speedup, and only came up with the current system design after many months of hard work,” said A/P He.
He added: “We are very excited about the recognition on our system research, and will continue to build high performance computing systems that can help more users in the community.”
