| Speeding up a machine learning algorithm involves efforts not only from algorithmic design, but also from software engineering perspectives. The study compares the performance of different implementations of Trust-region Based Gradient Boosting Machines (TRBoost) by benchmarking on several code snippets and datasets, using metrics such as training time, prediction time, memory usage, and prediction accuracy. The project firstly provides a comprehensive analysis of the algorithm and implementation weaknesses of TRBoost and then proposes methods to address them. The findings demonstrate the importance of considering both algorithmic design and software engineering implementation in performance optimization. It highlights the effectiveness of using profiling tools and benchmarking techniques in identifying performance bottlenecks and evaluating optimization strategies. The project also shows that parallelization and vectorization techniques can significantly improve the performance of TRBoost, and optimizing memory access and data structures is crucial in software engineering implementation. The results can guide future research in this area and suggest directions for further exploration, such as leveraging parallel computing and GPU computation and exploring more efficient and scalable C++ implementations of TRBoost. |
