According to modern portfolio theory, predicting the assets’ price trend is essential in optimizing portfolios. This study explores the potential and performance of machine learning for assets’ price trend prediction in the context of the cryptocurrency market. Three machine learning models, stacked Long Short-term Memory Network (LSTM), Vector Autoregression (VAR), and Autoregressive Integrated Moving Average (ARIMA), were used to predict the price trend of 14 selected cryptocurrencies based on the modern portfolio theory. The performance of these models was evaluated using R-square and MSE metrics. The results showed that the stacked LSTM model outperformed the other two models, demonstrating the potential of machine learning in cryptocurrency portfolio optimization. Furthermore, the study highlights the importance of carefully selecting machine learning models and cryptocurrencies for portfolio optimization. The study’s potential impacts include improving cryptocurrency portfolio management and inspiration for further research in this emerging field.