Independent Grouped Information Expert Model: A Personalized Recommendation Algorithm Based on Deep Learning
DOI:
https://doi.org/10.53469/wjimt.2024.07(02).10Keywords:
Machine learning, Recommendation System, Deep learning, Multi-Task LearningAbstract
Deep learning-based artificial intelligence applications have driven transformation across multiple industries, and recommendation systems based on deep learning have been widely adopted in the industry. For example, in clothing, dining, and TV show recommendations, recommendation systems can utilize big data to suggest items that users might like based on their behavioral data. They can also optimize the next recommendation results based on whether users accept or reject the recommendations. Multi-task learning refers to handling multiple tasks simultaneously during the modeling process, such as user clicks and user orders, or user likes and viewing duration. There are correlations among multiple tasks, and compared to training on a single task only, multi-task learning can significantly improve the effectiveness of each task. In this work, we propose a novel multi-task learning framework, the Independent Grouped Information Expert (IGIE) model. The IGIE model consists of two identical Multi-gate Mixture-of-Experts (MMoE) structures, with independent inputs for each part. The upper expert layer processes information based on different initialized embeddings, and after the two independent MMoE structures complete their information processing, the results are concatenated and passed to different task towers at the upper level.
References
Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions[J]. IEEE transactions on knowledge and data engineering, 2005, 17(6): 734-749.
Su X, Khoshgoftaar T M. A survey of collaborative filtering techniques[J]. Advances in artificial intelligence, 2009, 2009.
Zhang S, Yao L, Sun A, et al. Deep learning based recommender system: A survey and new perspectives[J]. ACM computing surveys (CSUR), 2019, 52(1): 1-38.
Wang S, Hu L, Wang Y, et al. Sequential recommender systems: challenges, progress and prospects[J]. arXiv preprint arXiv:2001.04830, 2019.
Ruder S. An overview of multi-task learning in deep neural networks[J]. arXiv preprint arXiv:1706.05098, 2017.
Wang Y, Lam H T, Wong Y, et al. Multi-task deep recommender systems: A survey[J]. arXiv preprint arXiv:2302.03525, 2023.
Tang H, Liu J, Zhao M, et al. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations[C]//Proceedings of the 14th ACM Conference on Recommender Systems. 2020: 269-278.
Misra I, Shrivastava A, Gupta A, et al. Cross-stitch networks for multi-task learning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 3994-4003.
Long M, Cao Z, Wang J, et al. Learning multiple tasks with multilinear relationship networks[J]. Advances in neural information processing systems, 2017, 30.
Lu Y, Kumar A, Zhai S, et al. Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 5334-5343.
Ruder S, Bingel J, Augenstein I, et al. Sluice networks: Learning what to share between loosely related tasks[J]. arXiv preprint arXiv:1705.08142, 2017, 2: 1.
Ni Y, Ou D, Liu S, et al. Perceive your users in depth: Learning universal user representations from multiple e-commerce tasks[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018: 596-605.
Yang Y, Hospedales T. Deep multi-task representation learning: A tensor factorisation approach[J]. arXiv preprint arXiv:1605.06391, 2016.
Ma X, Zhao L, Huang G, et al. Entire space multi-task model: An effective approach for estimating post-click conversion rate[C]//The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 2018: 1137-1140.
Diemert E, Meynet J, Galland P, et al. Attribution modeling increases efficiency of bidding in display advertising[M]//Proceedings of the ADKDD'17. 2017: 1-6.
Li P, Li R, Da Q, et al. Improving multi-scenario learning to rank in e-commerce by exploiting task relationships in the label space[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020: 2605-2612.
Ma J, Zhao Z, Yi X, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts[C]//Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2018: 1930-1939.
Zhang W, Bao W, Liu X Y, et al. Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning[C]//Proceedings of The Web Conference 2020. 2020: 2775-2781.
Steck H. Training and testing of recommender systems on data missing not at random[C]//Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. 2010: 713-722.
Wang H, Chang T W, Liu T, et al. ESCM2: entire space counterfactual multi-task model for post-click conversion rate estimation[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022: 363-372.
Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
Jannach D, Manzoor A, Cai W, et al. A survey on conversational recommender systems[J]. ACM Computing Surveys (CSUR), 2021, 54(5): 1-36.
Chen X, Yao L, McAuley J, et al. A survey of deep reinforcement learning in recommender systems: A systematic review and future directions[J]. arXiv preprint arXiv:2109.03540, 2021.
Shi, Peng, Yulin Cui, Kangming Xu, Mingmei Zhang, and Lianhong Ding. 2019. "Data Consistency Theory and Case Study for Scientific Big Data" Information 10, no. 4: 137. https://doi.org/10.3390/info10040137.
Zhenghua Hu, Xianmei Wang, Kangming Xu, and Pu Dong. 2020. Real-time Target Tracking Based on PCANet-CSK Algorithm. In Proceedings of the 2019 3rd International Conference on Computer Science and Artificial Intelligence (CSAI '19). Association for Computing Machinery, New York, NY, USA, 343–346. https://doi.org/10.1145/3374587.3374607.
Khan M M, Ibrahim R, Ghani I. Cross domain recommender systems: A systematic literature review[J]. ACM Computing Surveys (CSUR), 2017, 50(3): 1-34.
Batmaz Z, Yurekli A, Bilge A, et al. A review on deep learning for recommender systems: challenges and remedies[J]. Artificial Intelligence Review, 2019, 52: 1-37.
Yao L, Wang X, Sheng Q Z, et al. Recommendations on the internet of things: Requirements, challenges, and directions[J]. IEEE Internet Computing, 2019, 23(3): 46-54.
Gao Y, Li Y F, Lin Y, et al. Deep learning on knowledge graph for recommender system: A survey[J]. arXiv preprint arXiv:2004.00387, 2020.
K. Xu, X. Wang, Z. Hu and Z. Zhang, "3D Face Recognition Based on Twin Neural Network Combining Deep Map and Texture," 2019 IEEE 19th International Conference on Communication Technology (ICCT), Xi'an, China, 2019, pp. 1665-1668, doi: 10.1109/ICCT46805.2019.8947113.