Serverless AI-Powered Recommendation Engine with AWS Lambda and SageMaker |
||
 |
 |
|
| © 2024 by IJCTT Journal | ||
| Volume-72 Issue-12 |
||
| Year of Publication : 2024 | ||
| Authors : Joyanta Banerjee, Soumya Barman | ||
| DOI : 10.14445/22312803/IJCTT-V72I12P119 | ||
How to Cite?
Joyanta Banerjee, Soumya Barman, "Serverless AI-Powered Recommendation Engine with AWS Lambda and SageMaker," International Journal of Computer Trends and Technology, vol. 72, no. 12, pp. 153-163, 2024. Crossref, https://doi.org/10.14445/22312803/IJCTT-V72I12P119
Abstract
The opportunities for e-commerce, entertainment, and many other services, which can be provided through the internet, have stimulated the growing need for recommendation systems to improve the experience of users. Such recommendation systems are based on complex matrices and need substantial equipment with high maintenance costs; therefore, they are hindered by scalability and performance constraints. Aims: We want to show how organizations could use serverless computing and leverage the exponential development of artificial intelligence to manage the scalability of effective recommendation systems with ease of deployment and usually low operating costs. Study Design: This paper describes the architecture and development of a serverless recommendation system for an e-commerce application based on AWS Lambda and SageMaker. The potential of the serverless to reduce costs, scale automatically, and be deployed and maintained easily is also investigated. Furthermore, we incorporate Amazon SageMaker for training, deploying, and managing machine learning models behind the recommendation engine. Place and Duration of Study: Organizations across various industries have implemented this approach in 2023 and 2024. Methodology: Collaborative, content-based filtering and the hybrid approach are employed in the recommendation process, and the results are generated in real-time. The complete application is built using the serverless computing model, in which AWS Lambda runs simple code in response to events or user interactions. In contrast, Amazon Sage Maker is used to train the models and make predictions. Exposing APIs is done with AWS API Gateway; storing users’ data is done with Amazon DynamoDB, while the model artifacts and the big data are stored in Amazon S3. Results: This architecture helps to avoid provisioning and managing servers, which makes the operation less complex. In this paper, we will describe all stages of the work, from data preprocessing to the generation of recommendations. Conclusion: The results thus demonstrate the exceptional scalability and responsiveness of the recommendation engine, capable of accommodating users’ real-time needs with trivial time delay.
Keywords
Serverless computing, AWS lambda, Amazon sagemaker, Machine Learning, Collaborative filtering, Content-based filtering, Scalability.
Reference
[1] Badrul Sarwar et al., “Item-Based Collaborative Filtering Recommendation Algorithms,” Proceedings of the 10th International Conference on World Wide Web, Hong Kong Hong Kong, pp. 285-295, 2001.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Michael J. Pazzani, and Daniel Billsus, Content-Based Recommendation Systems, The Adaptive Web, Springer, Berlin, Heidelberg, pp. 325-341, 2007.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Robin Burke, “Hybrid Recommender Systems: Survey and Experiments,” User Modeling and User-Adapted Interaction, vol. 12, pp. 331-370, 2002.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Congying Guan et al., “Apparel Recommendation System Evolution: An Empirical Review,” International Journal of Clothing Science and Technology, vol. 28, no. 6, pp. 854-879, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Yi-Cheng Chen, and Yen-Liang Chen, “A Novel Virtual-communicated Evolution Learning Recommendation,” Industrial Management & Data Systems, vol. 124, no. 1, pp. 416-441, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Ta Phuong Bac, Minh Ngoc Tran, and YoungHan Kim, “Serverless Computing Approach for Deploying Machine Learning Applications in Edge Layer,” 2022 International Conference on Information Networking, Jeju-si, Korea, Republic of, pp. 396-401, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Peter Elger, and Eoin Shanaghy, AI as a Service: Serverless Machine Learning with AWS, Manning, pp. 1-328, 2020.
[Google Scholar] [Publisher Link]
[8] Amine Barrak, Fabio Petrillo, and Fehmi Jaafar, “Serverless on Machine Learning: A Systematic Mapping Study,” IEEE Access, vol. 10, pp. 99337-99352, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Meng Yu et al., “Review of Recommendation System,” Journal of Computer Applications, vol. 42, no. 6, 1898.
[Google Scholar]
[10] Denis Sinani, and Nuhi Besimi, “Application of Machine Learning in AWS for Manufacturing Companies,” Master’s Thesis, South East European University, pp. 1-88, 2023.
[Google Scholar]
[11] Nathalie Rauschmayr et al., “Amazon Sagemaker Debugger: A System for Real-time Insights into Machine Learning Model Training,” Proceedings of Machine Learning and Systems, vol. 3, 2021.
[Google Scholar] [Publisher Link]
[12] Jie Ding, Vahid Tarokh, and Yuhong Yang, “Model Selection Techniques: An Overview,” IEEE Signal Processing Magazine, vol. 35, no. 6, pp. 16-34, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[13] William Finnoff, Ferdinand Hergert, and Hans Georg Zimmermann, “Improving Model Selection by Nonconvergent Methods,” Neural Networks, vol. 6, no. 6, pp. 771-783, 1993.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Gianpaolo Cugola, and Alessandro Margara, “Processing Flows of Information: From Data Stream to Complex Event Processing,” ACM Computing Surveys (CSUR), vol. 44, no. 3, pp. 1-62, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Chris Fregly, and Antje Barth, Data Science on AWS, O'Reilly Media, Incorporated, pp. 1-90, 2021.
[Google Scholar] [Publisher Link]
[16] Peter Sbarski, and Sam Kroonenburg, Serverless Architectures on AWS: With Examples Using AWS Lambda, Manning, pp. 1-376, 2017.
[Google Scholar] [Publisher Link]
[17] Vijay Panwar, “Leveraging Aws Apis For Database Scalability And Flexibility: A Case Study Approach,” International Journal of Engineering Applied Sciences and Technology, vol. 8, no. 11, pp. 44-52, 2024.
[Google Scholar] [Publisher Link]
[18] S. Andreozzi, L. Magnoni, and R. Zappi, “Towards the Integration of StoRM on Amazon's Simple Storage Service (S3),” Journal of Physics: Conference Series, vol. 119, pp. 1-9, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Markus Lanthaler, and Christian Gütl, “Seamless Integration of Restful Services into the Web of Data,” Advances in Multimedia, vol. 2012, pp. 1-14, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Abhishek Mishra, Machine Learning in the AWS Cloud: Add Intelligence to Applications with Amazon Sagemaker and Amazon Rekognition, Wiley, pp. 1-528, 2019.
[Google Scholar] [Publisher Link]