Measuring and Improving AI Performance in Conversational Shopping Assistants |
||
 |
 |
|
| © 2024 by IJCTT Journal | ||
| Volume-72 Issue-11 |
||
| Year of Publication : 2024 | ||
| Authors : Abhai Pratap Singh, Prerna Kaul | ||
| DOI : 10.14445/22312803/IJCTT-V72I11P126 | ||
How to Cite?
Abhai Pratap Singh, Prerna Kaul, "Measuring and Improving AI Performance in Conversational Shopping Assistants," International Journal of Computer Trends and Technology, vol. 72, no. 11, pp. 241-247, 2024. Crossref, https://doi.org/10.14445/22312803/IJCTT-V72I11P126
Abstract
Generative AI has transformed the domain of conversational AI, opening doors for a new breed of e-commerce shopping assistants. Developments in Generative AI have the potential to improve customer experience through more natural, dynamic and context-aware dialog exchange. However, the current implementations have critical flaws — limited access to real-time data and not enough contextual knowledge, resulting in difficulties in attaining trust and satisfaction from the user. Addressing these gaps is key to unlocking the potential of generative AI in building seamless shopping experiences. Using the Natural Conversation Framework (NCF), this paper evaluates the performance gaps and recommends how generative AI can create world-class shopping assistants. By combining conversational UX principles with a holistic measurement framework, this study provides a structured way to improve reliability, personalization, and conversational depth. This study incorporates a blend of technical and user-centric metrics, such as product accuracy, latency, user engagement, and satisfaction; this provides us with a holistic view of a conversation AI system’s performance. Beyond mapping challenges, this study elaborates on the path towards scalable human-centered conversational agents. This study also illustrates how to design shopping assistants with interaction patterns that are scaffolding for a centralized technical architecture to drive long-term engagement and trust. This work has practical implications for researchers and builders aspiring to harness generative AI for e-commerce applications.
Keywords
Conversational AI, E-commerce, Engagement metrics, Generative AI, Shopping assistants.
Reference
[1] Ayse Pinar Saygin, and Ilyas Cicekli, “Pragmatics in Human-Computer Conversations,” Journal of Pragmatics, vol. 34, no. 3, pp. 227 258, 2002.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Harold Garfinkel, Studies in Ethnomethodology, 3rd ed., Social Theory Re-Wired, pp. 1-9, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Robert J. Moore, Raphael Arar, Conversational UX Design: A Practitioner's Guide to the Natural Conversation Framework, ACM Books, Association for Computing Machinery, New York, NY, United States, pp. 1-320, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Emanuel A. Schegloff, Gail Jefferson, and Harvey Sacks, The Preference for Self-Correction in the Organization of Repair in Conversation,” Language, Linguistic Society of America, vol. 53, no. 2, pp. 361-382, 1977.
[CrossRef] [Google Scholar] [Publisher Link]
[5] A. M. Turing, “Computing Machinery and Intelligence,” Mind, vol. 59, no. 236, pp. 433-460, 1950.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Joseph Weizenbaum, “ELIZA-A Computer Program for The Study of Natural Language Communication Between Man and Machine” Communications of the ACM, vol. 9, no. 1, pp. 36-45, 1966.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Ludwig Wittgenstein, Philosophical Investigations: The German Text, with a Revised English Translation 50th Anniversary Commemorative Edition, 3thed., Wiley, pp. 1-464, 1991.
[Google Scholar] [Publisher Link]
[8] Xinmeng Song, and Ting Xiong, “A Survey of Published Literature on Conversational Artificial Intelligence,” 2021 7th International Conference on Information Management (ICIM), London, United Kingdom, pp. 113-117, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Xusen Cheng et al., “Exploring Consumers Response to Text-Based Chatbots in E-commerce: The Moderating Role of Task Complexity and Chatbot Disclosure,” arXiv, pp. 1-22, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Sina Mohseni, Niloofar Zarei, and Eric D. Ragan, “A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems,” ACM Transactions on Interactive Intelligent Systems, vol. 11, no. 3-4, pp. 1-45, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Harvey Sacks, Emanuel A. Schegloff and Gail Jefferson, A Simplest Systematics for the Organization of Turn-Taking for Conversation, Language, Linguistic Society of America, vol. 50, no. 4, pp. 696-735, 1974.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Harold Garfinkel, Studies in Ethnomethodology, Prentice-Hall, Englewood Cliffs, New Jersey, pp. 1-288, 1967.
[Google Scholar] [Publisher Link]
[13] Emanuel A. Schegloff, The Relevance of Repair to Syntax-for-Conversation, Syntax and Semantics, Discourse and Syntax, New York: Academic Press vol. 12, pp. 261-286, 1979.
[Google Scholar] [Publisher Link]
[14] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean, “Distilling the Knowledge in a Neural Network,” Arxiv, pp. 1-9, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Emilia Lesiak et al., “Digital Assistant in a Point of Sales,” arXiv, pp. 1-15, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Four Conversational AI Transforms Digital Experiences, Grid Dynamic, [Online]. Available: https://www.griddynamics.com/blog/conversational-ai-ecommerce
[17] FAN Min et al., “Research on Users' Trust of Chatbots Driven by AI: An Empirical Analysis Based on System Factors and User Characteristics,” 2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE), Guangzhou, China, vol. 66, no. 3, pp. 215-228, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Salla Syv€anen, and Chiara Valentini, “Conversational Agents in Online Organization–Stakeholder Interactions: A State-of-the-Art analysis and Implications for Further Research,” Journal of Communication Management, vol. 24, no. 4, pp. 339-362, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[19] Arshit Gupta et al., “CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots,” arXiv, pp. 1-7, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[20] Rodrigo Bavaresco et al., “Conversational Agents in Business: A Systematic Literature Review and Future Research Directions,” Computer Science Review, vol. 36, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Justina Sidlauskiene, Yannick Joye, and Vilte Auruskeviciene, “AI-based Chatbots in Conversational Commerce and Their Effects on Product and Price Perceptions,” Electronic Markets, vol. 33, pp. 1-21, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[22] Vai Rawool, Pantea Foroudi, and Maria Palazzo, “AI-Powered Voice Assistants: Developing A Framework for Building Consumer Trust and Fostering Brand Loyalty,” Electronic Commerce Research, pp. 1-33, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Mateusz Dubiel, Sylvain Daronnat, and Luis A. Leiva, “Conversational Agents Trust Calibration: A User-Centred Perspective to Design,” Proceedings of the 4th Conference on Conversational User Interfaces (CUI 2022), Glasgow United Kingdom, pp. 1-6, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[24] S. Jayakrishnan, “Artificial Intelligence (AI) in Retailing-A Systematic Review and Research Agenda,” Contemporary Research in Management, vol. 65, pp. 1-14, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Podsakoff, et al., “Common Method Biases in Behavioral Research: A Critical Review of the Literature and Recommended Remedies,” Journal of Applied Psychology, vol. 88, no. 5, pp. 879-903, 2003.
[CrossRef] [Google Scholar] [Publisher Link]
[26] Sea-Jin Chang, Arjen van Witteloostuijn, and Lorraine Eden, “From the Editors: Common Method Variance in International Business Research,” Journal of International Business Studies, vol. 41, no. 2, pp. 178-184, 2010.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Hettie A. Richardson, Marcia J. Simmering, and Michael C. Sturman, “A Tale of Three Perspectives: Examining Post Hoc Statistical Techniques for Detection and Correction of Common Method Variance,” Organizational Research Methods, vol. 12, no. 4, pp. 762-800, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[28] J. Williams Larry, K. Brown Barbara, “Method Variance in Organizational Behavior and Human Resources Research: Effects on Correlations, Path Coefficients, and Hypothesis Testing,” Organizational Behavior and Human Decision Processes, vol. 57, no. 2, pp. 185-209, 1994.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Ned Kock, “Common Method Bias in PLS-SEM: A Full Collinearity Assessment Approach,” International Journal of e-Collaboration (IJEC), vol. 11, no. 4, pp. 1-10, 2015.
[CrossRef] [Google Scholar] [Publisher Link]