A Study on Leveraging Generative Artificial Intelligence and Text Clustering to Support Vendors

Abstract

This research is an initiative to discover how generative artificial intelligence (AI) tools can improve Amazon Last Mile’s feedback systems to enhance the Delivery Service Partner experience. Our specific focus is on the effectiveness of clustering algorithms like DBSCAN and K-Means for grouping text feedback based on semantic similarity and on the employment of retrieval augmented generation (RAG) for extracting actionable insights. Our findings indicate a relative effectiveness of K-Means over DBSCAN in clustering feedback, but the overall effectiveness is moderate, which necessitates the need for human verification to counter potential model hallucinations. Additionally the use of RAG with Claude 2.1 demonstrated promise in answering domain-specific questions in spite of limitations related to text-only input.

We propose future emphasis on the integration of AI in current listening mechanisms to offer concise, actionable recommendations for program leaders. This research also recommends continued exploration in embedding models and RAG framework to enhance feedback quality and information retrieval. The potential to integrate generative AI tools within Amazon Last Mile represents an underexplored opportunity for significant enhancements in efficiency, accuracy, and overall partnership satisfaction.