Menu
Since the advent of artificial intelligence, we’ve witnessed a surge in its applications across various domains and use-cases. Given its versatility and potential, we took on a project that explored the capabilities of AI within recommendation systems, particularly within the scope of a chatbot. Our aim was to create a system that, much like a salesperson, interacted with a customer to recommend products best suited for their needs.
We referenced the following dataset, a collection of monthly mobile data plans. For our particular use-case, the key fields we identified were the monthly price and mobile data requirements, with the goal to provide users with the top three recommendations.
We started off with a straightforward setup, tapping into the capabilities of GPT-3.5 Turbo to gather the essential data from users. Our solution was structured to have three layers of prompts for GPT:
The chatbot-driven approach, along with our custom-designed prompts, provide a seamless user experience, facilitating the user’s in their decision-making process for mobile plans. With the additional capability to address inquiries regarding the dataset, users can ask direct questions regarding plan features, enhancing the overall experience and streamlining the decision-making journey.
This snapshot of an ongoing conversation illustrates this sequential process.
The crux of our project lay in building an effective ranking system. Initially, we considered relying solely on GPT with the task of filtering out the top recommended plans. However, we quickly found that this approach fell short, with GPT’s variability introducing inconsistencies and unreliability into its selections. Additionally, with the small size of the dataset, GPT struggled especially in cases of misaligned data and budget requirements, failing to select plans that catered to both fields, instead sticking to one and giving skewed recommendations.
For example, when asked for unlimited data with an unreasonable budget of 300, the system overlooked plans within the user’s budget range, contradicting our intended outcome of providing diverse options.
Moreover, we encountered instances of hallucinations, where GPT would alter existing data entries. As an example, when asked to provide recommendations for a data requirement of 6GB, it modified the data plan of an entry from “unlimited” to “6GB”.
Though this initial approach proved nonoptimal, it provided valuable insights and set a benchmark for refinement.
Subsequently, we opted for a hybrid approach, dividing the ranking process between GPT and our own filtering mechanism. Using the JSON data extracted from user conversations, we filtered down the options into two smaller subsets: one tailored to data requirements and the other to budget constraints. These subsets were then handed over to GPT, alongside the user’s requirements, to filter the choices down to a final top three.
This strategy of creating separate subsets for data and budget not only addressed cases of misaligned requirements by offering diverse options, but also catered to scenarios where preferences aligned seamlessly, ensuring the best plan surfaced prominently in both filtered sets, thus facilitating an optimal selection process.
Given the small scale of our dataset, our simplistic approach yielded reasonably reliable outcomes, recommending suitable plans to users in most cases. The comparison table below illustrates the difference in performance between the two approaches we discussed. Notably, the GPT-only approach showed instances of hallucinations, marked in red. Additionally, when facing misaligned requirements, such as the second example, it faltered in selecting more budget-conscious options, diverging from the intended aim of presenting a diverse set of choices in such scenarios.
In projects featuring more expansive datasets, a more refined preliminary filtering system would likely be necessary, with a more comprehensive approach to filtering rather than the subset approach we opted for.
While our project showed some promising results within the confines of a limited dataset, it also shed light on the challenges inherent in AI-driven decision-making processes. Our exploration underscores GPT’s versatility in augmenting recommendation systems. While challenges persist, our iterative process unveils its potential in navigating nuanced user preferences. As AI continues to evolve, so too do our capabilities in crafting tailored experiences, paving the way for smarter, more intuitive interactions.