Podqi Study: Perplexity, ChatGPT, and Gemini Recommend Risky Shopping Sites in 15% of Queries

A study by IP protection firm Podqi found that AI chatbots from OpenAI, Google, and Perplexity recommended fake or risky retail websites in shopping queries, raising concerns for consumers using these tools for purchasing decisions. For broader context, explore our Top 100 AI Tools.

Study Details and Methodology

Podqi's research involved prompting ChatGPT, Gemini, and Perplexity with shopping questions concerning 151 different brands. The study meticulously analyzed the chatbots' responses to identify instances where they cited or displayed links to potentially harmful websites. These risky sites encompassed various categories, including outright fake websites, sellers of knockoff products, expired brand domains that had been reregistered by third parties, and dead domains that could be acquired and repurposed maliciously.

Performance Comparison of AI Chatbots

The study provided a clear comparison of how each AI chatbot performed:

• Perplexity: This AI showed the highest rate of recommending risky sites, appearing in 15 percent of its answers.
• ChatGPT: OpenAI's chatbot surfaced risky sites in 8 percent of its responses. Notably, ChatGPT did include warnings about risky sites in 24 percent of the instances where it provided such links.
• Gemini: Google's AI demonstrated the lowest incidence of risky recommendations, with these sites appearing in 4 percent of its answers.

The findings indicate a varying degree of susceptibility among the tested AI models to generate or reference unreliable shopping destinations.

Specific Risks and Brand Vulnerabilities

The research also uncovered specific patterns in the types of queries that led to risky recommendations. Queries about smaller brands were approximately twice as likely to trigger risky sites compared to those for larger, more established companies. Furthermore, questions related to regional brand websites produced the most concerning results, with chatbots sometimes inventing regional sites for brands that do not actually operate them.

The types of risky sites identified included:

• Fake websites designed to mimic legitimate retailers.
• Sellers offering counterfeit or unauthorized knockoff products.
• Expired brand domains that have been reregistered by new, potentially malicious, owners.
• Dead domains that could be purchased and used for deceptive purposes.

Industry Response and Ongoing Efforts

In response to these challenges, Google, OpenAI, and Perplexity have each stated that they are actively working to enhance their safeguards and improve the overall quality of information provided by their AI models. This commitment suggests an ongoing effort to address the vulnerabilities highlighted by Podqi's study and to mitigate the risks consumers face when using AI tools for shopping-related inquiries.

Conclusion

The Podqi study serves as a critical reminder of the need for caution when using AI chatbots for online shopping. While these tools offer convenience, the potential for encountering fake or risky retail websites remains a significant concern. Consumers should exercise due diligence, verify website legitimacy independently, and be aware that even leading AI models can inadvertently direct them to unreliable sources. The stated commitments from Google, OpenAI, and Perplexity to improve their systems are a positive step, but continuous vigilance from users is essential.

Sources

• GitHub - RumenDamyanov/npm-chatbot: A powerful, flexible, and type-safe AI chatbot library for TypeScript/JavaScript applications with support for multiple AI providers (OpenAI, Anthropic, Google AI, and more). · GitHub
• AI chatbots are recommending fake retail websites, study finds | The Verge
• AI chatbots are distorting news stories, BBC finds | The Verge
• Podqi Inc - Company Profile and News