Incident file · Security
Chevrolet of Watsonville chatbot "$1 Tahoe" (2023)
prompt-injection induced "binding offer" for $1, viral but unenforced
The system
A Chevrolet dealership deployed a third-party chatbot (built by Fullpath, "powered by ChatGPT") for sales inquiries on its website. The chatbot had no system/user privilege separation and no output-side check on what it committed to.
What was observed
A user, Chris Bakke, sent the chatbot a prompt designed to change its behavior, then asked it to agree to sell a 2024 Chevy Tahoe for $1.
Stop and think
- If a customer-facing chatbot has no separation between "instructions the company gave it" and "text the customer typed", what can the customer make it say?
- What output-side check, applied after generation and before the message is sent, would catch "this response commits to a price" regardless of how the model was talked into writing it?
- Is scoping the bot to only answer from an approved FAQ a complete fix, a partial fix, or no fix at all for this failure mode?
Answer the questions above before you read on: what failed, why, and what would have stopped it.