In February 2024, a Waymo robotaxi in San Francisco drove around a roundabout 17 times before engineers rescued it remotely. Nothing was broken. The car simply could not find a move it was allowed to make.
That is the problem this project is about.
Why this matters
A self-driving car can handle most of a trip. What it handles badly is the rare moment when the road stops making sense: a plastic bag in the lane, a broken-down car ahead, barricades across both lanes. The planner looks for a legal path, finds none, and does the safest thing it knows — it stops. And then it keeps stopping.
Today there are two ways out, and both have a hole in them:
- Call a remote operator. Someone, somewhere, takes over. It works, but it costs money to keep people on standby, and the car waits while one is found.
- Let the passenger drive. Only possible if the passenger can drive. That rules out older riders, riders with disabilities, and anyone without a licence — the very people who gain the most from a car that drives itself.
What is missing is a third option: the car gets itself out, or the passenger helps with a sentence instead of the steering wheel.
What we built
StuckSolver is a language model wired into an existing self-driving stack as an add-on. Nothing inside the car’s own perception, planning or control code changes. It reads what the car already knows and hands back a suggestion — and only when the car is actually stuck.
The reasoning runs in three plain steps:
- Look. From the front camera and the object list: what is ahead, how far, how fast, and what the traffic lights and signs say.
- Diagnose. Is the car genuinely stuck, or just correctly waiting at a red light with pedestrians crossing? If it is stuck, why?
- Decide. Choose one move the car is already allowed to make — stop, keep lane, follow, change lane — plus a point to restart the route from if a detour is needed.
Two design choices did most of the work:
- It stays out of the way. It is silent while the car drives normally and wakes only after the car has been under 5 km/h for more than a second. So it costs nothing in the 99% of driving that is fine, and it never fights the planner.
- It only picks from legal moves. The model does not invent a trajectory. It chooses from the behaviours the car’s own planner currently offers, so whatever it suggests, the car already knows how to execute safely.
And because it speaks language, the passenger can join in: “It’s just a bag — you can drive over it.” StuckSolver turns that sentence into a lane change the car can actually perform. No driving licence required.
Does it work?
We tested it in CARLA on Bench2Drive, a benchmark of 220 routes, each containing one hard corner case. Two numbers matter: Driving Score (how well the route was driven, counting violations) and Success Rate (how often the route was finished on time and clean).
Reading it in plain terms:
- Bolting StuckSolver onto a plain rule-based agent lifted its driving score by about a third, and doubled how often it finished a route cleanly.
- Add a sentence of passenger guidance where it helps — which happened on roughly 15% of the routes — and the pair lands level with Raw2Drive, the strongest end-to-end driving model reported on this benchmark.
That last point is the one we care about. A simple, interpretable, rule-based car plus a language model reached the performance of a heavyweight learned system, without retraining anything.
An add-on, not a rewrite
It connects through ordinary interfaces and changes nothing inside the car's own modules — so an existing fleet could adopt it without redesigning its stack.
Help without a steering wheel
A passenger who cannot drive can still get the car moving, by saying what they see. The model checks the suggestion against traffic rules before acting on it.
It explains itself
Every decision comes with the reason behind it in plain sentences — much easier to audit than a number coming out of a neural network.
A dial, not a switch
Change when it wakes up and you slide between "cheap rule-based car" and "language model involved almost continuously", trading compute for adaptability.
What it does not solve
- It is slow. About 2.8 seconds per query. Fine for a car that is already standing still; useless for anything time-critical. A distilled, faster model is the next step.
- It is simulation. Everything here is CARLA. Real roads, real sensors, real latency are still ahead.
- Passenger guidance is a demonstration, not a product. We gave simple instructions where they obviously helped. Knowing when to ask a human, and how much to trust the answer, is its own research problem.
The hard part was never the driving. It was knowing that a plastic bag is only a plastic bag.
Joint work with Qianwen Li at the University of Georgia. Published at the IEEE Intelligent Vehicles Symposium (IV 2026), Detroit, June 2026.
Read the paper on IEEE Xplore