User Intent Prediction #15: Position 40. Everything Collapsed.
My model loved users who clicked 60 times. It didn’t know they were rage-clicking.
The funnel has 60 screens. I assumed: “The more screens a user sees, the better the prediction.” More data = Better model, right?
Wrong.
My model worked okay until Screen 39. Then it fell off a cliff.
The Cliff
I plotted prediction accuracy by “Current Screen Position.”
- Screen 1-10: Weak (barely better than guessing).
- Screen 20-30: Okay (starting to see patterns).
- Screen 39: Peak (best performance).
- Screen 40+: Catastrophe (worse than random).
Wait. Worse than random? That means the model is actively wrong. It’s predicting “Buy” when the user will “Quit,” and “Quit” when the user will “Buy.”
It’s not just failing. It’s confidently wrong.
The “Zombie User” Phenomenon
Why does it crash at Screen 40?
Because users who reach Screen 60 without buying aren’t “highly engaged.” They are lost.
They are clicking in circles. They are “Zombie Users.”
- They click “Back.”
- They click “Next.”
- They change their answers.
- They re-read the same screen 3 times.
To the model, this looks like “Activity.” To a human, this looks like “Confusion.”
Example:
- User A (Buyer): Screens 1 → 5 → 10 → 15 → Paywall → Buy. (20 clicks, 3 minutes).
- User B (Zombie): Screens 1 → 2 → 1 → 3 → 2 → 5 → 3 → 10 → 5 → 15 → 10 → 20 → 15 → … (60 clicks, 15 minutes, no purchase).
The model sees 60 events and thinks: “Wow! So much interaction! They must love us!” Reality: They are frustrated and about to rage-quit.
The Data Poison
Long sequences aren’t just noisy. They are toxic.
The model learned:
- “More clicks = More engagement = Higher intent.”
But in reality:
- Efficient users (20 clicks) → Buyers.
- Confused users (60 clicks) → Churners.
The model was rewarding confusion and punishing efficiency.
The Numbers:
- Users with 20-30 clicks: 8% conversion.
- Users with 40-50 clicks: 3% conversion.
- Users with 60+ clicks: 0.5% conversion.
More clicks = Less likely to buy. But the model thought the opposite.
The Lesson
More data can be toxic.
I thought a long sequence was a rich signal. It turned out to be a signal of failure.
The best buyers don’t click 60 times. They click 20 times and pay. Efficiency predicts revenue. Loitering predicts churn.
I was feeding the model “engagement metrics” when I should have been feeding it “friction metrics.”
This leads to how we fixed it with Max-So-Far Aggregation—stop looking at the length, start looking at the depth.