User Intent Prediction #18: The More They Clicked, The Less Was Clear.
More clicks should mean more signal.
That’s the intuition behind every “real-time intent” system.
User keeps moving → model keeps learning → probability gets sharper.
In my funnel, the opposite happened.
The deeper users went, the less was clear.
The false promise
I treated the user journey like a clean time series:
- Every click is a new observation
- More observations reduce uncertainty
So I built models that produce a probability on every step.
Current probability.
“What’s the chance to buy right now?”
It felt logical.
Then I looked at the funnel by position.
The cliff
Long sequences didn’t improve predictions.
They destabilized them.
Beyond position 40, instability jumped 4.7×:
- Before: 0.0076
- After: 0.0356
When the model becomes 4.7× more unstable exactly when users have clicked a lot…
It’s not learning intent.
It’s reacting to chaos.
Why long sequences are poison
The user who clicks 60 times is not necessarily “high intent.”
Often he’s lost.
He scrolls.
He goes back.
He changes answers.
He re-reads.
To a model, that looks like engagement.
To reality, it’s friction.
And friction isn’t purchase intent.
Friction is confusion.
Confusion predicts churn.
The zombie user phenomenon
Here’s what a 60-click user looks like:
Screen sequence: 1 → 2 → 1 → 3 → 2 → 5 → 3 → 10 → 5 → 15 → 10 → 20 → 15 → …
To the model: “Wow! So much interaction!”
To a human: “This person is lost.”
Compare to a buyer:
Screen sequence: 1 → 5 → 10 → 15 → Paywall → Buy (20 clicks, 3 minutes)
Efficient.
Decisive.
Short.
The model learned: “More clicks = More engagement = Higher intent.”
But in reality:
- Efficient users (20 clicks) → Buyers
- Confused users (60 clicks) → Churners
The model was rewarding confusion and punishing efficiency.
The humiliating fix
I stopped trusting the current value.
Instead of asking:
“What’s the probability now?”
I asked:
“What was the highest probability at any point so far?”
Max-so-far aggregation.
Peak signal.
One number.
Not a new architecture.
Not attention.
Not more training.
A max.
And it worked.
Across the journey, max-so-far beat current/final values by ~4.3% AUC (average).
The improvement wasn’t evenly distributed.
It barely mattered early.
It mattered a lot late:
| Position Range | Current Value AUC | Max-So-Far AUC | Improvement |
|---|---|---|---|
| 1–10 (Early) | 0.45–0.54 | 0.45–0.52 | +0–2% |
| 11–38 (Mid) | 0.51–0.53 | 0.51–0.52 | +1–2% |
| 39–60 (Late) | 0.48–0.50 | 0.48–0.49 | +1–2% |
| 60+ (Very Late) | 0.35–0.48 | 0.44–0.57 | +10–15% |
That last row is the punchline.
When users click forever, “right now” becomes meaningless.
The peak becomes the only honest signal.
Why peak matters
Because the buying moment often happens earlier than the purchase.
The purchase is delayed.
But the decision spike isn’t.
Users decide around Screen 15-20.
Then they browse.
Then they hesitate.
Then they either buy or quit.
If you only look at the final click, you miss the moment.
Max-so-far captures that moment.
The lesson
The more they clicked, the less was clear.
Not because the model couldn’t learn.
Because the funnel isn’t a clean sequence.
It’s a fight between curiosity and doubt.
And long sequences are full of doubt.
I thought I needed better sequence models.
Turns out I needed to stop trusting sequences.
Next problem: even when the model ranked users correctly, it lied about probabilities. Calibration Fix.