Eval & Observability
Real telemetry from every model call.
FrictionLens scores app reviews across five sentiment dimensions using Gemini. This page shows the eval intuition layer behind that pipeline — Spearman correlation per dimension against a hand-labeled golden set, plus real cost, latency, and reliability metrics from production traffic.
Calls (30d)
0
Total model invocations
Spend (30d)
$0.00
Sum of input + output cost
p95 latency
0ms
Weighted by call volume
Error rate
0.0%
0 errors · 0 rate-limited
Cost by model
Total USD across last 30 days
No calls in the last 30 days.
Latency by prompt
p50 / p95 / p99 across last 30 days
No calls in the last 30 days.
Latest eval run
Spearman correlation per dimension vs. hand-labeled golden set
- Prompt
- review-v1
- Model
- gemini-2.5-flash
- n
- 19 reviews · golden vv1
- Date
- 2026-05-14
| Dimension | Spearman ρ | MAE (0–10 scale) | Verdict |
|---|---|---|---|
| love | 0.946 | 0.63 | strong |
| frustration | 0.951 | 0.95 | strong |
| loyalty | 0.971 | 0.68 | strong |
| momentum | 0.912 | 0.89 | strong |
| wom | 0.987 | 0.37 | strong |
Prompt registry
Deployed prompts with version, call volume, and last-deployed date
| Prompt ID | Name | Version | Calls (30d) | Deployed | Notes |
|---|---|---|---|---|---|
| report-v1 | report | v1 | 0 | 2026-05-13 | Initial aggregate Vibe Report synthesis prompt. |
| review-v1 | review | v1 | 0 | 2026-05-13 | Initial scoring rubric for single-review analysis. |
| batch-review-v1 | batch_review | v1 | 0 | 2026-05-13 | Same rubric as review-v1, applied across a batch of reviews. |
| report-v2 | report | v2 | 0 | 2026-05-13 | Adds verdict, the_one_thing, citations, confidence, wishlist/dealbreaker bucketing, vagueness rejection, de-duplication. |
| report-v3 | report | v3 | 0 | 2026-05-14 | Adds strict review-ID binding: cited_review_ids must use only the rNNN IDs supplied, never invented. Enables the click-to-see-receipts UI. |
| report-v4 | report | v4 | 0 | 2026-07-12 | Adds topic continuity: reuse the baseline analysis's friction/churn topic names verbatim so period-over-period deltas pair the same topic instead of naming drift. |
| report-v5 | report | v5 | 0 | 2026-07-12 | Topic continuity v2: v4's buried instruction was ignored live (0/10 topics reused). Moves the topic list to the TOP of the user prompt, adds a reinforcement at the produce-list decision point, and hardens the system rule. |
Recent traces
Last 20 model calls (anonymized — no user identifiers)
| When | Prompt | Model | Tokens (in / out) | Latency | Cost | Source | Status |
|---|---|---|---|---|---|---|---|
| 58d ago | batch-review-v1 | gemini-3.1-flash-lite | 6956 / 7856 | 17360ms | $0.0135 | — | success |
| 58d ago | batch-review-v1 | gemini-3.1-flash-lite | 7890 / 7286 | 17114ms | $0.0129 | — | success |
| 58d ago | batch-review-v1 | gemini-3.1-flash-lite | 865 / 346 | 1398ms | $0.0007 | — | success |
| 58d ago | review-v1 | gemini-3.1-flash-lite | 773 / 189 | 1208ms | $0.0005 | — | success |
| 58d ago | batch-review-v1 | gemini-3.1-flash-lite | 829 / 224 | 1106ms | $0.0005 | — | success |
| 58d ago | review-v1 | gemini-3.1-flash-lite | 773 / 189 | 1349ms | $0.0005 | — | success |
| 59d ago | report-v5 | gemini-2.5-flash-lite | 5179 / 2004 | 11890ms | $0.0013 | report_api | success |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 4165 / 6555 | 51199ms | $0.0030 | — | success |
| 59d ago | report-v5 | gemini-2.5-flash-lite | — / — | 9247ms | — | pipeline | rate_limit |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 4905 / 6809 | 20506ms | $0.0032 | pipeline | success |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 10046 / 13743 | 39737ms | $0.0065 | pipeline | success |
| 59d ago | report-v5 | gemini-2.5-flash-lite | — / — | 12254ms | — | pipeline | rate_limit |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 7321 / 10097 | 26626ms | $0.0048 | pipeline | success |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 9527 / 13209 | 32112ms | $0.0062 | pipeline | success |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | — / — | 42461ms | — | pipeline | error |
| 59d ago | report-v5 | gemini-2.5-flash-lite | — / — | 11118ms | — | pipeline | rate_limit |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 7321 / 10130 | 31789ms | $0.0048 | pipeline | success |
| 59d ago | batch-review-v1 | gemini-2.5-flash-lite | 8556 / 13645 | 38311ms | $0.0063 | pipeline | success |
| 60d ago | batch-review-v1 | gemini-2.5-flash-lite | — / — | 6326ms | — | pipeline | rate_limit |
| 60d ago | batch-review-v1 | gemini-2.5-flash-lite | — / — | 30657ms | — | pipeline | rate_limit |