Nine signals checked every week, and a 3-to-1 retention gap the reporting could not see
A B2B2C digital health and wellness platform
The Brief
The client runs a B2B2C digital health and wellness platform with two surfaces: a consumer app, and a practitioner portal used by medical practitioners and fitness operators to manage their own clients. They already had product analytics and session replay in place.
What they did not have was confidence in either, or an easy way to see the state of the product for themselves.
The result was a rich data stack that leadership did not fully trust and could not readily use, which meant it was not driving decisions across the product. A stack nobody trusts is worse than no stack at all, because it still costs money and it still gets quoted in meetings.
What the reporting could not tell you
Four failure modes, each one quietly undermining the numbers leadership was reading.
| Symptom | What it actually meant |
|---|---|
| Numbers moved week to week | Nobody could be sure whether the movement was real or an artifact of the tracking |
| Metrics colored green while signaling problems | A rising error count showed as an improvement, because nobody had set the direction of the indicator |
| Tracking broke silently on key events | Trend lines corrected quietly, with no alert and no way to know which weeks to distrust |
| Stakeholders needed a cut for a meeting | They had to wait on someone to run it, then hope the result was reliable |
The Approach
Fracto’s premise is that a gain only counts if it shows up in the client’s own numbers, so the job was to make those numbers trustworthy, visible and usable for decisions, not just to file reports.
We adopted a validation-first discipline: no figure entered a dashboard or report until it was checked against the raw source, the math re-derived by hand, the date window confirmed, and the direction of every indicator verified against what “good” actually means. A falling error count is an improvement even when the dashboard paints it red, and we treated it that way.
We standardized on a single production dashboard as the source of truth, defined every user segment precisely so that populations were counted consistently everywhere, and committed in advance to reporting whatever the data showed rather than shopping for a flattering cut.
And we never handed over a problem without a client-actionable next step attached to it.
What we Did
Built a stakeholder-facing dashboard suite. A source-of-truth product dashboard plus feature-specific dashboards for a flagship AI feature and for the notification system. Stakeholders get a live, self-serve view of onboarding, engagement and retention at any time, instead of waiting on someone to run a query.
Produced a validated weekly performance report. Covering onboarding, access expiry, practitioner-driven deactivations and subscriptions across every user segment, with every figure validated against a screenshot before a word of narrative was written.
Ran a weekly System Health Check. Nine core signals, five user segments plus four flagship features, classified Healthy or Warning every week. Every problem it flags carries a recommended, client-actionable next step and a documented cause. No red cell ships on its own.
The weekly System Health Check. Nine core signals, each Warning carrying a cause and a next step.
Tracked notification reliability on two metrics, not one. Reach rate and funnel conversion for the two recurring push sends, measured against a 95% reach target. Artifact weeks are flagged rather than smoothed away, so the dip is labeled, not deleted.
Notification reach against the 95% target. Dual-metric tracking put numbers on a past reach collapse for the first time and separated a recurring off-schedule artifact from genuine trend movement.
Turned around ad-hoc cuts and targeted deep-dives. Beyond the standing reports, the data cuts stakeholders need ahead of their meetings, and targeted deep-dives whenever they have a question the standard reports cannot answer, from segment-level adoption to feature-specific match failures. Each answered in the client’s own data.
Paired every quantitative finding with session-replay review. Scoped strictly to the sessions actually watched, with no extrapolation from replay alone, then filed the product issues found as structured tickets the delivery team can act on directly.
Session-replay review. The event stream is read against the quantitative finding, with conclusions scoped strictly to the sessions actually watched.
Results in Detail
A rich but distrusted data stack became something leadership could see, trust and act on, with every figure reproducible in the client’s own dashboards.
Leadership could see the product for themselves. The dashboard suite turned onboarding, engagement and retention into a live, self-serve view rather than something that had to be requested and waited on.
Combined with the weekly report and on-demand data pulls, it fed directly into leadership’s own meetings and roadmap discussions, giving them a validated evidence base for those calls.
Continuous monitoring caught problems the week they emerged. Several silent tracking breaks were caught and isolated, including a core onboarding event that had stopped recording and a backend outage cluster in a flagship AI feature.
Each would otherwise have skewed trend lines unnoticed. Once identified, the affected events were excluded from denominators so the surrounding numbers stayed honest.
Notification delivery moved from assumed to measured. Tracking the two recurring sends quantified reach against the 95% target for the first time, put numbers on a past reach collapse that had never been quantified, and separated a recurring off-schedule artifact from genuine trend movement.
Questions that were previously unanswerable got answered. A deep-dive on a single embedded partner settled whether weak usage was a product problem or an adoption problem.
In that partner’s own data, app adoption sat at about 7% of active members, yet the members who did use the app retained far better than those who stayed in the browser: roughly 54% / 36% / 14% at months 1, 3 and 6, against roughly 17% / 11% / 6%. An association of about three to one, with feature engagement running 3 to 9 times higher in the app. Onboarding was not the bottleneck, with matured cohorts completing at about 64% and a median completion time near 2.2 minutes.
The partner deep-dive. App members retained at roughly three times the rate of mobile browser members, while app adoption sat at about 7% of active members. The bottleneck was adoption, not the product.
Findings turned into actionable tickets. Problems surfaced during session review were filed as structured tickets, including a mobile bug where a core list silently emptied itself mid-session, so what analysis found, the delivery team could ship against.
Stated as an association, not a cause
- The retention gap is framed as a directional recommendation rather than a causal claim, because part of it reflects engaged members self-selecting into the app.
- Saying so is the point. A three-to-one gap presented as proof of causation is the kind of number that gets a roadmap built on it and then quietly fails to reproduce.
Tools & technologies
What the build runs on
verified