What we measure after launch, and how it picks the next release

Before launch, every decision is a guess. After launch, most of them can be checked. The products we run improve in steps because each step starts from something we measured.

Here is what we measure, and how it turns into the next release.

Section-level events, not page views

A page view says someone arrived. It does not say what they did. We track each section of each page: did the user reach it, did they interact, did they leave from there.

This shows the drop-off point inside a page. On one consumer reference tool, it showed that many first-time visitors did not understand the terminology used in the main action. The fix was not a feature. It was a rewritten introduction that explained what the tool checks and when to use it.

Events should describe meaningful behaviour rather than every possible click. “Started comparison”, “received result” and “copied summary” can answer a product question. Tracking every accordion and decorative control produces volume without clarity.

Define events alongside the feature. Record what question each one is meant to answer, which properties are allowed and how long the data will be kept. This prevents the analytics layer from becoming an undocumented second product.

Feedback that is stored, not just sent

A “contact us” link produces email nobody reads twice. We store feedback in a database, tied to the page and the item it was about, so we can search it and group it.

Stored feedback needs protection: rate limits, daily caps, duplicate detection and basic spam checks. Without them, the signal disappears under noise within weeks.

Ask focused questions. “What do you think?” produces broad answers that are hard to group. “What was missing from this result?” or “What stopped you completing this step?” connects the response to a decision the team can make.

Keep the context that helps interpretation—the page, feature and result type—but avoid collecting identity by default. People are often more candid when feedback does not become a sales conversation. Provide contact details as an explicit option when a reply would help.

Review feedback on a schedule and tag it consistently. A database is not useful merely because messages are stored there. Group repeated friction, separate requests from underlying problems and connect each accepted change back to the evidence that justified it.

The slow path and the failed path

For any core action, we measure two things: how long it takes, and how often it fails. A scan that takes 240 seconds and a scan that returns nothing are both failures, and they need different fixes.

Timing data decided the multi-pass scanner and the early-exit rule. Failure data decided the clear messages for denied microphone access and offline devices.

Measure recovery as well as failure. Did the person retry, choose a manual route or leave? A good error message is successful only if it helps someone continue. Recovery events often point to a cheaper improvement than eliminating every technical failure.

Performance distributions matter more than averages. A median completion time can look healthy while people on slower devices wait four times longer. Review the slowest meaningful group and test on hardware and networks that resemble the real audience.

Retention needs a natural window

Not every useful product should be opened daily. A planning tool may be valuable several times during one month and then unnecessary for a year. A reference tool may be used only when a particular decision appears. Choose a retention window that matches the job instead of borrowing a daily-active-user target from another category.

Look for return to the task, not return to the homepage. Did someone reopen a saved plan, repeat a check, update a comparison or share an output? These behaviours reveal continuing value more accurately than another anonymous page view.

Turning data into a release

Every few weeks we ask three questions:

1. Where do the most people stop?
2. What do the stored comments say about that place?
3. What is the smallest change that could move it?

The answer to the third question is the next release. It is usually smaller than the team expected.

Write the release hypothesis in one sentence: “If we clarify this term before the main action, more first-time visitors will complete it.” Record the current measure, expected direction and review date. The purpose is not to predict perfectly. It is to make the reason for the work visible.

Release one meaningful change when possible. Bundling copy, navigation and processing changes may move the metric, but the team will not know why. Small releases create cleaner evidence and are easier to reverse.

After review, keep the change, adjust it or remove it. Do not leave every experiment in the product. Accumulated experiments become the next source of friction.

What we do not measure

We do not track individuals across sites. We do not run analytics without consent where consent is required. A product that respects its users keeps the data it needs and no more. That constraint has never stopped us from finding the next problem.

We also avoid vanity metrics that cannot change a decision. Total page views may be useful for reporting reach, but they do not automatically belong in the release process. If the team cannot say what it would do differently when a number moves, that number does not need a prominent dashboard.

Keep a measurement rhythm

Dashboards do not create learning by themselves. Establish a short recurring review with product, design and engineering in the same conversation. Look at a small set of journeys, read representative feedback and choose one problem to investigate further.

Maintain a decision log: what changed, which evidence supported it and what happened afterwards. Over time, this becomes more valuable than a collection of screenshots because it shows how the team learns and where its assumptions were wrong.

Measurement is not a dashboard. It is the habit of asking what the product should do next, and having an answer that is not a guess.