Architected Voyager, a company-level product-intelligence framework that correlates customer voice, usage, and releases on demand to trace customer pain points and better understand the business impact behind them.

Executive summary

I designed, built, and own Voyager, an AI agent that gives LaunchDarkly's product teams and executives a way to track UX and business outcomes, as well as ask questions to conduct research. Because our metric data lives in separate tools, Voyager pulls customer feedback, usage, and release history into one short report, with the answer first and every number tied to its source. I took it from a five-week pilot to a project sponsored by the VP of Product Operations and the Head of UX.

The problem

Leadership started holding Strategy and Execution Reviews every two weeks, and UX got its own section in them. We needed numbers we could show executives and defend when they pushed back. We had plenty of data, but most of it wasn't reliable enough to use.

• Everything lived in a different tool. Customer feedback was in Enterpret, usage in Mixpanel and Snowflake, and releases in GitHub. Each tool named the same product area differently, and none of them lined up by date.
• Connecting them by hand took too long. Teams would do it for a big decision. For everything else they went on memory.
• Revenue data stopped at the feedback. Enterpret can show that a request came from a roughly $2M account. Nobody could connect that request to the usage number it described or the release that addressed it.
• The dashboards were wrong. I audited the Experimentation dashboards in Mixpanel and found that 18% of the 57 tiles were trustworthy. None of the five health metrics we planned to report were ready to show executives.
• AI research tools kept multiplying. Each new method became another tool that teams had to learn and decide whether to trust.

AI made the trust problem worse. PMs had told us they don't blindly implement Claude's suggestions, and they were right to be careful. If Voyager put one wrong number in front of executives, UX would lose credibility in the meeting it had just been given a seat in. So most of the design work went into making answers people could verify.

Starting with a pilot

Nobody owned this problem when I started, and there was no brief. I built a small version first, the Experimentation Weekly Digest, which posted usage and customer feedback to Slack with sources. I ran it for five weeks as a proof of concept.

The team read the short summary at the top and mostly skipped the full report, which was too long. The digest also only covered two of the four data sources, so it could say what customers asked for and what they did, and couldn't connect either to what we shipped. I kept the summary format and evolved the proof of concept into Voyager, which would cover all four sources.

I presented the pilot results to product teams and leadership. The VP of Product Operations and the Head of UX agreed to sponsor the project, and I wrote the PRD and the technical spec to plan it in phases, with success criteria and a list of what we weren't committing to yet.

The solution

Voyager is a Claude Code skill. You ask it a question about a product area in plain language. It looks up where that area's data lives, pulls customer feedback, usage, and releases at the same time, lines them up by date, and writes a short report. Session replay is a fourth source that runs only when you ask for it. The same engine will also run a daily dashboard for each product area, showing the health metrics UX reports in the reviews.

How the agent responds

There's no interface to design beyond the conversation, so I focused on how Voyager answers. I gave it four rules.

1. Give the answer first. Every report opens with its sources and a one or two sentence answer. The evidence comes after.
2. Let people refine by asking. You narrow an answer the way you would with a coworker, by asking a follow-up, asking for one source, or changing the date range. There's nothing to set up beforehand.
3. Ask before doing something expensive. A session-replay study reads through many recorded user sessions and takes much longer than a normal question. Voyager tells you that and waits for approval.
4. Say when it doesn't know. If two sources agree, Voyager reports a finding. If only one source shows something, it calls it a hypothesis. If a source has nothing relevant, it says "no related usage signal."

You:     What are customers saying about the experiment builder?
Voyager: Sources: voice + usage + business + releases.
         Customers are asking for A. Usage shows B on that surface.
         We shipped C. Business impact is D.

You:     Narrow that to the last 90 days, just the quotes.
Voyager: Sources: voice, last 90 days.
         Top themes: ...

You:     Make me a deck on friction in experiment creation.
Voyager: That runs a session-replay study over recent sessions,
         which is heavier than a normal question. Go ahead?

How the report is written

Reports are written BLUF, bottom line up front, because that's what the pilot showed people actually read, and executives in the reviews have a few minutes per topic. The format is the same every time, so reports can be compared across product areas and over time.

1. Title. The product area and the date range.
2. Sources. Which sources the answer actually used.
3. Bottom line. One or two sentences that answer the question. This is the part someone takes into the review.
4. Evidence. One line per source. Customer feedback themes ranked by how often they come up and weighted by revenue, with real quotes. The usage metric from the trusted source. What shipped and what's in progress.
5. Business impact. The revenue at stake, the risk, and the product change that would help.
6. Caveats. What's still a hypothesis, and any known gaps in the data.

Every rate states its date range and what it's a percentage of, because executives in the reviews question definitions. Every number names its source. Anything that wouldn't change a decision gets cut.

Here's a real report from September 18, 2026. The question was what happened after we switched the experiment builder's default statistical method from Bayesian to Frequentist. I've shortened it and redacted customer names, internal IDs, and raw counts for privacy.

Title:       User impact of the Bayesian to Frequentist default switch
             in the experiment builder, June 15 to September 18, 2026.
Sources:     voice + usage + releases. Session replay was run but
             couldn't answer this question (see Caveats).
Bottom line: The switch is changing behavior, with a lag, and without
             friction. Frequentist's share of viewed results rose from
             13% in July to 38% by mid-September while total viewing
             stayed flat. Customer feedback shows no reaction.
Evidence:
  - Voice:    No reaction. Mentions of statistical methods are flat to
              slightly down month over month, with no spike after the
              switch. Most recent mentions are sales and onboarding
              calls describing the feature. No cluster of complaints.
  - Usage:    Frequentist share of viewed results by month.
              Jul 13.3%, Aug 15.1%, Sep (to the 18th) 37.7%.
              Per day, Frequentist viewing roughly tripled, Bayesian
              declined, and the total stayed flat.
  - Releases: The new-experiment default flipped to Frequentist behind
              a feature flag on July 20. Holdouts and API-created
              experiments followed. A "New default" badge and a
              customer banner shipped in late August. The change is
              still behind the flag and rolling out.
Business impact: The default is working as intended. Adoption is rising
             through normal creation with no complaint signal. The one
             impact still unmeasured is how often users switch back.
Caveats:     The method is only tracked when results are viewed, so the
             mix at creation and the switch-back rate can't be measured.
             That tracking gap is worth a ticket. Voice is mostly sales
             calls, so a lack of complaints doesn't prove satisfaction.
             Session replay can't see the method selector in the form.

The release and the usage shift line up in timing and direction, so Voyager reported the change as a finding. It also named what the data couldn't answer, which is how often people switch back, and what tracking we'd need to find out.

How the skill is built

I designed and built the skill to follow the same structure as our team's other research skills, so anyone who has used one can read this one.

• The orchestrator reads the question, looks up the product area in the registry, runs the pipeline, and writes the report. If the area isn't registered yet, it says so and stops before guessing at sources.
• The registry has one file per product area, listing where each kind of data lives and which source to trust for each metric.
• Four reference docs tell the agent how to behave. They cover the report format, the rules for when signals count as corroborated, how to pull from each source, and a set of worked examples.
• The scripts do the math. A small script per source converts that tool's output into one shared format, and an alignment script lines everything up by date, weights feedback by revenue, and marks the weeks where two or more sources show something.
• The session-replay sub-skill has a playbook per product area that sets how far back to look, which sessions to include, and what to watch for.

The alignment script has a demo mode that runs on test data and fails if the pipeline breaks. The orchestrator treats a source with nothing to report differently from a source it can't reach. An empty source gets one line in the report. If a source is down because of an expired login or a disconnected connection, the agent stops before writing anything, tells you which source is down and the one step that fixes it, and waits. If you tell it to go ahead anyway, the report lists that source as unavailable, so readers know the answer is partial. The session-replay playbooks list what replay can't measure, such as whether someone changed a specific setting inside a form, and tell the agent to say that in the report.

Before building any of this, I tested the design with Claude Code. I gave it the framework proposal as its instructions and asked a real question about a real customer theme. In one prompt, with no follow-up, it pulled from Enterpret, Mixpanel, and GitHub and answered in the format I had specified. I checked its numbers against each source by hand. Since then I write specs for AI products so an agent can run them, which lets me try out a design before any engineering starts.

Key decisions

Fix the data before building anything people see. Phase 0 is only about the data sources. We audit each one, fix what's broken, and register it before any response or dashboard work starts. This means sponsors wait longer for a demo. I decided that was worth it, because one bad number in an executive review would make people doubt every good one after it.

Keep the math in code. The scripts do all the counting and weighting. The model does the parts that need judgment, like deciding that a customer quote, a drop in usage, and a release are about the same feature, and writing the summary. Every number in a report can be recomputed, which matters when an executive asks where it came from. The cost is a small script per data source that has to be updated when that source changes.

Keep one map of where the data lives. The chat and the dashboard both read from the same registry, so they always report the same numbers. Adding a product area means adding a file, with no code changes. Someone has to keep each file current as dashboards and tables change, and who owns that after Phase 0 is still undecided.

Bring existing research methods into Voyager. A UX colleague had a session-replay research method that worked well but was slow and manual, because it ran through a separate AI agent. I rebuilt it as one of Voyager's sources, so its results can be checked against feedback, usage, and releases. I also had Voyager read Enterpret's feedback and revenue data directly, since Enterpret already does that job well.

Let other teams adopt it when it helps them. The PRD only commits to Experimentation. Other product areas can join once Voyager has changed one of their planning decisions. This will spread more slowly than a mandate would, and I expect the teams that join this way to keep using it.

In the PRD I also answered a question I knew people would ask, which was why we didn't just use our general research agent. That agent answers from whatever it finds, including the broken dashboard tiles. Voyager only reads from sources we've checked, lines them up by date, and can run the same report on a schedule so results can be compared over time.

Impact

Fixing the P0 metric

The biggest problem Voyager's audit caught was in our most important Experimentation health metric, how often teams look at results after starting an experiment. The Mixpanel dashboard reported one number for it, and my audit flagged it as unreliable. Our Experimentation data lead rebuilt the metric from the data warehouse, one experiment at a time, and found two problems.

• A redirect inflated it. Clicking Start sends you straight to the results tab, which counted as viewing results. About one in eight results views came from that redirect.
• Empty experiments dragged it down. About one in seven experiments never got any traffic, so there were no results to look at, and they were still counted.

With both fixed, about half of the experiments that had results were never opened again after the day they started. The real problem was that people weren't coming back to check their results, which points design work at a different part of the product. The corrected metric went into the 10 September review as the evidence behind the Experimentation health signal. In Voyager's registry, this metric now comes from the warehouse, and every report that uses it says so.

What else changed

• Voyager answered a real product question. In September it showed that switching the experiment builder's default to Frequentist was changing behavior without customer friction, and it identified the tracking we'd need to measure how often people switch back.
• Only checked sources go into Voyager. The audit identified which dashboard tiles can be trusted, and a source only goes into the registry after it passes.
• Session-replay research moved into Voyager. It now runs as one of Voyager's sources, and its findings can be checked against feedback, usage, and releases.
• The work became a sponsored project. A five-week pilot turned into a project with two executive sponsors, a PRD, a technical spec, and a working skill covering three Experimentation product areas.

What's next

We're in Phase 0, auditing the rest of the Experimentation sources and connecting the warehouse so Voyager can read the corrected metric directly. Phase 1 refines the text reports and adds a slide report built from session replay. Phase 2 automates a recurring health-metrics report and a daily Experimentation dashboard for the reviews.

Success means three things.

• The Experimentation health metrics show up in the reviews automatically, from sources we trust.
• At least two more product areas adopt Voyager because it changed one of their planning decisions.
• Voyager regularly finds connections across feedback, usage, releases, and session replay that no single tool shows today.

Reflection

I built before I validated. The pilot showed me what the team would read and what the tool couldn't do, and I used both in the design. It also reported a Mixpanel number that my audit later showed was inflated. If I did it again, I'd audit the data sources before running the pilot, which is why auditing is now Phase 0.

Early on, all of this depended on me. Giving each product area its own registry file is how I plan to make Voyager run without me. The next step is agreeing who owns each file.