Behavioural metrics: your smallest escape from the Feature Factory

Learning without (much) cost.

Timo here. I write about the slower Product Discovery topics. The essentials, not the latest thing; the internet already has plenty of those.
Figuring things out as I go, 100% written by a human (me), with a new post every now and then.

Maybe I’m lucky, but I have yet to meet a Product team that doesn’t want to do more discovery and escape the Feature Factory.

All of them wish they could talk more to users, had more autonomy over their product, and have a bigger impact on the metrics they and their bosses are looking at.

Wanting is hardly ever the issue. It’s mostly can’t, for many different reasons.

There is no need for me to write yet another post about what’s wrong with a Feature Factory and all the nuanced complexity that turns teams into them. I’m more interested in the now what part.

When I ask my favourite LLM or even do an old-school Google on how to escape the Feature Factory, I get things like:

• Change the focus from outputs to outcomes

• Define and align around a clear product vision & strategy

• Invest in continuous Product Discovery

Oof.

I see that these things are needed if I want to make a permanent and sustainable change from the factory, but they also feel a little disheartening. They can seem impossible to get done.

These things are massive and require the rest of the org. I know that that’s the entire point, but what’s the smallest thing I can do today? The thing to get the ball rolling?

The smallest ‘Total cost to get it done’

The Total cost to get it done is a made up bucket term that encapsulates all reasons I’ve heard of not doing something that helps escape the factory:

• Don’t have the budget

• No time in my day

• No time in the day of the people I need

• Don’t have the skills required to execute

• Don’t have the buy-in

• Can’t because it’s not in the strategy

• Can’t do it because it will delay the delivery timelines

The things mentioned just now have a massive Total cost to get it done. They max out on all of these factors.

Instead of investing in continuous product discovery and fully embedding that in the way of working, I could do a one-off sprint. Or even smaller, I could do a one-off activity like a single user interview, A/B test or survey.

These things STILL have some Total cost to get it done though. All of these one-off activities still require either some basic knowledge and tooling, or the time / budget to unlock them.

I have access to AI now though, which can help reduce the Total cost to get it done. Some half decent prompting and a $25 Lovable subscription will get me a working prototype that I can use and share internally to get quick feedback on the thing I’m planning to build.

Though small, it still has some Total cost to get it done. Maybe not on time in my day and budget (I can pay the $25 out of pocket if needed), but what can I do with the outcome of a prototype test if I can’t delay delivery timelines?

So, what is something with an even smaller cost that doesn’t slow down delivery either?

Add a behavioural metric

Chances are that if you’re stuck in the Feature Factory you’re looking at GMV, Conversion Rate, MRR (or none at all). Those are all important things but they’re things I get for a job well done.

They are results for the business. They’re quite hard to influence directly with the things I’m building, if at all.

A behavioural metric sits in between the thing I’m building and the result for the business. It tells me if and how it impacts the behaviour of the user.

Example
For an ecom app we expanded the searchbox to be accessible from every screen, trying to increase the conversion rate (from search). The behaviour we wanted to see is whether it was now easier for more users to search, so one of the behavioural metrics was users that search.

Ideally the research work was done upfront to figure out whether this was worth doing at all. We could’ve checked whether search helps users convert, and if it’s worth expanding it to more screens.

But that’s not the way of the Feature Factory. Too much Total cost to get it done.

I didn’t stop or change what we were planning to build to do research first. That’s the whole point! But I was able to add a valuable feedback loop by adding the behavioural metrics.

What this enables me to do and how it helps me escape

What is the feedback loop and what does it enable? First, let’s look at my options when I built a feature WITHOUT adding behavioural metrics:

There isn’t much I can learn, so there isn’t much I can do either.

When I select a behavioural metric, I’m assuming there’s a link between the user behaviour I’m tracking, and the business impact that will have. More search = more conversion (basically a very barebones hypothesis).

Tracking both enables me to learn and do a whole lot more, depending on whether the link holds. Using the search example from earlier:

Obviously having these learnings isn’t instantly going to prevent me from On to the next feature. But at least I now have some extra information to work with, which unlocks more actions:

• Double down: taking the idea a step further to see how far the link holds. More search = more conversion, even more search = even more conversion?

• Flip: try the opposite direction. If more search = less conversion, does less search = more conversion? Try decreasing the visibility of the feature, i.e. removing it from every page.

• Find the other link: an impact on a business metric is almost always caused by a change in user behaviour. If it wasn’t search, what was it? Find it. It’ll tell me what to do next.

• Iterate: try a different approach to the same link. Maybe adding search to more pages wasn’t it, but autosuggest or better placement could be.

• Investigate: somehow I achieved the opposite of what I expected. It’s worth finding out why first. Could be faulty implementation or there’s something else going on that I’m not yet seeing.

I’m stuck building things anyway (because Feature Factory); I might as well take the opportunity and learn from what I’m doing by adding this metric.

The newly unlocked actions won’t change anything directly, but they will help me make better informed decisions next.

Eventually the ball will start rolling, which will help escape the Feature Factory. Bit by tiny bit.

The learnings from this will lead to better conversations. Those conversations will lead to better questions. Those questions need answers, which will create the need for (justifies) more Discovery. More Discovery will create more learnings, and on the flywheel goes.

Not overnight and not by itself, but what can I expect with basically 0 Total cost to get it done.