top of page

We tested five customer engagement approaches... The scores weren’t the interesting part

Writer: Andrew Mallaband
Andrew Mallaband
Sep 14
6 min read
Black background with white text: "More messaging adds detail. It doesn't add momentum." Curved lines on right. Button: "Why we started building Storylines."


What a sales engagement experiment taught us about activity, evidence and whether the conversation actually changed anything for the customer...

Over the last month I’ve been involved in an experiment around customer engagement. It began with a fairly ordinary question about first contact: if you have a legitimate reason to speak to a potential customer, what should that first interaction actually be trying to achieve?

Most sales processes have an answer, although it is usually expressed from the seller’s point of view. Get a reply. Secure a meeting. Start discovery. Create an opportunity. All perfectly understandable objectives, but I became more interested in a slightly different question: what, if anything, should be different for the customer afterwards?

That took the experiment in a direction I wasn’t expecting.


What we were actually testing

We looked at principles from several well-established approaches. Large-scale Gong research gave us useful evidence around things such as message length, pitching too early, calls to action and cold-call openings. SPIN reinforced the importance of investigating before demonstrating. Sandler brought the idea of low-pressure engagement and agreeing the basis of the conversation before trying to solve anything. Challenger introduced another useful possibility: sometimes a good conversation helps somebody see an issue differently.

We tested variants inspired by those approaches alongside the recognition-first and evidence disciplines we have been developing ourselves. Importantly, we weren’t trying to reproduce or grade entire sales methodologies. We were testing engagement principles under the same conditions.

We then repeated the exercise across three very different propositions, deliberately chosen to see whether the results depended on the type of problem, buyer or offer. The relative scores were surprisingly consistent.

I expected that ranking to be the interesting result. It wasn’t.

What caught my attention was how difficult it is to distinguish between evidence that the seller has done something and evidence that the customer now understands something they did not understand before.


Activity is much easier to measure

Sales organisations have become very good at recording activity. Calls, emails, replies, meetings, discovery sessions, demos, proposals and opportunity stages can all be captured, compared and reported.

There is nothing wrong with that. The difficulty comes when those measures quietly become evidence of progress.

A reply tells us somebody replied. It doesn’t necessarily tell us they recognise the problem we think they have. Agreeing to a meeting doesn’t mean they understand why an issue matters. A successful demo proves that we showed them something successfully; it does not tell us whether they can connect what they saw to a decision important enough to make.

Even an apparently positive statement such as, “Yes, we have that problem,” leaves quite a lot unresolved. They may recognise the situation without understanding its cause, consequence or importance. They may understand it but have no intention of changing anything.

This distinction has become increasingly important in our wider work because recognition is not understanding. Recognition can be the beginning of a useful conversation, but treating it as if it were already understanding creates a very different picture of what is actually happening.


A simple way of looking at what changed

For the experiment we needed some way of separating a response from a more meaningful change in the conversation. We ended up using a simple working scale.

R0 was a response or acknowledgement. R1 meant the customer recognised the situation, or corrected our assumption. At R2 they were prepared to give us, or reconstruct, a concrete example. R3 meant they were willing to examine a specific question or piece of evidence. R4 meant there was a legitimate reason for another conversation without us having to manufacture urgency.

I wouldn’t call that a new sales methodology. At this stage it is simply a way of making the experiment measurable.

What I like about it is that it also gives negative evidence somewhere to go. “We don’t have that problem.” “You’ve misunderstood what is happening.” “Our existing approach is good enough.” “This isn’t a priority.” Those answers are useful. Even no reply is information, although obviously we need to be careful about pretending we know why somebody did not respond.

Sales systems have an understandable tendency to favour continuation. The next meeting, the next stage, the next piece of pipeline. But there are times when the most useful result from an engagement is discovering that there isn’t a sufficient reason to continue.

That makes the purpose of first contact less about getting a meeting at all costs and more about finding out something legitimate about the customer’s situation, while establishing whether another conversation would actually be useful.


Then the experiment exposed a problem in our own measurement

This was probably the part I found most useful.

We deliberately included poor engagement examples so that we could see whether the scoring model would reject them: product-first pitches, assumed pain, intrusive personalisation, manufactured urgency and forced meeting requests. We also included the familiar follow-up, “Just bumping this.”

On the first run, “Just bumping this” did surprisingly well.

When we looked at why, the answer was obvious. The model was rewarding brevity and low pressure. That follow-up is certainly brief and it is not particularly aggressive, so it was benefiting from the way we had designed the measurement.

The problem is that it contributes nothing.

We changed the rubric so that an interaction also had to make a substantive contribution, applied a penalty to empty follow-ups and ran the experiment again. We kept the failed calibration in the engineering record rather than quietly removing it.

That example has probably taught me more than the ranking of the different approaches. If a metric is only an approximation of what we care about, it is very easy to become better at satisfying the metric without becoming any better at the thing we originally wanted to achieve.


AI makes that particularly relevant

AI is giving us an extraordinary ability to capture and process customer interactions. Calls can be summarised automatically, CRM fields completed, objections extracted, themes identified and follow-ups generated within seconds.

I use these capabilities and think they are genuinely useful. But there is a risk in the apparent completeness they create.

A CRM record can look exceptionally rich without the customer understanding the issue any better. We can have excellent notes from a meeting that should never have happened. We can produce a very sophisticated summary of a sales process in which everybody has been busy but nobody has learned very much.

Perhaps that means we need to ask a few more basic questions alongside the usual sales measures. What do we know after the conversation that we did not know before? Did the customer recognise something they had not previously considered? Did they correct one of our assumptions? Did they give us evidence? Are they prepared to examine the issue in more detail? Are they any closer to deciding whether it matters?

And equally, did we discover that our original assumption was wrong?

That last question may be one of the most important because a system that can only report positive progress is not really measuring evidence.


What I think is worth testing next

Our hybrid approach performed strongly in these engineering tests, including the label-blind comparison, but I don’t think that gives us grounds to declare that we have developed a better sales methodology. It doesn’t.

What we have so far is much narrower and, to me, more interesting. Different engagement principles can be tested under common conditions. Deliberately poor examples can expose weaknesses in the way we measure them. The same tests can be repeated across substantially different propositions without depending on one particular customer or market. And it appears possible to look beyond activity and ask whether we have evidence that something actually changed in the customer’s understanding or willingness to investigate further.

The next stage needs people rather than another model test. Humans should review the messages without knowing which approach produced them, followed by controlled field testing where the outcomes are agreed before we start.

Reply rates will still matter. Meetings will still matter. Pipeline and revenue certainly matter. I’m not suggesting we replace any of those.

I do think there may be something important between the conversation and the commercial result that we have not measured particularly well.

After the next customer conversation, I’m going to try asking a different question.

What changed, and what evidence do we have that it changed?

Comments


bottom of page