What to ask before you buy an attribution tool

I am evaluating attribution tools and every demo looks the same. What should I actually ask?

Attribution demos converge. Everyone shows a clean funnel, a channel breakdown that adds up, and a number larger than the one your platform reports. Thirty minutes in, the products are hard to tell apart, and the differences that will matter in six months are the ones nobody puts on a slide.

These are the eight questions that separate them. I build one of these tools, so this is not a neutral document and pretending otherwise would be worse than saying it. Two of the questions below are ones we have to answer with a limitation rather than a feature, and I have marked both where they come up. A buyer's guide that only asks questions the author passes is an advertisement.

1. What is the anchor?

Every attribution product starts from one of two places. It either begins with what the ad platforms report and reorganises it, or it begins with the orders your store actually recorded and works backwards to explain them.

The difference decides what the tool can ever tell you. A product anchored on platform reports can present them more usefully, deduplicate obvious overlaps and apply a nicer model. It cannot check them, because it has no independent reference. A product anchored on orders can do both, because it has a list that exists regardless of what any platform says.

Ask the question directly, and listen for whether the answer is specific. It is the single most predictive thing you can learn in a demo.

2. Can you show me one order and how it was attributed?

Aggregates hide everything. Ask for a specific order, ideally one you can identify in your own admin, and ask what evidence the tool has for the channel it assigned.

A good answer names the evidence: this order was matched to this session by this identifier, and that session arrived with this click identifier at this time. A weak answer describes a model. Models are fine, and every tool has one, but a model applied to unverified inputs produces a confident number with nothing underneath it, and you cannot tell the difference from a dashboard.

3. What does it do when it cannot attribute an order?

Some orders cannot be attributed. Someone blocked the pixel, arrived through a link that lost its parameters, or bought on a device that never touched the session that started the journey. That population exists in every dataset and it is often larger than people expect.

The right answer is that those orders go into a visible bucket with a count in it, and that the per-channel numbers plus that bucket sum to your total. The wrong answer is that they are distributed proportionally into the named channels, because that is invisible, it inflates every channel evenly, and it turns a known gap into a confident wrong number. If the answer is vague, ask where the unattributed count appears on screen. If it appears nowhere, that is the answer.

4. How does it handle refunds and cancellations?

A cancelled order stops being revenue in your store immediately. Ad platforms generally keep the conversion, because they were told about the purchase and never told about the reversal.

So ask which side the tool follows, and ask what happens to an order that is refunded a month after it was attributed. On a category with meaningful return rates this single behaviour moves reported ROAS more than most model choices, and it is almost never discussed in a demo.

5. What is your definition of a new customer, and where is it applied?

New customer acquisition cost is what most budgets are set against, and the definition underneath it is doing more work than it appears to.

Ask two things. First, over what history is first-order status computed? Anything less than the customer's entire history silently turns loyal buyers into new ones whenever their last purchase falls outside the window. Second, is the same definition used in the dashboard, in the audiences you upload, and in the exclusion lists? When those disagree, the same person is new in one place and returning in another, and the exclusions stop excluding.

This is one where we had to fix our own answer before we could give it.

6. What happens when the pixel is blocked?

A meaningful share of your visitors run an ad blocker, browse in a privacy-hardened browser, or check out on a domain your code never runs on. Every tool in this category loses some of them, including ours.

So the useful question is not whether the tool handles it. It is what specifically the tool does, and what it does with the sales it still cannot see. Look for server-side order ingestion, because an order webhook does not care what the browser did, and for honesty about the residue: the orders that arrive with no session to attach them to. A vendor who claims to lose nothing is describing a product that does not exist.

7. Can I reproduce a number without your dashboard?

Take one figure from the demo and ask how you would arrive at it yourself, from your store admin and the platform's own reporting.

If there is a path, the tool is doing arithmetic you can audit and it will survive a disagreement with your agency. If the only place the number exists is inside their interface, you have bought a second opinion rather than a way to check the first, and the next time two reports disagree you will have three.

8. What does it not measure?

Ask it last and pay the most attention to it.

Every honest answer to this question exists and is short. Attribution cannot prove causation: it can say where a sale came from, never whether the spend caused it, which is what an incrementality test is for. It cannot see a purchase influenced by an ad that produced no click and no visit. It cannot resolve a buyer who used a different email, a different device and paid with someone else's phone number.

A vendor who answers this cleanly has looked at their own product carefully. A vendor who says nothing is missing has either not looked or is not going to tell you, and you will find the gaps yourself, later, in the middle of a decision.

What a good answer sounds like

The pattern across all eight is the same. Good answers are specific, name a mechanism, and include a limitation without being asked twice. Bad answers describe a capability in the abstract and move on.

The underlying reason is worth stating plainly, because it is the whole argument. A platform can legitimately resolve a buyer you never saw, so a claim larger than what you measured is not automatically inflation. But no platform can resolve an order that does not exist. Your settled orders are the only list in this entire picture with no model inside it, and the tools worth paying for are the ones that treat that list as the thing to be explained rather than as one more data source to be blended in.