ICP Drift: When to Re-Score Accounts and Rewrite Your ICP

What is ICP drift?

An ICP is a snapshot of who was buying when someone wrote it down. The product then ships new features, pricing moves, a competitor appears and a different kind of buyer starts showing up. The document stays the same, and so does every score built on it.

That is what makes drift expensive. Nothing breaks. The queue still fills, and accounts still get ranked, against a customer who may no longer be your best one. It is different from lead score decay, which ages out old activity on one record. Decay keeps a score current; it cannot tell you the criteria are wrong.

Machine learning has the same problem under a more precise name. AWS separates concept drift, a change in the relationship between inputs and outcomes, from data drift, a change in the inputs themselves. The split is useful here. If your accounts changed, you need fresh scores. If what predicts a win changed, you need new rules.

Should you re-score, re-weight or rewrite?

Most advice on drift stops at a review cadence. The harder question is what the review should change:

• Re-score: same rules, fresh data. It is cheap and low-risk, so do it whenever the data refreshes.
• Re-weight: same criteria, different points. Reps see a new ranking, but your definition of the customer holds.
• Rewrite: a new definition of who you sell to. Targeting, routing, messaging and reporting all have to follow.

Which signals mean it is time to act?

A rep's complaint is a prompt to check, not a verdict, and the same goes for a good quarter. Compare what you hear against closed deals before you touch a weight.

Is a quarterly review the right cadence?

For most teams, yes, as a floor. It gives you a fresh batch of outcomes without leaving a stale model in place for long. The vendors show how much cadence varies with the setup: Microsoft Dynamics 365 can retrain a predictive lead model every 15 days, while Demandbase retrains its pipeline score every 3 months by default.

The limit of a calendar is that drift does not follow one. AWS lists retraining on a fixed schedule regardless of performance as an anti-pattern. So keep the quarterly check, but let the review change something only when the evidence says so, and let a pricing or product change pull it forward.

Automatic retraining does not settle this either. A retrained model still learns from the accounts your team chose to work, so it can sharpen a past choice without ever questioning it.

How do you run a drift review?

1. Pull recent outcomes. Closed-won, closed-lost and churned accounts since the last review, with the score each one had when it entered the pipeline.
2. Test each criterion. Does it still separate wins from losses? Mark it as still working, neutral or misleading. The method is the same as when you first validate an ICP with closed-won data.
3. Read the misses. Look at high-scored losses and low-scored wins, and ask the reps who worked them what the score could not see.
4. Check your exclusions. Disqualifiers and negative keywords drift too, and a stale one hides good accounts where nobody will look.
5. Decide, and write it down. Choose re-score, re-weight, rewrite or no change, with the evidence behind it. "No change" is a valid result.
6. Push the change through. Update the score, then routing, target lists and reports. Keep the old definition next to the new one so the next review can compare like with like.

A rewrite also has to reach everything that reads the old definition. Re-rank open accounts, or your reps will keep working a queue built on the profile you just retired. The Pain-First ICP Scoring framework keeps pain signals in their own track, so you can see whether fit or pain is the part that moved.

What changes at a seed or Series A startup?

Early-stage teams drift fastest and have the least data to prove it. The ICP can move with one product decision, and a handful of deals cannot support a statistical test. Read each recent win and loss by hand, and treat percentages as directional. Rely on events: a new price, a new product or a move to selling to buying groups is reason enough to review.

A first sales leader is another good moment. They inherit a score built on a founder's memory, so test it against closed deals before they build a team and a quota on top of it.

Where Artemis GTM fits

Artemis GTM is an AI GTM engineering firm. In an engagement it builds the ICP score inside your own CRM, wires in the fit and pain signals, and sets up routing and the review routine, so your team can re-score and re-weight without starting over. The systems stay with you. Book an evaluation to scope it.

Frequently asked questions

Sources and references

• AWS, "Retrain only when necessary," Machine Learning Lens: defines concept drift and data drift, and lists fixed-schedule retraining as an anti-pattern.
• Microsoft Learn, "Edit and retrain a lead scoring model": automatic and manual retraining for predictive lead scoring in Dynamics 365 Sales.
• Demandbase, "Retrain Pipeline Predict Scores": the default automatic retraining schedule for its pipeline score.