Before a deal review, check a predictive opportunity score's history minimums, model coverage, retraining settings, and licensing.
Key takeaways
- Confirm the won and lost counts for the selected period.
- Identify which model covers each deal.
- Check retraining settings and older models with the admin.
- Confirm quick-setup eligibility and the subscription renewal owner.
Scores give you a second view on open deals, and they can flag a deal that sounds healthy on the call but looks weak in the data. The catch is that a score is only as good as the model behind it.
The examples in this post come from Microsoft's documentation for Dynamics 365 Sales. Your tool may work differently, so read its own help pages before you adopt these checks. The four limits below help you decide whether a score belongs on the agenda.
Start with what the score is. Microsoft describes predictive opportunity scoring as a feature in which a predictive machine learning model calculates a score for open opportunities based on historical data. That sentence sets the boundaries. The score reflects patterns in past deals.
Limit 1: The model needs enough won and lost deals
The first limit is a minimum. Microsoft's documentation says: "Your organization must have a minimum of 40 won and 40 lost opportunities that were created and closed during the selected period." ( Microsoft, Configure predictive opportunity scoring)
Three details in that rule matter in a review. First, it sets two counts, not one. A team that wins often but loses few may clear a total-deals bar and still fall short on lost deals. Second, both counts apply to opportunities created and closed within the selected period. Third, the period is a choice. Someone set it, and it may not match the quarter you are reviewing.
The two counts matter for a practical reason. A model that has seen plenty of wins but very few losses has learned from a lopsided picture. You may see a high score on a deal that looks like the wins it knows, and a low score on a deal that resembles a loss it barely saw. You cannot tell from the number alone which situation you are in.
Before the review, ask for the won and lost counts for the period the model uses. If either count falls below the minimum, the stated prerequisite is not met for that period, and you should not present the score as settled evidence. Say so in the meeting, or ask the admin which period is in use and whether a different period would meet the rule.
Microsoft's page also shows a worked example in which the model "calculates a score of 80 for Opportunity A and 50 for Opportunity B." ( Microsoft, Configure predictive opportunity scoring) That example shows two scores side by side. It does not say what the 30-point gap means for the chance of winning. Do not translate scores into percentages in the room unless your tool documents what the number means.
Limit 2: You may have several models, and only some apply
The second limit is a cap on how many models you can keep. Microsoft says: "You can create up to 10 models, both published and unpublished, for different sets of opportunities." ( Microsoft, Configure predictive opportunity scoring)
Two points follow. First, the models are scoped to sets of opportunities. One team may have a model for large enterprise deals and another for smaller accounts. A deal in your pipeline may be scored by one model, or by none if it falls outside every model's set. Second, the cap counts unpublished models too. An unpublished model is not evidence that the score is being used, and a published model may not cover your segment.
Consider a simple case. Your pipeline has three segments: new logos, expansion deals, and renewals with a services attachment. Suppose the org has two models, and one was built for expansion deals only. Before comparing scores across segments, confirm which model produced each score and whether the comparison is useful.
Ask two questions before the meeting. Which model scores each segment of the pipeline in this review? And are any deals on the agenda outside the set the model was built for? If nobody can confirm which model covers a deal, pause the comparison. Treat it as a question to explore, not a ranking to act on.
Limit 3: Retraining settings and old models
The third limit is about freshness. A model trained on last year's deals may not reflect how your team sells now. Microsoft's documentation includes this instruction: "To retrain the model every 15 days, select Retrain automatically." ( Microsoft, Configure predictive opportunity scoring)
That setting is the one to check. Ask whether automatic retraining is on, and how often the model learns again. If the option is off, find out who retrains the model and when. Do not assume the score reflects recent closed deals just because it shows up on the screen.
Freshness also changes what a score means over a quarter. If the model retrains during the quarter you are reviewing, scores for the same deal can move between meetings. That is not automatically a problem, but it is a reason to note the date of each score in your review notes. When someone asks why a deal dropped, you want to be able to say whether the deal changed or the model did.
Old models need a separate check. Microsoft notes: "If you're using a model that you created in a version of Dynamics 365 earlier than 2020 release wave 2, delete the model before you create a new one." ( Microsoft, Configure predictive opportunity scoring) This rule matters when an org has moved across versions. A model built in an older version may still be in use in your org. Ask the admin to check the creation version and handle any required replacement. The deletion is the admin's call. Do not change the model yourself.
Limit 4: Licensing and subscription status
The fourth limit is about quick-setup eligibility and historical-data retention. Microsoft says: "If you have a Dynamics 365 Sales Enterprise license, you can enable predictive opportunity scoring in lead and opportunity scoring quick setup." ( Microsoft, Configure predictive opportunity scoring)
For a front-line manager, the practical step is simple. Ask the admin to confirm that your license covers this setup before you plan a review that depends on it. If your team's license is different, confirm before you plan around the score, and do not assume the feature is available just because your reps see a score field.
Microsoft also says: "If your Dynamics 365 Sales subscription expires or your org is deleted, the historical data is deleted after 30 days." ( Microsoft, Configure predictive opportunity scoring) The model learns from historical data, so a lapse threatens the thing the score depends on.
Put the subscription renewal date on the same calendar as your quarterly reviews, and ask who owns the renewal. A renewal that slips is a risk to the score, not only to the paperwork. If you find out the renewal is close, raise it early. The admin, finance, and the sales leader should all know that the score's history depends on the subscription staying active.
Illustrative example: A score changes the discussion
Suppose a deal has a score of 80, while another has a score of 50. The manager first checks which model produced each score and whether it covers the relevant segment. The manager then learns that the selected period contains 30 lost deals. That count falls below the documented minimum, so the manager does not use the scores to rank the deals. Instead, the team reviews rep notes and stage evidence, records the unmet prerequisite, and asks the admin to check the selected period before the next review. These figures illustrate a review decision, not a prediction of either deal's outcome.
Today, put the questions below on your review agenda and ask your admin or sales operations lead for written answers.
Turn the four checks into four questions
What are the won and lost counts for the period the model uses?
Ask the admin for separate counts and the selected period. Check the counts against the minimum above before treating the score as evidence.
Which model scores each segment in our pipeline, and how many models exist in the org?
Ask for a model-to-segment list that includes unpublished models. Use it to identify coverage gaps before comparing scores.
Is automatic retraining on, and are older models in use?
Check the retraining setting and each model's creation version with the admin. If automatic retraining is off, record who handles it and when.
Does our license cover quick setup, and when does our subscription renew?
Ask the admin to confirm quick-setup eligibility and the renewal date. Record who owns the renewal so concerns have a clear destination.
If the answers are clear, bring the score to the review as one input. If they aren't, bring the question instead. "We don't know which model scored this deal" is a useful agenda item. It is more honest than a number nobody can explain, and it gives the admin a clear task before the next cycle.
Keep the checks short. You do not need a full audit before every meeting. You need the four answers, written down, with the date you got them. Once the answers are stable, you can check less often and only when something changes, such as a new model, a license change, or a renewal date.
Make the check repeatable
A check you do once from memory will drift. Write it down as a short sheet that sits next to the review agenda. Keep it to one page, and update it each time something changes.
For each row, record the setting or count you were given, where you got it, the date you confirmed it, and the name of the person who confirmed it. The last column matters more than it looks. When a number changes after a renewal or a model rebuild, you want to know exactly who to ask.
Add a fifth row for open questions. Phrase each item as a task, such as "confirm which model covers the renewals segment" or "ask whether the period in use matches this quarter." An open question on the sheet is a sign the review is being run carefully, not a failure of the process.
Share the sheet with the sales leader and the admin. If they disagree with a row, settle it before the meeting, not during it. Reps need to see only the outcome: whether the score is a main input, a secondary input, or not used at all this quarter. That keeps the conversation on the deals rather than on the plumbing.
For the wider agenda, use the weekly pipeline review checks.
Use the score to ask better questions
Once the checks pass, the score still needs judgment. The most useful place for it is where the score and the rep disagree. A high score on a deal the rep calls stalled is a question. What does the model see that the rep does not? What does the rep know that the record doesn't show? Ask that before you change the forecast.
Keep the rep in the lead. Ask them to explain the deal in their own words first, then show the score. The rep knows the buyer, the champion, the budget owner, and the timing. The model knows the patterns in past records. Neither one is complete on its own, and the meeting works best when both are on the table.
Use the sales call review notes to prepare that discussion.
Pay attention to the cases where the score has been right and the rep has been surprised, and the cases where the rep was right and the score was not. Both kinds of case teach you something about how far to trust the number. Say the lesson out loud in the meeting, so the whole team learns it together instead of one manager quietly adjusting their own view.
Be careful with the score as a forecast. A score is a model output, not a commitment. Do not roll it into a team number unless your team has agreed how it should be used, tested it against outcomes, and written down the rules. A score that moves a forecast without a documented basis is hard to defend when the quarter goes sideways.
Keep a short record
Each review, note the model used, the score for each deal on the agenda, the date of the score, and any change you made because of it. After a quarter, compare the scores with what happened. This is not a statistical test. With a small team it's a manual check, but it tells you whether the score deserves more weight, less weight, or a different model.
Over several quarters, the record also helps you spot patterns in your own team. If the score is consistently high on deals that stall, or low on deals that close quickly, that is useful information for the admin and for the sales leader. It may point to a model that needs attention, a segment that is poorly covered, or a habit in how reps update records.
Predictive scoring is a tool for better conversations, not a replacement for them. The four limits are the conditions that make it worth trusting. Check them before the review, keep the rep in the room, and write down what you learned. Then the number on the screen will mean something when you put it in front of your team.