Automated call scoring uses predefined questions to standardize rep evaluation
Set up automated call scoring by defining evaluation questions, confirming a transcript for AI evaluation, and checking the license that includes scoring.
Key takeaways for how to set up automated call scoring for sales coaching
- Aircall states that AI Assist scores calls using a predefined set of evaluation questions that provide a standardised framework for coaching and performance reviews, as noted in Aircall.
- Avoma specifies that each question in the AI scorecard will be evaluated on the scale of 5, according to Avoma.
- To make the AI Coaching Assistant automatically score conversations, Avoma requires you to either assign purpose(s) to scorecards OR assign these scorecards to the team(s) whose calls you wish to auto-score, according to Avoma.
- Avoma notes that AI Coaching is available only to users with a Conversation or Revenue Intelligence license, as stated in Avoma.
- Aircall explains that users can evaluate any call with a recording, but only calls with transcripts benefit from AI evaluation, according to Aircall.
Aircall AI Assist scores calls using a predefined set of evaluation questions that provide a standardised framework for coaching and performance reviews, according to Aircall. The following table outlines the specific documented outputs, requirements, and integrations for each publisher based on their documented features.
| Publisher | Documented output | Scale or Format | Transcript Requirement | Language Support | License Requirement | Summary Integration |
|---|---|---|---|---|---|---|
| Aircall | AI evaluated questions and manual criteria | Predefined set | Transcripts required for AI evaluation | 24 languages in two tiers | AI Assist Standard does not support automated call scoring | HubSpot, Salesforce, and Zendesk |
| Avoma | AI Coaching Assistant scorecards | Scale of 5 | Not stated on Avoma page | Not stated on Avoma page | Conversation or Revenue Intelligence license | Not stated on Avoma page |
| Gong | Call brief for processed calls | Fixed-format summary | Generated from call transcript | Not stated on Gong page | Not stated on Gong page | Not stated |
| Microsoft | Suggestions added on the call summary | Action items, commitments, minutes, next steps | Not stated on Microsoft page | Not stated on Microsoft page | Not stated on Microsoft page | Call summary page |
To begin configuring this system, identify the specific evaluation questions that align with your team's current coaching priorities. Ensure that the selected tool supports the necessary transcript quality and language tiers for your specific market. Verify that your current license tier permits the use of automated scoring features before proceeding with the setup.
Defining the evaluation scale and question types
Standardizing how scores are calculated prevents ambiguity when reviewing performance. You must define the scoring logic for each question type before the system can process calls automatically.
Avoma defines the default scoring scale for its AI scorecards. According to Avoma, each question in the AI scorecard will be evaluated on the scale of 5. This baseline applies to the standard evaluation format within the platform.
For questions with two distinct outcomes, the scoring logic is fixed. According to Avoma, for binary evaluation (No/Yes) type of questions, the score will be a 5 for Yes and 1 for No. This creates a clear distinction between the presence and absence of a specific behavior.
According to Avoma, if a question has three options with 1 being the worst and 3 being the best, the best option will be scored with a 5 and the worst will be scored with a 1. This mapping ensures that the highest possible score reflects the best available option.
Some platforms allow for mixed evaluation methods within a single scorecard. According to Aircall, Call Scoring allows managers to create scorecards that combine AI evaluated questions and manually evaluated criteria to assess call performance. This structure supports hybrid review processes where automated checks coexist with human judgment.
Recommendation: Map your specific coaching behaviors to the available question formats in your chosen tool with help from Sales coaching scorecard setup with 6 documented checks. Confirm whether your platform uses a fixed scale for binary or three-option questions before building your first scorecard.
Illustrative example
A manager adds one question with 3 options. The best option scores 5 and the worst scores 1. A yes or no question scores 5 for yes and 1 for no. Across 10 recorded calls, 6 score 5 and 4 score 1. Read those 10 results before you assign the scorecard to one more group.
Assigning scorecards to teams and call purposes
To trigger automatic scoring, you must link the scorecard to specific call contexts or user groups. Avoma requires you to either assign purpose(s) to the scorecards or assign the scorecards to the team(s) whose calls you wish to auto-score, as stated in their documentation for Using Avoma AI Scorecards for AI Coaching. This dual-path logic means you can target scoring based on the reason for the call, such as a demo or discovery, or based on the organizational unit, such as a specific sales pod. If you leave both fields empty, the AI Coaching Assistant will not automatically score the conversations using that scorecard.
According to Aircall, all licensed users with Admin and/or Supervisor roles can create and edit a scorecard from AI Assist → Call Scoring.
The actual scoring action depends on the question type within the scorecard. For scorecards containing predefined AI questions or Pro level custom AI questions, AI automatically fills scores and explanations using the conversation transcript, according to Aircall. This automation applies specifically to those question types; other question formats may require manual input or different processing logic. Ensure that the questions you assign to your teams are of the type that supports automatic filling to achieve the intended speed and consistency.
Managing language and transcript quality requirements
Transcript availability determines whether automated scoring can run. According to Aircall, users can evaluate any call with a recording, but only calls with transcripts benefit from AI evaluation. This creates a hard dependency: if your phone system does not generate text transcripts for a specific call, the AI scoring feature cannot process it. You must verify that your telephony setup produces transcripts for every call type you intend to score. Any call with a recording can still be evaluated, and only calls with transcripts benefit from AI evaluation.
Language coverage affects where you can deploy these scorecards. According to Aircall, Aircall supports AI Assist and AI Assist Pro in 24 languages, across two coverage tiers. The specific breakdown includes 17 additional languages across Eastern Europe, the Nordics, and other regions, such as Polish, Romanian, Bulgarian, Czech, Hungarian, Slovak, Croatian, Russian, Catalan, Ukrainian, Swedish, Norwegian, Danish, Finnish, Turkish, Greek, and Filipino, according to Aircall. These two tiers suggest different levels of support or accuracy for different language groups. You should not assume that every language in the list performs identically in your specific workflow.
Quality control requires testing before full rollout. According to Aircall, if you rely on transcription or AI output quality for a business-critical workflow, we recommend using a Verified language where possible, or testing Additional languages on a representative sample of your calls first. This distinction between "Verified" and "Additional" languages implies a difference in reliability or support status. If your sales team operates in multiple regions, you must map which languages fall into which tier for your specific vendor. Do not apply a single quality expectation across all 24 supported languages.
Before configuring scorecards for international teams, check the transcript generation settings for each region. Confirm that the phone system creates text files for calls in the target languages. Then, run a small batch of calls in the "Additional" tier languages to check if the transcription quality meets your coaching standards. If the errors are too high for reliable scoring, restrict automated scoring to the "Verified" tier languages until you can validate the output further. This prevents bad data from entering your coaching pipeline.
Checking license requirements for automated features
Before you configure scorecards, confirm your current plan includes the necessary automation features. Vendors restrict these capabilities to specific tiers, so a basic subscription may not support the full workflow.
Avoma limits access to its AI Coaching Assistant. According to Avoma, AI Coaching is available only to Avoma users with a Conversation or Revenue Intelligence license. If your team holds a different Avoma plan, you cannot use this specific automated coaching feature. You must verify your license type before attempting to set up scorecards.
Aircall also restricts automated scoring based on the AI Assist tier. According to Aircall, AI Assist Standard does not support automated call scoring. This means teams using the Standard tier of Aircall AI Assist cannot rely on the system to generate scores automatically.
Check your billing dashboard for both tools. Look for the specific license name listed in your subscription details. Avoma AI Coaching needs a Conversation or Revenue Intelligence license, and Aircall AI Assist Standard does not support automated call scoring. Contact your vendor's support team to confirm which license includes automated scoring. Do not assume that a paid plan includes all AI capabilities. Each vendor defines its own feature gates.
Document the license requirements in your onboarding checklist. New managers should verify these permissions before assigning coaching tasks. This ensures that every rep has the necessary software access from day one. It avoids confusion during the first week of a new manager's tenure.
Logging call summaries and notes
Once the scoring engine processes a call, the resulting data must flow into the systems where your team already tracks pipeline and follow-ups. Each platform handles this integration differently, so you need to map the output to your existing CRM workflow.
Aircall's AI Assist logs call summaries and key topics directly into HubSpot, Salesforce, and Zendesk, according to Aircall.
Microsoft's Conversation Intelligence app provides a Notes tab within the call summary page that includes intelligent suggestions for action items, commitments, call minutes, and next steps, according to Microsoft. You can add these suggested fields to the summary to structure the follow-up tasks. Use Timestamped call notes: 5 fields before you post when you write the notes.
Gong generates a brief automatically for every processed call, providing a fixed-format summary, according to Gong. This brief is generated from the call transcript, according to Gong. The fixed format ensures that every call summary contains the same core information, making it easier to scan multiple calls in a weekly review.
To integrate these outputs effectively, start by identifying which CRM fields each tool populates. If your tool does not automatically write to your CRM, you may need to manually transfer the summary data or use a connector. Check your CRM's activity timeline to see if the summaries appear as notes or tasks. If they do not, create a manual step in your coaching workflow to copy the summary into the relevant deal record. This keeps the summary, topics, brief, or notes in the review the team already uses.
Check one recorded call today
Open one recorded call that already has a transcript, confirm the license that allows automated scoring, and assign the scorecard to one team or one call purpose.
FAQ: How to set up automated call scoring for sales coaching
Which license is required to use AI Coaching Assistant in Avoma?
Avoma states that AI Coaching is available only to users with a Conversation or Revenue Intelligence license, according to Avoma. Verify your current plan tier to confirm access to this feature.
How does Avoma score binary yes or no questions?
For binary evaluation questions, Avoma assigns a score of 5 for Yes and 1 for No, according to Avoma. This fixed scoring applies specifically to No/Yes question types within the tool.
Can Aircall AI Assist score calls without a transcript?
Aircall allows users to evaluate any call with a recording, but only calls with transcripts benefit from AI evaluation, according to Aircall. Ensure transcripts are generated for calls you intend to score automatically.
What languages does Aircall AI Assist support for scoring?
Aircall supports AI Assist and AI Assist Pro in 24 languages, across two coverage tiers, according to Aircall. Check which tier applies to your specific language needs before configuring scorecards.
How does Gong generate the brief for a call summary?
The brief is automatically generated for every processed call and gives you a fixed-format summary, according to Gong, and the brief is generated from the call transcript, according to Gong.