Skip to main content

Deep Analysis report for conversation quality

In this tutorial you build a Deep Analysis report that scores the quality of an agent's conversations in Flametree. You write an analysis configuration that classifies every session by customer satisfaction and issue resolution, run it over the agent's history, and read the resulting dashboard down to the individual chats behind each number.

The example continues with EventHub, the fictional online-events company from the Inbound Customer Service Agent tutorial. EventHub Assistant has been answering customers for a while, and you want to know how well those conversations are going — not how many there were, but how many ended with a satisfied customer and a resolved issue. By the end you have:

  • A Deep Analysis configuration attached to the agent that scores Customer satisfaction and Issue resolution.
  • A calculation run over the agent's recent sessions.
  • A dashboard you can read and drill into — trend charts, summary cards, and the conversations behind each result.

Plan 30–45 minutes, plus background calculation time that depends on how many sessions the agent has. Each stage gives only the steps you need and links to the page that covers it in full.

Analyzing recorded calls instead?

This tutorial scores the quality of conversations a live agent already had. To turn a folder of recorded phone calls into structured data and then analyze the batch, see Call Analytics for Recorded Calls.

Before you start​

  • You are signed in to the portal. Configuring and running an analysis requires a role that can edit the agent — see Roles & permissions.
  • An agent that has had real conversations. This tutorial reuses EventHub Assistant from the Inbound Customer Service Agent tutorial; any agent with sessions in the last month works.
  • A working knowledge of the configuration format helps. This tutorial writes the YAML inline and links each field to the Deep Analysis YAML reference rather than re-explaining it.

Stage 1: Plan the report​

Decide what "conversation quality" means before you write any YAML. A Deep Analysis configuration extracts metrics from each conversation and charts them, so each quality question becomes one metric.

For EventHub, score two things:

  • Customer satisfaction — was the customer satisfied at the end? A fixed-choice metric with three values (satisfied, neutral, dissatisfied) you can chart as a distribution.
  • Issue resolution — did the conversation resolve the customer's issue? A fixed-choice metric with two values (resolved, unresolved).

You also want to read why a conversation was scored the way it was, so add a short free-text reasoning field that the report attaches to each conversation in the drill-down.

Keep the first version small. As the Deep Analysis page recommends, start with one or two clearly defined category metrics, run them, read a handful of sessions to confirm the scoring matches your judgment, then extend.

Stage 2: Write the analysis configuration​

The configuration is a YAML document with three required sections: scheduling_rules (when to run and how much history to process), analysis_types (the metrics to extract), and visualization (the tabs and charts). The full field reference is the Deep Analysis YAML reference; the configuration below uses it.

Build up the EventHub configuration section by section.

Scheduling

Run daily and process the last 30 days of conversations:

scheduling_rules:
  cron_exp: "0 0 * * *"
  depth: 30

cron_exp is when the scheduled run starts, and depth is how many days of history each run processes — conversations older than depth days are not evaluated. See Scheduling rules.

Metrics

Define one analysis task that scores satisfaction, resolution, and a reasoning field. Each Literal metric lists its values with a display name and a chart color; the reasoning field is a str, which is recorded per conversation but not charted as a distribution:

analysis_types:
  - id: "ConversationQuality"
    order: 0
    description: "Score satisfaction and issue resolution from the conversation"
    prompt_template: |
      # Task
      {task}
      # Chat history
      {chat_history}
      # Response JSON schema
      {formatting}
    prompt_parts:
      task: |
        Score the quality of this customer conversation:
        customer satisfaction, whether the issue was resolved,
        and a short reason for the scores.
    llm_metrics:
      - id: "satisfaction"
        name: "Customer satisfaction"
        kind: "llm"
        type: "Literal['satisfied','neutral','dissatisfied']"
        description: "Customer satisfaction at the end of the conversation"
        prompt: |-
          Classify customer satisfaction:
          - satisfied: the customer expressed gratitude, positive feedback, or got what they needed
          - neutral: the customer ended without clear positive or negative sentiment
          - dissatisfied: the customer expressed frustration, a complaint, or left without help
          Return exactly one label: satisfied, neutral, or dissatisfied.
        values:
          - id: "satisfied"
            name: "Satisfied"
            color: "green"
          - id: "neutral"
            name: "Neutral"
            color: "gray"
          - id: "dissatisfied"
            name: "Dissatisfied"
            color: "red"
      - id: "resolution"
        name: "Issue resolution"
        kind: "llm"
        type: "Literal['resolved','unresolved']"
        description: "Whether the conversation resolved the customer's issue"
        prompt: |-
          Classify whether the customer's issue was resolved:
          - resolved: the customer's question was answered or their problem was solved
          - unresolved: the customer left without an answer, or the conversation was handed to a human
          Return exactly one label: resolved or unresolved.
        values:
          - id: "resolved"
            name: "Resolved"
            color: "green"
          - id: "unresolved"
            name: "Unresolved"
            color: "red"
      - id: "reasoning"
        name: "Reason"
        kind: "llm"
        type: "str"
        description: "Short reason for the satisfaction and resolution scores"
        prompt: |-
          In one sentence, explain the satisfaction and resolution scores.

Field by field — id, name, kind, type, description, prompt, and values: LLM-based metrics. The {task}, {chat_history}, and {formatting} placeholders in prompt_template: Prompt template placeholders.

Visualization

Show one tab with a trend chart and a summary card per metric. Bind each chart to a metric by its id, count conversations with the unit feature, and attach the reasoning metric to the satisfaction card with additional_info so its text shows under each conversation in the drill-down:

visualization:
  tabs:
    - id: "quality"
      title: "Conversation quality"
      plots:
        - id: "satisfaction_trend"
          kind: "trend"
          type: "bar_time_series"
          title: "Satisfaction over time"
          metrics:
            - id: "satisfaction"
              features:
                - id: "unit"
                  aggregation: "sum"
        - id: "satisfaction_summary"
          kind: "summary"
          type: "detailed_bars"
          title: "Satisfaction distribution"
          metrics:
            - id: "satisfaction"
              features:
                - id: "unit"
                  aggregation: "sum"
          additional_info: "reasoning"
        - id: "resolution_summary"
          kind: "summary"
          type: "detailed_bars"
          title: "Resolution rate"
          metrics:
            - id: "resolution"
              features:
                - id: "unit"
                  aggregation: "sum"
                  percentage: true

Tabs, plots, the kind/type chart styles, the unit feature, aggregation, percentage, and additional_info: Visualization. The color names you can assign to values: Colors.

note

Indent with exactly two spaces, and define each classification value with explicit criteria, as above — vague prompts produce inconsistent scoring.

Stage 3: Attach the configuration and run it​

The configuration lives in the agent's Analytics section in Advanced mode.

  1. Open EventHub Assistant from AI Agents > Agents and switch on the Advanced mode toggle in the upper-right corner.
  2. Scroll to the Analytics section and paste the full configuration from Stage 2 into the editor.
  3. Run the calculation with the buttons in the section header:
    • Click Recalculate to save the configuration and process all sessions from scratch. Use this the first time and after you change metric definitions.
    • Click Continue calculation to save and process only sessions not yet analyzed — use it later to pick up new conversations.
    • Stop halts a calculation in progress; it is available only while a calculation is running.
  4. Watch the calculation status next to the section title. Hover over the question icon next to it to read the status message.
note

The analysis evaluates each conversation once. The scheduled run (cron_exp) and Continue calculation both pick up only sessions that have not been analyzed yet; Recalculate reprocesses everything.

How the analysis runs and the calculation buttons in full: Configure an analysis. If you attach an analysis to a campaign instead of an agent, you upload the YAML from the campaign's YAML Analytics dialog — see For a campaign.

Stage 4: Open the report​

When the calculation finishes, the configuration drives the Deep Analysis dashboard.

  1. In the left menu, open Analytics > Deep Analysis. (You can also reach it pre-filtered for the agent with the View Deep Analysis button on the Agents list.)
  2. Keep the analytics type on Agents and pick EventHub Assistant in Select agent.
  3. Set Start date and End date to cover the period you want — it defaults to the last month — and choose a granularity of Day, Week, or Month for the trend charts.
  4. The Conversation quality tab from your configuration opens. If the screen shows No data available., the calculation has not produced results yet — return to Stage 3 and run Recalculate or Continue calculation.

The header controls, tabs, and how a configuration maps to the screen: Find your way around.

Stage 5: Read the charts​

The Conversation quality tab shows the charts you defined.

  1. Read Satisfaction over time: one bar series per satisfaction value — Satisfied in green, Neutral in gray, Dissatisfied in red — grouped by your granularity. Hover over a period for the exact values, and click a legend entry to hide or show a series.
  2. Read Satisfaction distribution below it: one row per value with its total and a proportional bar for the whole selected period.
  3. Read Resolution rate: because its metric uses percentage: true, each row shows the share of conversations that were Resolved or Unresolved rather than a raw count.
  4. Hover over a row name to read the metric description from your configuration.

Trend charts and summary cards in full: Read the charts.

Stage 6: Read the conversations behind a score​

A score is only useful if you can see the conversations behind it. This is the everyday workflow: spot an unusual number, open the conversations, and read a few before deciding what to change.

  1. On the Satisfaction distribution card, click Details. A full-screen view opens with three panels: Category, Sessions, and Chat.
  2. In Category, click Dissatisfied. Until you pick a value, the middle panel shows Select a metric to view sessions.
  3. The Sessions panel lists the matching conversations, with the count next to the panel title. Because you set additional_info: reasoning, the model's one-line reason appears under each row. Click Load more to fetch more.
  4. Click a session to read it in the Chat panel.
  5. Use the buttons in the Chat header — both open in a new browser tab, so you keep your place:
    • Go to session — opens the conversation on the Sessions screen, where you can also open logs and traces.
    • Open in 360 — opens the customer's profile when the session is linked to one. See 360 View.

The drill-down panels in full: Open the sessions behind a number.

tip

Read three or four dissatisfied conversations against the model's reason text. If the scoring matches your judgment, trust the chart; if it does not, tighten the prompt for that metric in Stage 2 and run Recalculate.

Stage 7: Export a follow-up list​

Turn any scored category into a contact list — for example, every customer whose issue was left unresolved, ready for a follow-up.

  1. With Details open and a category selected — for example Unresolved on the Resolution rate card — click Download contacts in the Sessions panel.
  2. The portal downloads a CSV file with the contacts behind the listed conversations.

The export covers the rows currently loaded, so click Load more first if you want the full set for a large category.

Exporting and what the file contains: Export contacts.

What you built​

  • A Deep Analysis configuration attached to EventHub Assistant that scores Customer satisfaction and Issue resolution and records a reason for each score.
  • A calculation run over the agent's recent sessions, refreshed with Continue calculation as new conversations arrive.
  • A Conversation quality dashboard with a trend chart, summary cards, and a drill-down that links every score back to the conversations and the reason behind it.
  • A CSV export of the customers behind any score, ready for a follow-up campaign.

To analyze recorded phone calls instead of live conversations, see Call Analytics for Recorded Calls. To read the ready-made volume reports that need no configuration, see Dashboards.

Common issues​

SymptomLikely causeFix
The Deep Analysis screen shows No data available.No calculation has produced results yet.Open the agent's Analytics section and click Recalculate (Stage 3).
Recalculate and Continue calculation are disabled.The editor is empty.Paste a configuration into the Analytics editor first — the buttons enable once it has YAML.
The calculation status shows an error.The configuration is invalid.Hover over the question icon next to the status to read the message, fix the YAML, and run Recalculate — see Configure an analysis.
Stop does nothing or is unavailable.No calculation is running.Stop is enabled only while a calculation is in progress.
Recent conversations are missing from the charts.They have not been analyzed yet, or the End date is too early.Run Continue calculation and check that End date includes today.
Older conversations are missing.They fall outside the depth window or the selected dates.Increase depth in scheduling_rules and run Recalculate, and widen Start date/End date.
The Details view shows Select a metric to view sessions.No category value is selected.Click a value in the Category panel first.
A series is missing from a trend chart.A struck-through legend entry is hidden.Click the legend entry to show the series again.
The scoring does not match your judgment.The metric prompt is too vague.Tighten the value criteria in the prompt (Stage 2) and run Recalculate.

Was this article helpful?