Admin Portal – Calling Metrics Dashboard

Introduction

The Calling Metrics Dashboard provides an overall call quality view that includes call and site statistics, per-call media ratings, and SIP signaling details. Service assurance is included in all packages and on-premise equipment is not required. All devices are supported.

Before you begin

Understanding the Calling Metrics dashboard and the data it provides is key to using it effectively.

• Data is collected for all devices from the SBCs that endpoints register to at the edge of the cloud. For example, if Alice calls an external PSTN number (phone number), the data shown refers to the packets transmitted by Alice’s device and received by the cloud.
• The dashboard collects data from access side SBCs, therefore only calls answered on a registered device will be listed. A call received from an external phone number and redirected to another external phone number is an example of a call that would not be listed in the dashboard.
• The Calling Metrics Dashboard functionality differs from the Rialto Calling Admin Portal (CAP). The Analytics section in CAP lists the entire call detail records (i.e. completed, redirected, cancelled calls), whereas the Calling Metrics Dashboard provides per call quality for those calls that have media sessions.

Launching the Calling Metrics Dashboard

The Calling Metrics Dashboard is accessed through the admin portal. To open the full calling metrics dashboard, follow these steps:

1. Log in to the admin portal.
2. The Service Assurance panel is on the admin portal dashboard.

Click image for large view

Note: If you do not see the Service Assurance panel within your admin portal, contact your service provider.

3. From the Service Assurance panel, select the Calling Metrics Dashboard.

Click image for large view
4. The Calling Metrics Dashboard opens in a new window on the main dashboard screen.

Understanding the Dashboard

The main dashboard gives an at-a-glance visualization of the usage of the system and the media quality experienced by a customer.

The bell notification icon showcases new feature releases available for the Calling Metrics dashboard and provides insight to “What’s New” and a link to the new feature documentation.

Note: Each new feature notification displays for 2 weeks.

Date Range

The Date Range option, located on the main landing page, allows a user to view through day, week or month of data, as needed.

The date range applies for the following graphs.

• Calls for all sites
• Call quality for sites
• Devices

Note: The date range applies only to the main dashboard landing page and does not apply to the “Total Calls” graph. This graph always shows data for the last 30 days.

Examples for date ranges

Choosing multiple days’ worth of data:

In this example, the start date is December 14, 2020 and the end date is December 17, 2020.

Resetting and choosing a different date range:

For choosing a different date range, move your mouse either over the start date or end date. Then click on the close” button to reset the current selection.

Call Quality

• The Poor Calls section shows the number of calls within your domain having MOS scores below 3.2 in the last 24 hours.
• The Good Calls section shows the number of calls within your domain having MOS scores above 3.8 in the last 24 hours.
• The Call Quality section identifies the top poor call sites in conjunction with a call quality breakdown. The call quality is provided in 3 categories within the last 24 hours for an individual site:
• Good (MOS > 3.8)
• Fair (MOS 3.2 to 3.8)
• Poor (MOS < 3.2)

Click image for large view
• The Calls for site & Quality for site sections show a split up for the “Poor, Fair and Good calls”, based on the date range that was selected.
  
  Note: The data populated under these widgets is only based on per-call media sessions that have MOS scores. This has no correlation with the Total calls in last 30 days widget. The values under Calls & Quality for sites should match against the call entries that are listed under the Calling Metrics tab.





Click image for large view
• The Total calls in last 30 days widget indicates all the call sessions that are completed, not-completed & that may have redirected.




• The Devices pie chart displays a breakdown of which devices or apps (user agents) were in use across the selected customer site.
  
  Note: In a call scenario where an MPP calls a UC app, both user agents will be listed on the pie chart.

Click image for large view
• You can view usage on particular sites by using the drop down menu at the top right of the dashboard and making your selection. If a customer has several sites, you can switch from one site to another. Simply begin typing the name of the site to quickly find it, the search is flexible and matches any text from the site name.

Click image for large view
• The graphs and visualizations presented on the dashboard are all exportable in a variety of formats for further analysis outside of the dashboard. To export a graph, select the More Options icon in the top, right-hand corner of the graph you would like to print or download.

Click image for large view

Reading the Calling Metrics View

The Calling Metrics view displays a list of calls made during a given time period with complete call details. You can select a site and time period to view a list of calls and associated media quality information. Columns can be sorted to display calls in order of whichever metric is pertinent to the problem that is being worked.

Within the Customizable Column, the Additional Settings button allows a user to customize the column they want to view under the Call Metrics tab. This provides the ability to add or remove additional fields as well as organize and reorder columns.

To customize your column view, click on the “Additional Settings” tab and select the individual fields from the drop-down menu or click the “Select All” option.

Note: Only fields under the “Additional Settings” tab can be customized (ie, added or removed)
The fields listed below are set to default and cannot be removed.

• Start time
• Duration
• Calling Number
• Caller
• Called Number
• Callee
• Call Quality
• Packet loss (PL) %
• More Info

Example for customized view

For this example, a selection of fields are chosen for viewing under the Call Metrics tab: Call type, Jitter average callee, Packet loss caller and callee.

Download button

The download button for Call Detail Records (CDR) provides the ability to download the contents of CDR metadata. It offers insight on how a specific phone number and/or users are utilizing the phone system and helps to identify their calling trends.

By navigating to the Calling Metrics tab, you can choose any site and date then select the download button. This will download the CDR contents in excel file format.

Note: The downloaded file is not site specific and contains calling trends for all sites. Additionally, only a day worth of data can be downloaded.

Call Type

Call Type Column

The Call Type column in the Call Details list allows you to quickly identify what type of call a particular record is associated with. It is important to note that the only records that are shown on the Call Metrics Dashboard are those that had a media session. The dashboard, by design, does not show unanswered calls. A complete view of call history can be seen under the Analytics tab in the Calling Admin Portal.

Call Type Examples:

1. Call Transfer:
   

   Click image for large view
   • At 09:34 PM, Alice called Bob (+14307973003 -> +14307973004), Call duration 223 seconds.
   • Bob initiated a call transfer to Carol.
   • At 09:36 PM, Alice and Carol were in conversation (+14307973003 -> +14307973005).
   • The dashboard lists two calls because there were two media sessions. First call would be Alice to Bob (09:34 PM) and the second call would be Alice to Carol (09:36 PM).
   • The “Call Type Column” Indicates that this is a “Call Transfer” scenario.
   • The “redirecting number” explains who attempted the transfer. In this case, it would be Bob’s number (+14307973004).
   • The “reason” explains that there was a “deflection” during a course of the call which indicates call transfer.
2. Hunt Group:

Click image for large view
• At 10:27 PM, Dave called a Hunt Group number.
• Alice is an agent on this Hunt Group, and she attended the call.
• Dave and Alice spoke for 121 seconds (+14307973002 To +14307973003).
• The dashboard lists one call as the call had one media session. I.e., Dave (caller) to Alice(callee) at 10:27 PM.
• The “Call Type Column” indicates that this is a “Hunt Group” scenario.
• The “redirecting number” denotes the actual “hunt group” number (+14307973009) which then forked the other agent phone number.

Call Quality and Packet Loss (PL%) Columns

The Call Quality and PL % columns in the Call metrics tab list allow you to quickly identify media quality problems for a particular call.

PL (%) Value vs Packet Loss (call metrics) Graph:

It is essential to understand the differences between the values under the PL % column vs the More Info->Call stream metrics view.

• The stats shown on the dashboard PL % column represent lost packets (i.e. the actual packet loss %).
• The Call stream metrics view attempts to compute further stats and also indicates "Packet Loss" when the packets didn’t actually make it, but were delayed/discarded.

For cases, when the PL % column shows 0 value, whereas the call stream metrics view graphs had packet loss, it is a potential indication that there aren’t necessarily fully lost packets. This just means that its delayed/discarded packets are displayed rather than the lost packets.

Packet Loss Example:

• Alice called Bob
• Alice’s media stream to Bob experienced 2% packet loss
• Bob’s media stream to Alice experienced 8% packet loss
• he PL% columbine the dashboard lists the worst of these two values, in this case 8%

Below is a simple example to demonstrate troubleshooting media quality issues using the Calling Metrics dashboard.

• The dashboard shows a site where poor quality has been measured. Clicking on the “poor calls” takes you to that site immediately.

In the example below:

• The poor calls were clustered during office hours, from 10:42am to 11:09am and were mostly between Alice and Bob.
• Alice’s phone number is +14307973003 and Bob’s phone number is +14307973004.
• For each of the 5 calls listed, the PL% column shows consistent packet loss ranging from 5-8% and, as a result, the call quality was poor.

To troubleshoot further, choose a particular call, click on the More Info column, as shown in the below image, and choose the Call stream metrics graph. This will show the trend of media packets flowing between both Alice and Bob.

• The “X” axis is the call duration and the “Y” axis is the Packet Loss %.
• There are two datasets on the graph, in the below example. The gray line represents media packets flowing from Alice to Bob (+14307973003 to +14307973004).
• The blue line represents the media packets from Bob to Alice (+14307973004 to +14307973003).
• For this call example, it is clear that the media flow from Bob to Alice had issues indicating a potential issue on Bob’s end.
• This would point to there being a problem with the network on Bob’s end, which would require further investigation.

On the Call Details page, the SIP call flow section shows the requests and responses made by the caller and callee for the selected call. Only SIP signalling between the endpoint and the access side SBC are shown.

Select the Call Stream Metrics column to show the Packet Loss over time, if it’s available for a particular call.

Call Statistics

The call statistics view in calling metrics dashboard allows users to troubleshoot media quality issues with individual calls. You can have information related to the call, view its media statistics, identify where the issue occurred and resolve the potential problems.

This view applies only for individual media sessions, and it does not show complex call sessions.

For example, Alice calls Bob who has set up CFA (Call Forward All) to Carol, and Carol answers the call. In this scenario, the view shows that the call occurred between Alice and Carol, as the focus is on troubleshooting the individual media experience rather than the entire call lifecycle.

Note: The calls under this view will only appear after the call ends.

This view helps to identify the problem area by providing all the relevant metrics and cannot necessarily provide you the root cause for a poor call. It is recommended that you look at the tips below to identify various factors and determine the resolution options:

• The end-to-end experience of the user which is determined using reporting End-of-Call Statistics in SIP BYE Message.
• Send or receive metrics from the calling or called user to the calling cloud (media relay point).

Understanding the view

Before we get into the details, the representations below might help you to understand the flow & segments.

The arrows pointing towards Alice’s direction and Bob’s direction are the call quality metrics determined at the device/app end. It denotes the end-to-end experience Alice & Bob had during their call.

Note: These statistics are determined using End-of-Call reporting from the SIP BYE message.

The arrows pointing towards the cloud symbol direction (in the center) are the call quality metrics determined at the Calling Cloud end. It denotes the SBC’s view of the metrics it received from Alice or Bob.

Here is how an example view would look in the “Calling metrics dashboard”

Legends under the Call Statistics view

Example Call flows

1. Basic call media flow:
   • Alice (+12819798003) called Bob (+12819798005).
   • Alice is the calling user on the left and Bob is the called user on the right.
   • Hover any of the arrows at left or right portions to have a look at the gathered metrics.

Click image for large view

Note: The metrics vary during a session depending on the sampling time and variability in the network. The values that are reported by the Device/Apps and that of SBC may not fully align. They should, however, be in close alignment to allow segmentation view along the path.

2. Indication of potential issue on Caller/Calling user (Alice’s) end:
   • Alice (+12819798003) called Bob (+12819798005).
   • There is a problem with the Alice’s network hop(s) that affects the outbound stream (represented in dotted arrow).
   • Alice’s inbound stream though had no issues.
   • Bob’s user experience is poor due to the issue from Alice’s network.
   • Both the measurements gathered at the Calling Cloud (SBC end) and Bob’s device end indicate the same.

Click image for large view
3. Calls made using Non-Cisco device:
   • Alice (+12819798003) called Bob (+12819798005).
   • Both Alice and Bob were using “Yealink” phone models.

Note: Only Cisco Devices/Apps are currently enabled to send the End-of-Call Statistics. For Non-Cisco calls (like Polycoms, Yealinks etc…) call stats are not supported yet. For these call scenarios only the SBCs view of the metrics will be available. Additionally, for the SBC’s view of metrics, both the calling and called users must be registered to the Access side SBC.


Click image for large view
4. Incoming call from a PSTN number:

For calls in and out of an organization where the other party is on the PSTN network. The media is relayed from a PSTN provider.

In this example, the calling user is the other party where the media is incoming from a PSTN provider through the calling cloud to the called user (Bob).

• There is a problem with the other party’s network hop(s) that affects the outbound sending stream (represented in dotted arrow).
• The called user (Bob’s) network, however, did not have any issues.
• Bob’s user experience is poor due to the issue from the other party’s PSTN network.
• Both the measurements gathered at the Calling Cloud (SBC end) and Bob’s device end indicate the same.
• The receive stream metrics of the Calling user/Other party who is on the PSTN network shows "Data unavailable" as it is not supported because these metrics are not transmitted to the Calling Cloud.

Data Retention

Data retention for the Calling Metrics Dashboard is 30 days for general call and media quality data. SIP signalling and Call Stream data is available for 7 days.

Troubleshooting Examples

These examples are common media quality issues and how you would investigate them using the calling metrics dashboard. With these tips, you can quickly narrow down an issue.

Example 1: A single user reports poor media quality

If a single user reports poor media quality, use the service assurance dashboard to do the following:

• Check the user’s site on the dashboard, and by looking at individual call records, determine if all users are experiencing problems?
• If other users are also seeing issues, use the filtering tools to view past calls to see when this issue began and troubleshoot the sites network connectivity, using CScan or other tools.
• If other users are not seeing issues, filter/sort the calls to view that user’s calls only.
• From this list, it should be clear if there is a pattern to their calling issues:
• All calls from a particular device have issues, but calls from other devices are ok.
• Troubleshoot that single device.
• Only calls at busy periods are showing issues.
• This points to bandwidth contention during busy periods.
• Other users using a particular device type are also having issues.
• These devices may be having issues on the customer’s network.

Example 2: A single site shows bad media quality in the admin portal

The calling metrics dashboard can isolate a list of calls for a particular site by filtering on the Call Details page.

• From this list, problem calls can be sorted and the following patterns identified to isolate the problem.
• Are the problem calls coming from the same user?
• Are the problem calls happening every day at the same time?
• Are the problem calls specific to a device type?

Example 3: An entire customer reports bad media quality

• Is the issue across a single site or multiple sites?
• If it is across a single site, there are no issues with the cloud service, and that sites connectivity should be investigated.
• If it is across multiple sites, are other customers also having issues? The BroadCloud Status page should list any active issues with the overall service.

If there are no service issues, the site network connection should be investigated using CScan or other tooling.