Who are the Experts? The Peritus.ai Forum Benchmark – Part 3

By Robin Purohit

In part 3 of our IT Industry Forum Benchmark blog series, Peritus data scientist Vipul Mathur explains how we classify expert forum contributors. Why are we so interested? Because these experts are the ones creating a great community experience and having the most impact on case deflection.

The goal of our Peritus AI Assistant is to create more community experts and help every expert do even more. Here is the breakdown using Cisco as an example. Coming soon we will publish our analysis for all forums in our data set.

Community Contributor

Any individual who has resolved (by posting the response that was marked accepted) at least one form post.

For instance, in the last one year (approx) Cisco has had 2019 Community Contributors.

Community Expert

These are the subset of community contributors who are above a given (usually 90th) percentile of all community contributors, ranked by the number of replies posted and number of resolved posts.

In other words, to be a Community Expert, an individual needs to have posted more than 90th percentile of replies, and also have more than 90th percentile of resolved posts.

To compare with the count for Cisco mentioned above, there are only 227 individuals who meet this criteria of being a Community Expert.

Community Experts are further classified into three categories, MVP, Rising Star, and High Potential. For the purposes of this classification we determine three midpoints, calibrated across the community experts, as follows.

  1. Replies Midpoint: A point calculated to be splitting the range of reply-counts for Community Experts.
  2. Resolutions Midpoint: A threshold number of resolved posts, that splits the range of posts resolved by Community Experts.
  3. Acceptance Midpoint: Acceptance Rate is the ratio of Resolved Posts divided by Replies Posted by each Community Expert. Expressed as a percentage. The acceptance midpoint is a threshold that identifies experts with high acceptance rates.

The diagram below shows the three zones of classification for the Community Experts.

Expert Zones

The three classes are defined below:

MVP

MVPs (Most Valuable Player) are a subset of Community Experts who have posted higher than mid-point number of replies, and also have higher than mid-point number of resolved posts.

Peritus will help these folks by boosting their acceptance rate, since it is commonly observed that MVPs have lower acceptance rates than other community experts.

Continuing the example from above, Cisco has 33 MVPs in the same time frame.

Rising Star

Those Community Experts who are not MVPs but have a high potential for becoming MVPs in the future. They have posted a lesser number of replies than the Replies Mid Point. However, they have a high acceptance rate, since they are above the Acceptance Mid Point.

Peritus will help these folks quickly boost the number of replies they post, without adversely affecting the acceptance rate.

Continuing the example from above, Cisco has 40 Rising Stars in the same time frame.

High Potential

Community Experts that are neither MVPs, nor Rising Stars, are classified as High Potential. These folks don’t meet the Acceptance Midpoint criteria to be called Rising Stars, and don’t meet the Replies Midpoint and Resolution Midpoint criterias to be called MVPs.

Peritus will help these folks to convert to Rising Stars/ MVPs by helping them boost their acceptance rate and reply count.

Continuing the example from above, Cisco has 154 MVPs in the same time frame.