user experience monitoring system.

2017 | Qualtrics

 

context

In 2017 I led the creation of Qualtrics' comprehensive, automated user experience monitoring system across its various SaaS products. The goals for the project:

  1. track customer experience across several dimensions,
  2. observe how customer experience changes over time and as a result of product enhancements
  3. compare products across any factor,
  4. do all of the above with automatic data collection and presented on a dashboard, and
  5. notify key stakeholders of important trends and feedback so they can act on the data

My goals for the team:

  1. prioritize research team around generative, foundational research, not usability testing & establish a team culture around data-driven decision making

my role

I led a team of 4 researchers in the creation of the system. My specific contributions included proposing the overall approach (based on a single intercept survey a team member created), working with stakeholders, building the system dashboards, and building the system automations (automated messages, tickets, etc.)

process

The process for creating the system was similar to most research projects. I interviewed stakeholders to understand their research questions, I worked with research colleagues to determine the best way to answer those questions, I co-authored a survey instrument to collect that data, I piloted the survey and got feedback on the data, I then led a full deployment of the system across all the major products. 

how it works

System architecture

The system is represented in the diagram at right. Essentially every product in the Qualtrics offering has an in-product survey injected into it. Those surveys then drive internally-facing dashboards, messages & tickets, and populate a research panel.

in-product+diagram.png
 

customer experience

Let me describe what the customer experiences of the system. The balance between collecting enough data to make the program valuable and not bothering customers was a chief concern. We found that balance in a lightweight invitation that appears occasionally in the bottom of the Qualtrics user interface.

The customer experience

Here's what users/respondents see when they interact with the program.

X4_demo_short.gif
 

stakeholder experiences

The value of this program is allowing internal stakeholders to understand customer feedback can monitor and improve the customer experience. Along those lines there are a number of communication channels and actions I set up to make the data and insights actionable.

System dashboards (internal)

Dashboards showing the "health" of the products hangs in the office and updates continually. (This example features fictional data.)

kiosk.jpeg
 

System messages (internal)

Email summaries of response themes, trends, and other data are automatically sent to internal product leadership.

Screen+Shot+2018-02-23+at+3.48.09+PM.png
 

Slack messages

Our organization uses Slack heavily, so I created Slack channels that automatically send in customer feedback so stakeholders can see discuss feedback as it comes in.

Slack+message+snippet.png

  

Product-centric dashboards

We have dashboard pages for every product that stakeholders can explore and query. (Again, the data is fictional.)

full+page+sample.png
 

outcomes

The system is used on a continual basis by Qualtrics stakeholders at every level. Customer issues are frequently addressed, and just as importantly, everyone at the company has the ability to remain connected to how customers.

This system was so impactful I was encouraged to speak about it. Below are a couple of examples of external information-sharing I've done on the system.

Check out the case study I wrote for Boxes & Arrows (click to read):

I also presented a case study of the system at the X4 Summit:

feedback

 

"Jerrod is excellent in front of an executive team and working on high level product direction, but also great hands-on, and everything in between. He is a great team leader & builder and excellent at both managing up and down.


-Head of UX, Qualtrics