user experience monitoring system.

2017 | Qualtrics

 

goals

  1. track customer experience

  2. observe how customer experience changes

  3. compare products

  4. automate data collection

  5. notify key stakeholders of important trends and feedback

  6. team goal: prioritize research team around generative, foundational research, not usability testing & establish a team culture around data-driven decision making

context

In 2017 I led the creation of Qualtrics' comprehensive, automated user experience monitoring system across its various SaaS products. 

 

 

 

 

 
 

my role

I led a team of 4 researchers in the creation of the system. My specific contributions included proposing the overall approach (based on a single intercept survey a team member created), working with stakeholders, building the system dashboards, and building the system automations (automated messages, tickets, etc.)

process

The process for creating the system was similar to most research projects. I interviewed stakeholders to understand their research questions, I worked with research colleagues to determine the best way to answer those questions, I co-authored a survey instrument to collect that data, I piloted the survey and got feedback on the data, I then led a full deployment of the system across all the major products. 

how it works

 

System architecture

The system is represented in the diagram at right. Essentially every product in the Qualtrics offering has an in-product survey injected into it. Those surveys then drive internally-facing dashboards, messages & tickets, and populate a research panel.

in-product+diagram.png
 
 
 

customer experience

Let me describe what the customer experiences of the system. The balance between collecting enough data to make the program valuable and not bothering customers was a chief concern. We found that balance in a lightweight invitation that appears occasionally in the bottom of the Qualtrics user interface.

 

The customer experience

Here's what users/respondents see when they interact with the program.

X4_demo_short.gif
 
 
 

stakeholder experiences

The value of this program is allowing internal stakeholders to understand customer feedback can monitor and improve the customer experience. Along those lines there are a number of communication channels and actions I set up to make the data and insights actionable.

 

System dashboards (internal)

Dashboards showing the "health" of the products hangs in the office and updates continually. (This example features fictional data.)

kiosk.jpeg
 
 

System messages (internal)

Email summaries of response themes, trends, and other data are automatically sent to internal product leadership.

Screen+Shot+2018-02-23+at+3.48.09+PM.png
 
 

Slack messages

Our organization uses Slack heavily, so I created Slack channels that automatically send in customer feedback so stakeholders can see discuss feedback as it comes in.

Slack+message+snippet.png
 

  

Product-centric dashboards

We have dashboard pages for every product that stakeholders can explore and query. (Again, the data is fictional.)

full+page+sample.png
 
 
 

outcomes

The system is used on a continual basis by Qualtrics stakeholders at every level. Customer issues are frequently addressed, and just as importantly, everyone at the company has the ability to remain connected to how customers.

This system was so impactful I was encouraged to speak about it. Below are a couple of examples of external information-sharing I've done on the system.

 

Check out the case study I wrote for Boxes & Arrows (click to read):

I also presented a case study of the system at the X4 Summit:

feedback

 

 

"Jerrod is excellent in front of an executive team and working on high level product direction, but also great hands-on, and everything in between. He is a great team leader & builder and excellent at both managing up and down.


-Head of UX, Qualtrics