Enabling an organization to monitor user feedback.
Qualtrics | 2017
context
In 2017 I led the creation of Qualtrics' comprehensive, automated user experience monitoring system across its various SaaS products.
my role
UX Research Manager.
I led a team of 4 researchers in the creation of the system. My specific contributions included proposing the overall approach (based on a single intercept survey a team member created), working with stakeholders, building the system dashboards, and building the system automations (automated messages, tickets, etc.)
goals
track customer experience
observe how customer experience changes
compare products
automate data collection
notify key stakeholders of important trends and feedback
team goal: prioritize research team around generative, foundational research, not usability testing & establish a team culture around data-driven decision making
process
The process for creating the system was similar to most research projects. I interviewed stakeholders to understand their research questions, I worked with research colleagues to determine the best way to answer those questions, I co-authored a survey instrument to collect that data, I piloted the survey and got feedback on the data, I then led a full deployment of the system across all the major products.
how it works
System architecture
The system is represented in the diagram at right. Essentially every product in the Qualtrics offering has an in-product survey injected into it. Those surveys then drive internally-facing dashboards, messages & tickets, and populate a research panel.
customer experience
Let me describe what the customer experiences of the system. The balance between collecting enough data to make the program valuable and not bothering customers was a chief concern. We found that balance in a lightweight invitation that appears occasionally in the bottom of the Qualtrics user interface.
The customer experience
Here's what users/respondents see when they interact with the program.
stakeholder experiences
The value of this program is allowing internal stakeholders to understand customer feedback can monitor and improve the customer experience. Along those lines there are a number of communication channels and actions I set up to make the data and insights actionable.
System dashboards (internal)
Dashboards showing the "health" of the products hangs in the office and updates continually. (This example features fictional data.)
System messages (internal)
Email summaries of response themes, trends, and other data are automatically sent to internal product leadership.
Slack messages
Our organization uses Slack heavily, so I created Slack channels that automatically send in customer feedback so stakeholders can see discuss feedback as it comes in.
Product-centric dashboards
We have dashboard pages for every product that stakeholders can explore and query. (Again, the data is fictional.)
outcomes
The system is used on a continual basis by Qualtrics stakeholders at every level. Customer issues are frequently addressed, and just as importantly, everyone at the company has the ability to remain connected to how customers.
This system was so impactful I was encouraged to speak about it. Below are a couple of examples of external information-sharing I've done on the system.
Check out the case study I wrote for Boxes & Arrows (click to read):
I also presented a case study of the system at the X4 Summit:
feedback