Several of the Customer-Driven Quality practices involve testing customer behavior while the customers are not aware they are part of a test. These practices include:
- A/B Testing
- Dry Testing
- Usage Analytics
- Behavior Tracking
- Staged Deployments
Meeting with and engaging customers is very valuable, but sometimes you hear what you want to hear. Customers are people too, and sometimes they tell you want you want to hear. This video shows a great example where interviews can sometimes lead to misleading results:
As you see in the video, these people answered questions about an event that never happened. Typical customer engagements, like design sessions, interviews, and usability and beta tests do have a Heisenberg effect. The test affects the output.
Its important to also observe and test with customers, while they don’t realize they are part of the test.