In the Define stage of Customer-Driven Quality, your team identifies what should be built, which seems to be a natural fit for customer-driven practices. The marketing, product management, or product owners usually lead this phase; each of these roles are dedicated to understand customer wants and needs. So, where does the test/quality team play a role?
The test/quality team can help the product definition team in two key areas:
- Make meaningful and mindful investments in three categories: growth of the business, satisfying current customers, and investing in technology for the long term.
- Test the requirements with customers, before expending the time and effort to build the product.
Investment Level for Customer Quality
One of the most powerful decisions that can be made during the definition phase is deciding how much of the limited development bandwidth to apply towards three broad categories: product infrastructure or technology, satisfying current customers, and building new value for current or future customers.
These questions are not easy to make, but each category must be considered. Often, the product definition phase is dominated by building value for new customers, but product infrastructure and current customer pain points are important as well.
The product infrastructure must be maintained and improved to prevent future pain points, such as performance or availability issues. The quality team can make a difference by measuring the technical debt and communicating with all of the stakeholders, including the product definition team.
It’s important to communicate with business stakeholders in business language. They might not understand or appreciate the lack of test automation coverage, but will appreciate the opportunities to reduce the development cycle-time. Learn to speak the language of your stakeholders.
Measuring the customer feedback mainly drives solving current customer pain points. Satisfaction surveys, Net-Promoter scores, and Voice of the Customer are the measures, and we should dedicate some effort to satisfy current customers (who are likely to be influencing future purchase decisions).
A few ways of managing this prioritizing the current customer activities include:
- Applying a “tax” of development time before considering new features; keeping some development resources in reserve so they can concentrate on satisfying current customers.
- Managing requirements and features in a common prioritized backlog along with infrastructure and customer issue. Having a common repository will help the team make these tradeoffs.
- Periodically, a “customer love” or “net promoter” release can be useful, where the entire focus is just solving pain points.
Testing requirements before building product
Customer-Driven Quality does have a natural predator, the HIPPO. Not the aquatic mammal from Africa, but the “Highly Paid Person’s Opinion”. The HIPPO is a term we use to remind ourselves that it’s the customer’s opinion that counts, not the boss’s.
Brainstorming is also used to bring out the best ideas from the whole team. Too often, the results of the brainstorming sessions are not the best ideas, but driven from the most charismatic participants.
Instead of building the feature set that is desired by the influential people in the organization, the customers could help decide what we build. One set of practices that features customer learning comes from the Lean Startup arena.
The Lean Startup process can be thought of as a product management process, and being tangentially related to “quality.” However, a better way to think about it: testing the requirements before the product is built. This is where the test team can play a strong role.
If you think about the idea for a new product or a new feature as a hypothesis, you can use the scientific method to validate the hypothesis through testing, and using Lean Startup terminology, the testing results in Validated Learning.
Validated Learning is all about achieving results from customer behavior, not just what they say they would do. Too often, when talking with a customer (or potential customers), and asking if they would value a feature, the answer is “yes”. Especially in face-to-face encounters, the customers (who are nice people) agree with the idea. Early surveys show tremendous interest, but eventual sales fail to get even close to those projections. Learning from actual customer behavior is intended to eliminate Heisenberg or Hawthorne effects from the learning.
Illustrating customer-driven testing with an example, suppose the product team believes customers want to import their contact list from their email account.
The HIPPO way: The VP uses outlook for email, so we build a utility to extract contacts from Outlook and an upload service. We build a training video and help guide for customers. This launches in 3 months.
The traditional way: The marketing team sends a survey out asking customers if they want to import their contact list from email, and asks which email service they use. In one week, the team knows the percentage of customers who say they will import contacts, and a distribution of email services.
The customer-driven way: The team adds a link which says “import contacts from your email”, and a landing page with a drop-down list of email providers. In one day, you have information from customers who actually tried to import their contacts. Of course, if no one ever clicked the link, you learned sometime extremely valuable: customers don’t care about importing contacts. In this case, you saved 3 months of development (and all of the test cases and bugs that come with that much development).
The customer-driven way is more powerful because it provides information on actual customer behavior, not their intent, and not influenced by trying to be nice. It also has the chance to provide results much quicker than a survey, and almost certainly quicker than building the boss’s solution.
In the preceding example, the validated learning came from customers clicking the “import contacts” link, not knowing it was a test. The relative power of validated learning varies with the level of investment or commitment made by the customer. In our example, the customer invested some time to attempt the import contacts and to select their provider from a drop down. This is a relatively low level of investment. Other examples, in descending order of commitment are:
- Pre-purchasing a product or upgrade.
- Intending to purchase/upgrade by entering a credit card. Of course, if you don’t have a product to sell yet, don’t actually charge the card. (also called a dry test)
- Providing personal information, like email, phone, or login credentials, and signing up to be contacted later. This can come in a message like, “Thanks for your interest, please fill out the form to learn when this is available”
- Clicking the link, and filling out a survey.
- Clicking a link, then clicking a second “learn more” page.
Using the “dry test” method can get tedious if over-used. Use caution and only use this when you are experimenting for a very large investment.
User Stories not Requirements
In the traditional model of software development, a special team of people, called business analysts or product managers, talk to customers, understand their needs, and document those needs in the formalized language of requirements.
Great care is taken to write the requirements in a manner that facilitates traceability, completeness, and abstracts any hint of implementation. When reading these formal requirements, it’s frequently easy to miss the point completely.
Documenting the product requirements in the form of User Stories helps keep the focus on customers, and removes an opportunity for ambiguity. User Stories can also be read and understood by customers
In the product definition stage, the quality/test team are usually recipients of the definition, not participants in the process. These practices show several ways in which the quality/test team can make a difference in defining the right product. The next post in this series will show several practices for including customers in the design and construction phase.
Visit the main page for more customer-driven quality practices or continue on to learn about building quality into your product.
