Measuring Customer Satisfaction

As obvious as it may seem, and as important as it is, it is surprising how few companies take customer opinion into account. Jupiter Media Metrix noted this lack of interest in their "Understanding Customer Loyalty" report:

Customer loyalty, as many companies currently define it, is a flawed metric: It relies only on easy-to-grasp monetary data such as customers' spending habits and order values. Because most businesses focus on these quantitative measures to identify loyal customers, they too often identify only high-transaction customers as loyal ones. A Jupiter Executive Survey found that 63 percent of Web sites define their loyal customer segments-and the subsequent value they place on these relationships-by customers' spending habits and order values. However, customers tend to be loyal to merchants that win their trust over time via a series of positive events, and many satisfied and loyal customers whose spending is not in the high-dollar category can evade a company's radar screen altogether. To more accurately identify loyal customers, companies need to incorporate rich data that provide a broader view of who their customers are, such as behavioral and satisfaction data; just 13 percent of respondents to the survey said they use satisfaction data when identifying loyal customers.

The incomplete approach to identifying loyal customers companies use currently alienates valuable but lower-spending clients who, through their recommendations to others, may provide a low-cost means of customer acquisition.

Very few companies are looking at what their customers have to say about offered services.

According to a Modalis Research Technologies (www.modalis.com) study commissioned in August 2001 by WorldCom, 90 percent of U.S. consumers have used the phone for customer service, but only 46 percent gave it an effectiveness vote of "high satisfaction." On the other hand, of the 12 percent of consumers who have used online chat for help, 62 percent were highly satisfied.

While more and more customers are finding Web customer service more satisfying, few companies are asking customers how those services can be made better. Odd. For heaven's sake, ask your customers what they think!

How You Are Perceived

Dave Taylor, president and founder of Intuitive Systems (www.intuitive.com), always reminds his clients to get direct feedback about their Web efforts from their customers:

• Is the site well designed and easily understood?

• Are problem reports promptly followed up?

• Are problems quickly resolved?

• Are customers able to get a high priority when they have a real emergency?

• Are customers given training opportunities?

• Are they informed of system and product changes-software upgrades, bug fixes, etc.?

At this point, it's no longer about logs and charts and statistics. It's about perception. If your customers don't feel they're getting the information they need or getting the attention they deserve, it doesn't matter if your logs show more pages viewed this week than last. So if you really want to know what your customers think about your Web site, ask them. You can easily incorporate online surveys ranging from a simple plus or minus on the page to an all-out telephone blitz.

Plus or Minus

There's an interesting feature on the OpinionLab home page. In the lower-right corner, there's a little "[±]" hovering on the page. As you scroll up and down, it floats back to its place in the bottom right corner. When you roll your mouse over it, up pops a very succinct questionnaire

Pop-Up Surveys

Popping up a survey is a time-honored tradition, and companies like Gomez (www.gomez.com) and BizRate.com are well-known players in that field. They can provide services and aggregated results suitable for benchmarking. Everybody's experience adds to the mix.

Steve Robinson from Xerox likes pop-up surveys as a touchstone: "From a metrics standpoint we're trying to gauge usefulness on our driver download capabilities. So what we did is put an online survey on there that asks 'Was the process of downloading a driver as expected or less than expected or better than expected?' with a simple click-box that they can check off. We also provide a place for them to add text if they feel strongly enough to send us a comment. I surf through them to figure out what kind of enhancements and changes we should think about."

Kris Carpenter at Excite is well versed in the fine art of customer surveys for checkups: "We have a real-time survey tool that will just pop up and have either one or two quick questions that can be quickly answered, and the idea is we have a way to get some very immediate feedback; we can run it for a couple of hours or a day or a handful of days, whatever we think is appropriate to get enough body of information. Those we use pretty regularly to test new features or to just get quick responses on how satisfied you are with our service.

"But we also rely on the third-party research to help us look not just at our own responses collected through direct interaction with our audience, but having a third party ask someone about how they feel about the experience they got on Excite and how it compares to similar sights they've visited and so forth. So, you get another level of detail there .... And you know, in every case there's always a little bit of a disconnect between what consumers say and what they actually do, but given both of those dimensions and the customer responses that we get through email and so forth, we're able to get a pretty good body of data from the consumers themselves."

An e-business director from one energy company told me that they have eight separate surveys on their site, depending on what kind of transaction you perform. But she was more interested in the cross-functional results. "Our research group samples from all of the contacts that we have had with customers by channel, whether it was a Web contact or a call center contact or an office contact. So they standardize that satisfaction questionnaire across the different channels. They are very comparable numbers and the Web site quality numbers are actually slightly higher than the others."

I can hear her smile.

"I think that," she continues, "as good as some of my associates are, you can't quality control that experience as well as you can on the Web site. I think we deliver more consistent service electronically."

Then she told me one of those statistical validations to one of my general, unfounded convictions.

"We did a one-time research study online and we saw a really interesting and statistically valid trend. Regardless of what they did on the Web site, if they did anything, they were more satisfied with the company at large. It significantly improved their desire to stay with us. So those are the kinds of things that continue to fuel my funding."

While measuring customer satisfaction is an industry unto itself and it would be silly for me to try and duplicate all of the philosophies and findings here, there are three points I'd like to make before moving on:

1. Ask questions about your questions.

2. Synthesize your answers into business outcomes.

3. Go straight to the horse's mouth.

Ask Questions about Your Questions

I first saw this used at Cisco and then discovered that it's a time-honored tradition. But I was so impressed at the time that I make sure to impress it upon others: When you ask how well things are going, be sure to ask how important it is that things go well.

I recently emailed Dell a tech support question. After the issue was settled, I received an email inviting me to take a survey online that included not only questions about how satisfied I was but also questions about how important it was that I be satisfied. Dell wanted to know if the speed with which they answer their email is any more important than their ability to solve my problem.

This gives the organization the ability to determine which things that make customers unhappy are the most important to fix first.

Besides threaded discussion groups, PeopleMetrics does online surveys and they do them very well. After getting his Ph.D., President Sean McDade spent several years at Gallup Consulting Group. What do they do well? Allowing you to analyze the data.

It's nice to know that 65.33 percent of the employees you talk to are satisfied or very satisfied with the company as a place to work. But how do those numbers change when the employee has had, say, more than 6 days of training in the prior 12 months?

Synthesize Your Answers into Business Outcomes

Asking questions and pouring over the answers is interesting, but you can't just hope to trip over a golden nugget, even if they do call it data mining. Sean McDade offers some good advice about asking questions: Make sure the answers roll up into something meaningful. One December morning, over a couple of cups of hot chocolate overlooking the Santa Barbara coast, McDade pulled out an example of an analytical model for a software company. It showed how satisfaction with the sales process was dependent on technical skill, product knowledge, and market intelligence. Product satisfaction was dependent on software performance, ease of obtaining information, and server scalability, and so forth. Here, with a tip of the hat to Sean McDade, is a model of customer satisfaction of a Web site.

This model provides checks and balances to ensure the recording of accurate attitudes. You know what your revenue and profitability are. You know what your retention looks like. Compare those numbers to the way people answer questions about overall satisfaction, but don't stop there. Which of the people/process drivers is having the biggest impact on overall satisfaction? Find out by asking people to rate your visibility, usability, content, and so on. By mixing in and jumping back and forth from questions about your navigation, your knowledge base, your search engine placement, your voice-over-IP services, and so forth, you create a much richer set of verifiable opinions about your site.

Go Straight to the Horse's Mouth

There is a whole wealth of information to be garnered from people on a one-to-one basis, even if it's over the telephone.

This was brought home to me recently while doing a series of interviews with Web masters. I had retained English & Associates in Nashville, Tennessee, to make the calls, and, as part of their standard practice, they had provided a written transcript and an audio tape of each conversation. While statistical survey results are great for pie charts and bar charts, and a transcript is great for pulling out specific, pithy quotes, the audio tapes were golden. Being able to hear the tenor of people's comments revealed not just what people were saying and what they were thinking, but how they felt. Standard customer satisfaction style surveys are very valuable, but there's nothing better than hearing it straight from the horse's mouth. Derrith Lambka founded Insights for Action (www.insightsforaction.com) to become a conduit of personal feelings from customers to the companies they buy from. Her team goes to your customers where they live and work to find out how they think and feel. Insights for Action is a trained ear that listens hard to your customers because, as Lambka puts it, "Customers are great at identifying problems. We are great at generating new and innovative ideas to solve those problems. We become 'investigative reporters' about customers. We uncover what they hate, what they see as a minimum requirement, a nice to have, and something you could do that would really 'wow' them."

The result is that they are well equipped to define and design optimal customer experiences that make it easier, faster, and more enjoyable for customers to find and buy products. "I've seen researchers who knew a lot about customers, but after they'd presented their information and written their report, the binder sat on a shelf and wasn't acted on," says Lambka. "What was missing was putting the insights from the research into a practical application for the customer."

One-on-one customer interviews with a video camera bring faceless customers to life. It's part Q&A, part empathy, and part Web site usability testing. If you're not hearing your customers speak for themselves, you should talk to Lambka.