The Parallel Worlds of Vendors and Users

Just came across a fascinating study by Dimension Data (in collaboration with Cisco) on the perception gap between "vendors" and "consumers" of speech-enabled self service solutions. By "vendors" the study refers to platform developers, system integrators, voice application developers, and speech technology vendors. 128 such vendors were surveyed for the study. By "consumers" they refer to callers who have interacted with speech-enabled self-service applications. They surveyed 1,203 such consumers.

The key findings revolve around 6 questions:

(1) How often would you prefer to use a speech recognition system rather than a touch-tone system? 9% of vendors answered "As little as possible," while 45% of users gave that answer. A huge disconnect. On the flip side, 47% of users gave a qualified "Yes" -- that is, they would prefer speech under some circumstances (depending on time of day, where the caller is, etc.), which tells us that users are not necessarily reflexively rejecting speech-enabled automation under all circumstances.

(2) What do you think is the main reason organizations provide automated services in their call centers? 69% of vendors said "to save money" compared to 54% of users. In other words, callers are no dupes: they fully understand what motivates to deployment of these solutions.

(3) What do you think is the most important benefit of using an automated system when you phone a call center? 51% of vendors mentioned "to avoid wait time" while 49% of users mentioned "24 x 7 service" against 18% who mentioned "Avoid wait time"! A remarkable mis-alignment and a clear opportunity for marketers and designers to exploit for increasing adoption.

(4) In general, when you've used a speech recognition system, which of the following best describes how well it helped you deal with your query? 77% of vendors said that it "Partially addressed the reason I called" while only 43% of users did. Another large gap. 2% of vendors responded with, "Did nothing I needed," while 13% users gave that response. Again, another noticeable gap that points to excessive optimism from vendors. On the other hand, only 8% of vendors responded with "Fully addressed the reason I called," while 18% of users gave that answer. In other words, it seems that vendor answers are driven by mushy conservative wishful thinking rather than insight into actual user reception.

(5) Having used a speech recognition automated system, would you now...? 44% of vendors responded with, "Be neutral to use one again" vs. only 28% of users giving the same answers. What is noteworthy is that a greater proportion of users (36%) responded with "Be happy to use one again" vs. 32% of vendors giving that answer, and a greater proportion of users (also 36%) responded with "Be reluctant to use one again" vs. 24% from vendors. In other words, just like question 4, users are more opinionated and have a less neutral disposition than vendors.

(6) The thing that annoys or irritates me most about using an automated speech application is when.... 41% of vendors answered with "System didn't understand me," vendors' number one answer, while users' number one answer was, "Transfer to agent with no context." This is a fascinating disconnect. Only 17% of users responded with, "System didn't understand me." Which simply means that it's not speech recognition that users find annoying or irritating, but the experience with the application: an additional 16% of users said, "Can't skip ahead" and 14% said, "No alternatives". In other words, 67% of dissatisfaction revolves around the experience with the application. Vendors by contrast focused on technology, in this case ASR and CTI ("Transfer to agent with no context" receiving 38%). "Can't skip" received 4% and "No alternative" a mere 1%.

The report gives a couple of general recommendations such as establishing "cross-functional engagement within organizations" and ensuring "contributions from non-technology stakeholders, e.g., marketing, customer services, and usability experts." But that is no revelation to anyone who seriously engages in voice user interface design.

What would have made the study complete would have been including a third category of stakeholders: the companies that deploy these applications -- i.e., the actual customers of the vendors. I suspect that since many of these customers are sold on the value of self-service applications by the very vendors surveyed in the study, a parallel mis-alignment between customer expectations and those of the ultimate users also holds.

The authors promise to run the survey year over year. Let's keep our eyes open. Hopefully, vendors and customers will read the report and will begin to actually align their goals and values along those of end users.

Putting up with Bad Interfaces

A thought I want to jot down and elaborate on in future posts.

Why do people have such low tolerance for bad usability in the IVR but are (or have been) able to endure without a peep of complaint the DOS and UNIX prompt? I don't ever remember anyone saying, "stupid computer! Why can't you understand me when I type 'check disk' instead of stubbornly insisting on 'chkdsk'. " Instead, people usually curse their own "stupidity" and "carelessness," and exclaim how they are just bad with computers and how their 10 year old child was far better than them, and all that.

So, why is that?

One reason, I guess, is the perception that humans have that whatever a human can do, the brainy machine should be able to do faster and better. A human can understand speech and engage in conversations with almost no effort (or so we think), and if a computer can't do it, then clearly there is something wrong with that computer (or, in this case, the IVR technology).

The perception is exacerbated by the illusion well designed VUIs give of sounding and behaving like a human. Which results in a raising of intelligence expectations and a lowering of tolerance for mistakes.

Sometimes I wonder if a VUI that was designed like a DOS command would result in people complaining less than otherwise! Would a VUI that sounded like a robot (but was perfectly intelligible), that tolerated no variations, that sounded like the almighty Computer itself speaking, without a hint of negotiating, who was in complete and unshakable control, that didn't fret or apologize when it made a mistake, that accused the user of making errors when things failed, that responded with things like, "Your response is not recognized as an internal or external command" -- would such a VUI have the effect of having people blame themselves when things didn't work? An experiment worth conducting....

Thoughts on launching a movement

Elaborating on my post of yesterday regarding the troubled waters Gethuman is navigating, I think the key to building a reform movement that will take hold and have a concrete and lasting positive impact is to assemble the right alliances from stakeholders who have a vested interest in pressuring businesses to invest in customer-centric solutions.

If you look back at all progress say in product safety that has been made, in almost every case, it has been a battle between on the one hand businesses that want to protect their short term interests and their margins by resisting anything that leads to production cost increases, and on the other consumers and their advocates who moved to pressure them to build safer products.

The classic example being of course the car seat belt and Ralph Nader's crusade for automotive safety. Initially, there was great resistance to the seat belt, but eventually, companies realized that safety was a competitive advantage they could exploit (and did). And that turned out to be great not only for the companies who went the safety route, but for the automotive industry in general because it opened up a whole new market and new set of customers (safety conscious Moms) who had up to then been excluded.

The stakeholders in our case are consumers and their advocates, VUI designers (we want to have jobs and make money), companies that deploy IVR solutions (they want to build applications that will be adopted have a truly positive impact for their clients), integrators, companies that host the deployments, and companies that buy them and deploy them (our customers).

The challenge in our situation, I believe, is that the ultimate user is not the technology buyer, so, obviously, the interest of the buyer is not aligned with that of the consumer.

A strategy for moving forward, in my view, would look something like this:

(1) Educate consumer groups on the shabby state of deployed automation and support. That shouldn't be too hard given the universal dislike of currently deployed IVR systems.

(2) Educate consumer groups on the possibilities of the technology: they need to understand that a great deal of consumer pain can be alleviated if businesses invested in the deployment of quality speech solutions.

(3) Have consumer advocates pick one or two key features that can be delivered and that are most wanted by the consumer and agitate for their adoption. For example: telling the caller how long they need to wait when they are placed on hold, or never having the caller repeat information they give the agent.

(4) The features in (3) need to be (a) easily implementable (technology exists and it is not too expensive), and (b) easily monitor-able.

(5) Have the consumer groups establish watchdog units that will monitor and mobilize when the key feature in (3) is absent from an application.

(6) Enlist legislators that will serve as a Democles Sword. Nothing mobilizes an industry to do the right thing than the threat of legislated regulation.

(7) Build capacity: i.e., make sure that the supply of VUI designers meets the demand for them.

(8) Once the structures are in place for transferring best practices into actual deployments (after the initial strategic insinuation in mobilizing for (3)) and the investments needed to deploy quality solutions are systematically made, quality of deployments has no
way to go but up.

Gethuman in this scheme could fulfill the role of consumer advocate. They would be the agitators making demands for better solutions from companies that deploy IVR contact center applications. For this to be taken seriously, though, Gethuman would need to tone down its "bypass the IVR" gimick and make demanding better automation its central demand rather than doing away with automation....

Establishing a consortium of some sort that speaks on behalf of businesses that deploy IVR solutions would be a concrete first step to take.