The Light Bulbs Have Ears: Why Listening Is Voice-Activated Devices' Most Important Skill

If one picture’s worth a thousand words, why is everyone rushing to replace graphical interfaces with voice-activated systems?

The question has an answer, which we’ll get to below. But even though the phrasing is a bit silly, it truly is worth asking. Anyone who’s ever tried to give written driving directions and quickly switched to drawing a map knows how hard it is to accurately describe any process in words. That’s why research like this study from Invoca shows consumers only want to engage with chatbots on simple tasks and quickly revert to speaking to a human for anything complicated. And it’s why human customer support agents are increasingly equipped with screen sharing tools that let them see what their customer is seeing instead of just talking about it.

Or to put it another way: imagine a voice-activated car that uses spoken commands to replace the steering wheel, gear shifter, gas and brake pedals. It’s a strong candidate for Worst Idea Ever. Speaking the required movements is much harder than making them movements directly.

By contrast, the idea of a self-driving car is hugely appealing. That car would also be voice-activated, in the sense that you get in and tell it where to go. The difference between the two scenarios isn’t vocal instructions or even the ability of the system to engage in a human-like conversation. Some people might like their car to engage in witty banter, but those with human friends would probably rather talk with them by phone or spend their ride quietly. A brisk “yes, ma'am” and confirmation that the car understood the instructions correctly should usually suffice.

What makes the self-driving car appealing isn’t that it can listen or speak, but that it can act autonomously. And what makes that autonomy possible is situational awareness – the car's ability to understand its surrounding environment, including its occupant’s intentions, and to respond appropriately.

The same is ultimately true of other voice-activated devices. If Alexa and her cousins could only do exactly what we told them, they’d be useful in limited situations – say, to turn on the kitchen lights when your hands are full of groceries. But their exciting potential is to do much more complicated things on their own, like ordering those groceries in the first place (and, eventually, coordinating with other devices to receive the grocery delivery, put the groceries in the right cabinets, prepare a delicious dinner, and clean the dishes).

This autonomy only happens if the devices really understand what we want and how to make it happen. Some of that understanding comes from artificial intelligence but the real limit is the what data the AI has available to process. So I’d argue that the most important skill of the voice-activated devices is really listening.  That’s how they collect the data they need to act appropriately. And the larger vision is for all these devices to pool the information they gather, allowing each device to do a better job by itself and in cooperation with the others.

Whether you want to live in a world where the walls, cars, refrigerators, thermostats, doorknobs, and light bulbs all have ears is debatable. But that’s where we’re headed, barring some improbable-but-not-impossible Black Swan event that changes everything. (Like, say, a devastating security flaw in nearly every microprocessor on the planet that goes undetected for years…wait, that just happened.)

Still, in the context of this blog, what really matters is how it all affects marketers. From that perspective, voice interfaces are highly problematic because they make advertising much harder: instead of passively lurking in the corners of a computer screen, appearing alongside search results,  larded into social media feeds, or popping up unbidden during TV shows, voice ads are either front-and-center or nowhere. Chances are consumers will be highly selective about which ads they agree to hear, so marketers will need to gain their permission through incentives such as discounts and coupons. Gaining the consent required by privacy regulations such as GDPR* will be good practice for this but it will soon seem like child’s play compared with what marketers need to do on voice devices. So one change is marketers will need a new set of skills around creating aural ads and convincing consumers to agree to listen to them.

A related skill will be making those ads effective. Remember that people are vastly better at processing visual images than words.  That’s why we have the 1000:1 word:picture cliché. That efficiency is why visual ads can be effective even if people don’t focus on them – they are still being registered on some level and people will pay closer attention to those that look interesting at a glance. Aural ads will transfer much less information per moment of attention and chances are most of that information will be forgotten more quickly. We’re in early days here and there’s much to learn. But if you can buy stock in a jingle-writing company, do it.

Another obvious change will be that the device vendors themselves have more control than ever over the messages their customers receive. This gatekeeper function is already at the center of the business models for Amazon, Facebook, Google, Apple and others (increasingly including non-net-neutral broadband operators). But as fewer channels become available to reach consumers and as the channels themselves deliver fewer messages per minute, the value of those messages will increase dramatically. Insofar as separately-controlled devices compete for consumer attention, the device vendors will have even more reason to deliver experiences that consumers find pleasant rather than annoying. Of course, as I’ve argued extensively elsewhere, “personal network effects” make it likely that most consumers will find themselves dealing primarily with a single vendor, so actual competition may be limited.**

The gatekeepers’ control over their customers’ experience means that marketers will increasingly need to sell to the gatekeepers to earn the opportunity to reach consumers. What’s different in a voice-driven world is the scarcity of contact opportunities, which means that gatekeepers don’t have enough inventory (e.g., ad impressions) to sell to all would-be buyers. This isn’t entirely new: even today, impressions for Web display, paid search, and paid social are auctioned to a considerable degree. But a huge reduction in inventory (and the impact of serving ads that lead consumers to opt out, assuming they really have that option) will make the gatekeepers much more selective and, no doubt, raise prices. The gatekeepers will also have more conflicts with potential advertisers as they sell more services of their own, adding yet another level of complexity and more opportunities for deal making.

Finally, let’s come back to the sensors themselves. Assuming that the gatekeepers are willing to share what they gather, marketers will finally be able to understand exactly how consumers are responding to their messages. It’s not just that they’ll be able to know exactly who saw which messages and what the subsequently purchased. The new systems will be collecting things like heart and respiration rates, creating the potential to measure immediate physical reaction to each advertisement.  It almost seems unnecessary to point out that listening devices will also capture conversations where consumers discuss specific products, not to mention their needs and intentions. The grand mysteries of marketing impact will suddenly be exposed with thoroughness, precision. and clarity. The change will be as revolutionary as X-rays, ultra sounds, and CAT scans becoming available to doctors. As with radiology in medicine, these new information streams will require new skills that form the basis of entirely new specialties.

In short, voice-activated devices will change the world in ways that have nothing to do with the interaction skills of chatbots or ease of placing orders on Alexa. Marketers' jobs will change radically, demanding new skills and creating new power relationships. Visual devices won’t really go away – people are too good at image processing to waste the opportunity. But presenting information to consumers will ultimately be less important than gathering information about them, something that will use all the sensors that devices can deploy.

Who knew the sentient housewares in Disney's Brave Little Toaster were really a product roadmap?

__________________________________________________________________________
*General Data Protection Regulation.  Have we reached the stage yet where I no longer need to spell it out?

** The essence of personal network effects is the value of pooling data to create the most complete information about each customer. In many ways, situational awareness is another way of describing the same thing.

The Personal Network Effect Makes Walled Gardens Stronger, But There's Still Hope

I’m still chewing over the role of “walled garden” vendors including Google, Amazon, and Facebook, and in particular how most observers – especially in the general media – fail to grasp how those firms differ from traditional monopolists. As it happens, I’m also preparing a speech for later this month that will touch on the topic, which means I’ve spent many hours working on slides to communicate the relevant concepts. Since just a handful of people will see the slides in person, I figured I’d share them here as well.

In pondering the relation of the walled garden vendors to the rest of us, I’ve come to realize there are two primary dynamics at work. The first is the “personal network effect” that I’ve described previously. The fundamental notion is that companies get exponentially increasing value as they capture more types of information about a consumer. For example, it’s useful to know what’s on someone’s calendar and it’s useful to have a mapping app that captures their locations. But if the same company controls both those apps, it can connect them to provide a new service such as automatically mapping out the day’s travel route.  Maybe you even add helpful suggestions for where to stop for fuel or lunch.

 In network terms, you can think of each application as a node with a value of its own and each connection between nodes having a separate additional value. Since the number of connections increases faster than the number of nodes, there’s a sharp rise in value each time a new node is added. The more nodes you own already, the greater the increase: so companies that own several nodes can afford to pay more for a new node than companies that own just one node. This makes it tough for new companies to break into a customer’s life. It also makes it tough for customers to break away from their dominant network provider.

My best visualization of this is to show the applications surrounding an individual and to draw lines showing how many more connections appear when you add nodes.  If it looks like the customer is trapped by those lines, well, yes.

The point that’s missing from the discussions I’ve seen about walled gardens is that personal networks create a monopoly on the individual level. Different companies can coexist as the dominant networks for different people.  So let’s assume that Google, Facebook, Amazon, and Apple each manage to capture one quarter of the population in their own network. If each member spends 100% of her money through her network owner, the over-all market share of each firm would be just 25%. From a classical viewpoint, that’s a highly competitive market. But each consumer is actually at the mercy of a monopolist.  (If you want a real-life example, consider airline hub-and-spoke route maps.  Each airline has an effective monopoly in its hub cities, even though no airline has an over-all monopoly.  It took regulators a long time to figure that one out, too.)  

In theory the consumer could switch to a new network. But switching costs are very high, since you have to train the new network to know as much about you as the old network. And switching to a new network just means you’re changing monopolists.  Remember that the personal network effect makes it really inconvenient to have more than one primary network provider.

The second dynamic is the competition among network providers to attract new customers. As with any network, personal networks hold a big first mover advantage: whichever provider first sells several apps to the same consumer has a good, and ever-growing, chance of becoming that consumer's primary network.

Once the importance of this becomes clear, you can recognize the game of high-stakes leapfrog that network vendors have been playing for the past two decades. It starts with Amazon in 1994, intercepting buyers before they can reach a physical retailer. A few years later, Google starts catching buyers in the browser, making searches before they’re ready to buy through Amazon. Then Facebook shows up, first with a social network where people discuss their purchases before they make a Web search, and later with a mobile app that bypasses the Web browser altogether. A decade after that, Amazon strikes back with voice search on Alexa, which can happen even before someone types in a social post.

Remember, this isn’t just about selling advertising. Vendors can share that pie. What they can’t share is control over one consumer’s personal network. Since that, in turn, gives control over actual purchases, it’s a much bigger prize and, therefore, worth a great deal more effort to win. Now you see why Amazon has put so much effort into hardware over the years.  It's not just that Jeff Bezos likes cool gadgets.

At this point, you might pause to wonder what happens next.  Is there something that can intercept consumers before they say what they're thinking?  AI- and/or implant-enabled mind reading are certainly possibilities.  But the next frontier right now is subscriptions, which let purchases happen without any specific action for a voice system to intercept.

This is exactly why subscriptions are getting so much attention right now.  (I do need to admit that Dollar Shave Club, Blue Apron, and Birchbox messed up my chronology by launching around 2011, several years before Alexa).

Of course, there’s nothing to prevent the network vendors from launching subscription services. In fact, the price of Blue Apron’s IPO was depressed precisely by the fear that Amazon would enter its business through the Whole Foods acquisition.

But what’s really interesting about subscriptions is they’re less subject to the personal network effect than other types of purchases. A subscription company comes to understand its customers’ needs in one particular area very deeply.  Potentially, it can fulfill those needs better than a company working from less detailed data gathered in other domains.

Certainly a great deal depends on execution.  But if I’ve trained my wine-by-mail company to understand my precise tastes, I’ll probably buy through them when I’m stocking up for my next party, even though Amazon knows I’m planning to have people over and has some general idea that my friends like to drink.

In short, the walled gardens are not impregnable.  Subscriptions might offer a way to help customers escape. But marketers are going to have to work harder than ever to create relationships strong enough to pull their customers away from the networks. All I can do here is to clarify the issues so marketers can better understand the tasks ahead.

Amazon Buys Whole Foods: It's Not About Groceries

Most of the comments I’ve seen about Amazon’s acquisition of Whole Foods have described it as Amazon (a) expanding into a new industry (b) continuing to disrupt conventional retail and (c) moving more commerce from offline to online channels. Those are all true, I suppose, but I felt they missed the real story: this is another step in Amazon building a self-contained universe that its customers never have to leave.

That sounds a bit more paranoid than it should. This has nothing to do with Amazon being evil. It’s just that I see the over-arching story of the current economy as creation of closed universes by Amazon, Facebook, Google, Apple, and maybe a couple of others. The owners of those universes control the information their occupants receive, and, through that, control what they buy, who they meet, and ultimately what they think. The main players all realize this and are quite consciously competing with each other to expand the scope of their services so consumers have less reason to look outside of their borders. So Amazon buys a grocery chain to give its customers one less reason to visit a retail store (because Amazon’s long-term goal is surely for customers to order online for same-day delivery). And, hedging its bets a bit, Amazon also wants to control the physical environment if customers do make a visit.

I’ve written about this trend many times before, but still haven’t seen much on the topic from other observers. This puzzles me a bit because it’s such an obviously powerful force with such profound implications. Indeed, a great deal of what we worry about in the near future will become irrelevant if things unfold as I expect.

Let me step back and give a summary of my analysis. The starting point is that people increasingly interact with the world through their online identities in general and their mobile phones in particular. The second point is a handful of companies control an increasing portion of consumers’ experiences through those devices: this is Facebook taking most of their screen time, Google or Apple owning the physical device and primary user interface, and Amazon managing most of their purchases.

At present, Facebook, Apple, Google, and Amazon still occupy largely separate spheres, so most people live in more than one universe. But each of the major players is entering the turf of the others. Facebook and Google compete to provide information via social and search. Both offer buying services that compete with Amazon. Amazon and Apple are using voice appliances to intercept queries that would otherwise go through to the others.

Each vendor’s goal is to expand the range of services it provides. This sets up a virtuous cycle where consumers find it’s increasingly convenient to do everything through one vendor. Instead of a conventional “social network effect” where the value of a network grows with the number of users, this is a “personal network effect” where the value of a vendor relationship grows with the number of services the vendor provides to the same individual.

While a social network effect pulls everyone onto a single universal network, the personal network effect allows different individuals to congregate in separate networks. That means the different network universes can thrive side by side, competing at the margins for new members while making it very difficult for members to switch from one network to the other.

There’s still some value to network scale, however. Bigger networks will be able to create more appealing services and attract more partners, The network owners will also provide sharing services that make it easy for members to communicate with each other (see: Apple Facetime) but harder to interact with anyone else. So the likely outcome is a handful of large networks, each with members who are increasingly isolated from members of other networks. Think of it as a collection of tribes.

Even without any intentional effort by the network owners, members of each network will have shared experiences that separate them from outsiders: try asking an Android user for help with your iPhone. The separation will become even more pronounced if the network owners more actively control the information their members receive – something that’s already happening in the name of blocking terrorists, bullies, and other genuinely bad actors. Of course, people who prefer a particular world view will be able to form their own networks, which will be economically viable because the personal network effect with outweigh the social network effect. These splinter networks might be owned independently (it’s easy to imagine a Fox News tribe) or owned by a bigger network that just gives each tribe what it wants. Either way, you have a society whose tribes that are mutually unaware at best and actively hostile at worst.

Let’s put aside the deeper social implications of all this, in the best tradition of “other than that, how did you like the play, Mrs. Lincoln?” My immediate point is that marketers and technologists should be aware of these trends because they help to explain much of what’s happening today in our industry and help to prepare for what might happen tomorrow. Here are some things to keep an eye on:

- Growth of Voice. As I’ve already mentioned, voice interactions are an alternative to conventional screen interactions. What’s important is the voice interaction often happens first: it’s easier to ask Alexa or Siri to do something than to type that same request into Google, Facebook, or Amazon. This means whoever owns the voice interaction can intercept customer behaviors before anyone else. So pay close attention to voice-based systems: far from a gimmick, they could be keys to the kingdom.

- Owning the Pipes. Network owners want above all to keep their customers’ data to themselves. This will make them increasingly interested in owning the pipes that carry that data and in blocking anyone else from tapping those pipes. Don’t be surprised to see the network owners take an interest in physical networks (cable and phone companies) and alternative connections (community wifi). Also expect them to argue that physical network owners shouldn’t be allowed to use the data they carry (an argument they just lost in Congress but will likely resurrect on privacy grounds) and that they should be able to buy preferential access (the “network neutrality” debate they are now winning in that same Congress). Could a pipe owner grow its own network? The folks at Verizon apparently think so: that’s why they bought AOL and Yahoo!

- Data Motels. It goes pretty much without saying that network owners are eager to take data from other companies, but stingy about sharing their own. So they’re happy to import other companies’ customer lists and serve them ads, conveniently getting paid while gaining new information. But they’re less interested in exporting data about those same common customers. It’s the information version of a roach motel: data checks in but it can’t check out.

- Expanding Services. We’ve already covered this but it’s so important that it bears repeating: network vendors will continue to extend the services they offer, tightly integrating them to increase the “personal network value” of their relationships. Watch carefully and you’ll notice each new service gives customers a reason to share more data, gives the network owners still more information to better personalize customer services. Everybody wins, although the networks win more.

- The AIs Have It. The networks’ ultimate goal is to handle all their members’ purchases. The best way to do this is to have members delegate as many decisions to the network as possible, starting with things like subscriptions for restocking groceries and on-demand transportation. This saves the effort of making individual sales and, more important, eliminates opportunities for members to leak out of the system. Delegation requires the members to trust the network to make the right decisions on their behalf. Gathering more data is one key to this; artificial intelligence to make good decisions with that data is another. So if you’re thinking the networks are investing in AI only because they’re nerds who like science projects, think again.

- Trust. Arguably, trust is the result of experience, so making good decisions for members should be enough to earn permission to make more decisions. But in practice it will be impossible for consumers to know if the network is really making the best possible choices. So building trust through conventional branding and relationship management will be critical skills for the network marketers, especially when it comes to recruiting new members. (Of course, with network usage starting somewhere around age 2, membership is likely to be more hereditary than anything else.) Data and AI systems will help network marketers know the best way to build trust with each individual, but human marketing skills will also be needed – at least for now.

- Marketing to Networks. If the networks really do take control of their members’ commercial lives, the role of marketers at non-network companies is much diminished. This is already happening: every dollar spent on pay-per-click search or social advertising is essentially a dollar the network spends on the owner’s behalf, based on data only the network possesses. Today, non-network marketers still set budgets, write copy, and select keywords. But those tasks are well on their way to being automated and it won’t matter much whether the automation runs on a machine at the network or the non-network company. The role of the non-network marketer in this world is to market to the network itself. This is already a reality: search engine optimization is really marketing to network search algorithms. It will be even more important when the member isn’t directly involved in the purchase process. No doubt there will be a certain amount of “incentivizing” of the network to pick a particular product, some of it under the table. But there will also be competition to build products and services that best meet member needs and to create brands that members are pleased to have chosen on their behalf.

- Whither MarTech? Marketers at non-network companies will still have jobs whether or not they sell directly to their customers. But martech vendors could face a threat to their existence. Simply put, if the networks capture all direct customer interactions and don’t share their data with outsiders, the market for customer data platforms, journey orchestration engines, predictive analytics, content management systems, and other martech mainstays will vanish. This probably overstates the problem: presumably companies will still interact directly with people after have made a purchase, even if the purchase itself is managed by the network. But the majority of marketing technology is used for customer acquisition, and much of that could become obsolete.

- Alternate Routes. Like Dickens’ Ghost of Christmas Future, I’m only showing you what might be. Non-network marketers have a strong incentive to preserve direct access to their current and future customers and many suppliers have ways to help. Non-network advertising media are first in line, of course, although they’ve been losing ground at an alarming rate. But many other companies are finding creative ways to capture customer data and attract customers’ attention. Location data and mobile apps are especially contested territory because they let firms reach customers directly in ways that customers find highly valuable. The lowly mobile wallet, if it remains outside the networks’ control, could be an alternative channel for reaching a mass audience. Telecommunication providers, with their deep pockets, broad reach, physical access to mobile devices, and vast government relationships, are probably a better bet. Of course, the telcos would probably rather join the network oligopoly than break it. But the broader point is there are still many players in the game and the outcome is far from decided. I hope this helps you make a little more sense of what's happening on the field.