One of the greatest tensions in a mature business is the question of where innovation will come from. Producing what Clayton Christensen called sustaining innovations isn’t necessarily so hard – building upon what already exists, delivering more efficient or higher quality versions of existing products. But how do you produce something genuinely new?

One approach is to establish a research and development team. Freed from the responsibility to deliver immediate commercial value to customers or to shareholders, these R&D departments experiment in an attempt to push the boundaries of what’s possible. That makes total sense: starting a commercial project that relies on making advancements to bleeding-edge technology is a risky endeavour. As Bill Buxton put it:

“You can’t schedule scientific breakthroughs, so it is a bad idea to have your produce schedule depend on them. Save your gambling for the casino, and manage your research group separately from your development team.”1

Vaughn Tan, who has spent his career observing innovation in high-end, Michelin-starred restaurants, draws a similar conclusion. If you want to push the boundaries in food, it’s necessary to dedicate time, money, people and effort to pure research and development. Writing about Amaja, an Argentinian restaurant that consistently reimagined what was possible in cooking by researching new ingredients, recipes, and processes, Tan finds:

“These basic and applied research projects only sometimes bear fruit that the restaurant guest can detect in the form of new dishes. Maintaining so extensive a program of basic research while producing a steady stream of highly refined new dishes requires a dedicated R&D team.”2

How do you square this need for dedicated research and development resource with the attitude, particularly common in the creative industry, that innovation isn’t something that you can build a separate department for? “As soon as you have an ‘innovation department’, you’re buggered,” says Creative England CEO Caroline Norbury. “Absolutely everyone should be involved in innovation.” Don Sharp agrees: “When companies decide they need to innovate, they sometimes think the answer is to create an innovation department. It’s not.”

These two approaches aren’t necessarily contradictory. The point is that they apply to two different types of innovation. R&D departments are necessary when your objective is to push boundaries, when you’re trying to come up with ideas without knowing in advance what the commercial exploitation of those ideas might be, and when you’re willing to accept a high rate of failure. Separate from that, improvements can be made to everything that the organisation already does – improvements that aren’t necessarily transformational, but are important nevertheless.

Those kind of improvements are important partly because improvements to existing products and processes are straightforwardly valuable. But it’s also important because innovation improves the organisation itself. One phenomenon Tan observed among high-functioning teams was that they spent a lot of time outside their comfort zone: they practised “productive discomfort”. One example of this is their tendency to deliberately bite off more than they could chew, taking on projects that they knew were beyond them:

“These teams commit repeatedly to desperation projects which they know – explicitly – they can’t currently accomplish. Knowing this forces the teams to learn and change, to take on cognitive discomfort they would have avoided if they’d had the choice to. If the desperation project isn’t too far beyond the team’s capacity, this enforced learning lets them bridge the gap. They successfully complete projects that would previously have been impossible.”

These desperation projects aren’t reckless or foolhardy. They’re deliberately slightly outside the team’s comfort zone, not far outside it. But they remain risky, and by their nature will sometimes fail. Writing about the Copenhagen restaurant Noma’s MAD conference, a quintessential desperation project, Tan observed that Noma had staked their impressive reputation on the success of the project. As organiser Lars Williams said: “We’re totally committed. There’s no way we can wimp out now.” These high stakes and the threat of failure can, strangely, increase the chances of success – pushing the team to achieve things that they otherwise wouldn’t.

The distinction, then, becomes clearer. For speculative, cutting-edge research that pushes the boundaries of your field or your industry, and that has no immediate connection to commercial reality, it’s necessary to set up a dedicated R&D team. For day-to-day innovation that improves the dynamic of the organisation, what’s needed is to push yourself out of your comfort zone – applying perhaps more stick than carrot, using the fear of failure to spur you on. Recognising the difference will mean neatly avoiding trapping yourself at either extreme: believing that innovation is something that can only be achieved by a dedicated team, or believing that dedicated innovation teams will always fail.

Further reading

Vaughn Tan. “The Uncertainty Mindset”. Columbia University Press, 2020

Vaughn Tan. “Productive Discomfort”. Substack, 2020

Bill Buxton. “Sketching User Experiences”. Morgan Kaufmann, 2007

  1. Bill Buxton, Sketching User Experiences, p. 85 

  2. Vaughn Tan, The Uncertainty Mindset, p. 5