Cultivating an Innovator’s Mindset: Q&A with Michael Schrage
MIT research fellow Michael Schrage was a guest speaker at Quadrants 2022, participating in a keynote discussion with Constellation Software President Mark Leonard at our flagship learning event for Volaris Group business leaders.
How can software leaders discover new ways to provide value? Michael Schrage advocates for rigorous and thoughtfully-designed business experiments as a tool for unlocking new potential. As a research fellow with the MIT Sloan School of Management's Initiative on the Digital Economy, Schrage runs design workshops and executive education programs on innovation, experimentation, and strategic measurement for organizations all over the world. His research, writing, and advisory work focuses on the behavioral economics of models, prototypes, and metrics as strategic resources for managing innovation risk and opportunity. He has authored several books, including the award-winning The Innovator’s Hypothesis (MIT Press, 2014).
Schrage shares his views with Acquired Knowledge magazine about how to nurture a culture of innovation through business experimentation.
1. Your talk at Quadrants won’t be your first interaction with Volaris employees—we introduced a business experiments program for some of our interested companies at the end of summer 2022. What key messages have you emphasized to our program participants?
The key emphasis is that businesses want to have lightweight and high-impact experimentation as part of their culture and capabilities—in terms of how value is identified, defined, explored, and exploited. In an organization like Volaris where long-term investment is so important, I view experimentation as a capability that should be part of a leadership and organizational portfolio.
While it’s true that businesses need to be able to conduct analysis, sell, market, and brand their products—increasingly, digitalization transforms the economics of experimentation. I think organizations are foolish if they underinvest in business experimentation, or don’t think intentionally about how they want to use fast, cheap, simple experiments to generate actionable insights. Simple experiments can do a better job of managing risk, identifying value, and exploring next steps that can reap the benefits of that value.
Michael Schrage on stage at Quadrants 2022
2. Why is it important for businesses to experiment, especially for vertical market software businesses in the fast-paced technology space, such as the ones that are part of Volaris?
If one were to look at my portfolio of clients and students, I would say that the organizations best positioned to take advantage of an experimentation culture are those in the digital spaces—whether they be legacy software companies, telecoms, or companies undergoing digital transformation.
In the past, with analog instruments such as telescopes, microscopes, and laboratories, experimenters faced latencies, delays, and fairly high fixed costs. Experimenters needed to be highly trained, with PhDs or postdocs. There was a large cost associated with traditional approaches to experimenting, so it was more economical to conduct analysis.
Now with increasing digitalization, software companies can cost-effectively exploit the economics of experimentation. If an organization has an innovation or new product roadmap, or if they are seeking to change the dynamic of their relationship with clients, they can implement an experiments agenda in place to test hypotheses about how that product will be used, or how certain features and functions will be explored.
3. One of the messages of your book is that anyone in a business can run an experiment, not just the C-suite. Why is this important to highlight?
This reflects a cultural value that I take very seriously, and to a certain extent would like to attribute to a former MIT collaborator, Eric von Hippel. This cultural value is the democratization of innovation.
The ability to run an experiment should, in my professional opinion, no longer be the province of R&D, or the VP of Innovation. Technology advancements have enabled that, with almost every employee having a supercomputer in their purse or pocket. To be sure, the CEO is talking at the very highest level with customers, clients, and prospects. But more often than not, the CEO and C-suite are not doing customer support and don’t have first-hand, direct line of sight into customer experience.
Frontline people, particularly those who experience customer and client touch (such as staff working in the contact centre, technical support, or remote diagnostics) should be empowered to conduct business experiments. Those employees get to observe patterns in their businesses: things that go very well, or go very poorly. They should be empowered to not just put a suggestion in a suggestion box, but say: “This is the pattern I’m observing. It implies a certain hypothesis of: ‘If we do this, that may be the result. If we don’t do this, that may be the result.’”
Adopting experimentation at the frontline level is a practice that facilitates bottom-up innovation. Customer touch can be a source of inspiration and insight for value creation, and I think Volaris CEO Mark Miller feels the same way, which may be one of the reasons the book resonated with Volaris.
Adopting experimentation at the frontline level is a practice that facilitates bottom-up innovation.
-Michael Schrage, MIT research fellow and author of The Innovator's Hypothesis
4. What should businesses keep in mind if they want to design a successful experiment?
What we are focusing on is not to test a technical or scientific hypothesis, but to test a business hypothesis. A hypothesis in science is a search for truth, but a hypothesis in business is a search for value.
An experimentation framework needs to reflect the rigor of thinking, strategic intent, and notion of customer lifetime value that serious leaders and organizations would accept. That’s where leadership and technical subject matter experts need to collaborate to frame and define a hypothesis that lends itself to statistically and pragmatically designed experiments.
5. Some vertical market software companies are smaller businesses. Can you make the case for conducting a business experiment if a company has a smaller staff or limited resources, and it may seem easier for them to pursue other sources of knowledge, such as analyzing existing data?
Certainly depending on one’s size, scale, scope, or intent, experimentation as a capability in your portfolio may merit a different kind of investment. Obviously, in terms of sheer statistical significance, it’s preferable to run an experiment with 100,000 trials rather than one with 15-20 trials.
That said, one of the advantages that smaller organizations have is that—precisely because they’re constrained—there should be certain kinds of clarity around the sorts of experiments they should run and hypotheses they should consider.
For example: For a business whose product is overwhelmingly used by the IT group, they may note that their product really supports sales, the contact centre, or marketing. Their staff might ask if the company would benefit from developing a product or service that brings in members of the client’s marketing group. They might ask: What might that look like? Can we do an A/B test on this? Can we build something into our existing product that encourages legacy users to reach out inside their own organization? What kind of approach might we explore to go beyond our core clients and build the community of users?
That’s the sort of constraint-based hypothesis formation that I think even niche companies should take seriously. They should move away from merely thinking, “Hmm, this is an interesting idea,” to framing a hypothesis correctly to create a scalable insight that an experiment might yield.
If you are part of a 50-person company with 50 core clients, then your experiments will be different than a 300-person company with 300 clients. Scale matters—but focus matters, too.
The most effective organizations have experimentation be not just a technique, but one of their values for learning.
-Michael Schrage, MIT research fellow and author of The Innovator's Hypothesis
6. What are your top observations of businesses that have successfully integrated an experimental mindset into their company culture?
The whole notion of how cultures recognize the value of experiments is a big deal. Organizations need to think about how they want to socialize the economic and cultural importance of an experimental mindset.
Recently, I gave a talk on digital transformation and experimentation to one of the largest fashion retailers in Brazil, who put their younger generation in charge of digital innovation. I asked them, “Do you run experiments?” They said yes. I then asked, “Do you celebrate the most interesting and provocative experiments? How do you socialize them? Do people who run interesting experiments become internal heroes in the company?”
Businesses should ask: How do we make experimentation a relevant value for the organization, not just another tool or technique to create economic value? The most effective organizations have experimentation be not just a technique, but one of their values for learning.
In my most successful classes and clients, the cultivation of an experimentation mindset is part of professional development. If you look at how organizations like Amazon, Spotify, Google, or Meta spend their time, a lot of it is spent on designing and running experiments. That’s something that is baked into their culture and recognized and rewarded by upper management.
7. Any pitfalls to avoid when designing a business experiment?
There’s a common piece of advice people give that says if you have a choice to make, you should flip a coin because you know what you really want if you are rooting for the coin to come up heads or tails.
The biggest trap that organizations fall into is that they do not simply want to test a hypothesis—they may set out to prove that an existing belief they hold is valid. The best experiments are the ones where you really don’t know what the answer is going to be or how it’s going to test out, and you genuinely care about finding out.