From Wall Street trading to warehouse logistics, artificial intelligence has proved it can outperform humans. In new research from Texas McCombs, AI takes on humans at an even higher-level business function: strategic decision-making.
Harsh Ketkar, assistant professor of management, finds AI can enhance the speed, quality, and scale of strategic analysis. In matchups against human creators and evaluators of business plans -- conducted with Felipe Csaszar from the University of Michigan and Hyunjin Kim from INSEAD -- AI equaled or bested its challengers.
"Artificial Intelligence and Strategic Decision-Making: Evidence from Entrepreneurs and Investors" is published in Strategy Science.
Besides writing plans, he finds AI can critique existing strategies and suggest others.
"AI can be the sparring partner that can come up with an alternative set of ideas that can't be easily envisioned," says Ketkar. "It can help executive teams to break out of that blind spot that can sometimes happen."
Using AI for strategic decisions, he says, could affect competition between companies, survival or failure of startups, and companies' long-term growth and success.
Better Business Plans
The authors drew their original inspiration from financial trading. It's an activity where humans once dominated, but algorithms now account for 78% of trading decisions.
Strategic decision-making, by contrast, involves more nuanced kinds of analysis and judgment -- the kinds that experts acquire through years of study and experience. How capable were machines of doing it well?
Ketkar and his colleagues ran two experiments. For the first, they partnered with a European startup accelerator. The researchers selected five business plans it had accepted and five it had rejected during 2021 and 2022.
For each plan, they used the program GPT 3.5 to create an AI-generated double. For a prompt, they used language from the entrepreneur's original plan, which described the problem the plan was addressing. They then asked GPT to create the rest of the business plan.
They showed the plans to 250 evaluators, with an average of five years' investing experience and seven years' managerial experience. Each evaluator reviewed a mix of 10 entrepreneur-generated and AI-generated plans, but never the same plan and its double.
The evaluators rated each plan's key attributes -- such as innovation and value proposition, viability and writing quality, and potential to invest in the idea -- on a 1 to 10 scale. They also rendered overall judgments: whether they would accept the startup into the accelerator, invest in it, or request an introduction to its founder.
Collectively, the evaluators rated GPT-written plans higher across all criteria. They were:
AI picks winners
The second experiment reversed the first. Instead of writing business plans, AI assessed them.
Ketkar and his fellow researchers partnered with a startup competition at an elite business school. They selected 138 all-text business plans submitted in the competition's previous 10 cycles. Each plan had already been evaluated by three to five competition judges: a mixture of venture capitalists and investors, many with backgrounds as entrepreneurs.
The researchers had GPT evaluate the plans, using prompts that mirrored instructions given to judges. To simulate real-world judging, each plan went through three GPT assessments.
The GPT evaluations correlated strongly with the average scores from experienced investors -- more strongly, in fact, than the human scores correlated with one another.
"We were quite surprised with the correlation we got," Ketkar says. "AI was quite accurate in picking the winners, and it was also better at predicting which plan was ultimately going to be successful."
The results, he says, suggest AI could stand in for humans for evaluating business plans.
Expanding access to strategic planning
The research could have implications across the entrepreneurial landscape, Ketkar says.
"Everyone now has access to high-level strategic decision-making," says Ketkar. "It's McKinsey-in-a-box. It democratizes it."
Over time, he predicts, companies that incorporate AI into their strategic planning may get a competitive edge and enhance performance. He compares it to the advent of algorithmic trading on Wall Street.
"This is going to lead to new strategy frameworks being developed," he says. "It will change strategy decision-making, not just in the way that we do it, but in the way that we perceive it."
