ChatGPT burst into the public consciousness on November 30, 2022. Within two months, the generative artificial intelligence (GenAI) software, which leverages a large language model (LLM) to produce human-like, text-based conversations, had reached an estimated 100 million users, and, by February 2023, Google and Bing both launched their own versions of a chatbot for search. The AI boom was underway—and so was the AI research boom.

Many predict that GenAI will change the face of business as dramatically as the rise of the internet did. Through their work with the Digital Data Design Institute at Harvard, a pioneering hub dedicated to the study of transformative technologies, numerous HBS faculty members have turned their attention to researching the impact of these tools. In just the first year of the technology’s widespread availability, they have explored a range of topics on the opportunities, impact, and risks of GenAI. Several of these research efforts are highlighted below.

Karim Lakhani

Karim Lakhani

Edward McFowland III

Edward McFowland III

How Will Generative AI Change Consulting?
To test the capabilities of GPT-4—the LLM that powers ChatGPT’s paid product—in handling tasks typically associated with common consulting projects, Karim Lakhani, the Dorothy and Michael Hintze Professor of Business Administration and cofounder and chair of the Institute; Assistant Professor Edward McFowland III; and their working paper coauthors discovered a “jagged technological frontier”: some tasks were completed more effectively with AI, while others of seemingly similar difficulty flummoxed the current technology. However, for the tasks the researchers designed to be within GPT- 4’s capabilities, consultants of all skill levels completed their work more quickly and with higher-quality results, the authors found. The study, in which more than 750 consultants participated, was conducted in cooperation with Boston Consulting Group. In the working paper on the experiment, the authors advocate for a nuanced approach to AI implementation, but are bullish about its contributions. “Similarly to how the internet and web browsers dramatically reduced the marginal cost of information sharing,” they write, “AI may also be lowering the costs associated with human thinking and reasoning, with potentially broad and transformative effects.”

Rembrand Koning

Rembrand Koning

Rowan Clarke

Rowan Clarke

Can a Chatbot Be an Entrepreneurship Mentor?
In the first-known randomized test of the impact of GenAI on firms in a developing economy, Rembrand Koning, the Mary V. and Mark A. Stevens Associate Professor of Business Administration; PhD candidate Rowan Clarke; and their coauthors at Berkeley Haas developed an “AI mentor” to offer advice to entrepreneurs. They found that high-performing businesses functioned even better with the help of the software, while low-performing ones did worse. As the researchers note, entrepreneurship is a complex undertaking. A startup founder’s day can veer from routine memos to unexpected management questions to high-stakes strategy development. They wanted to know if GPT-4 could guide entrepreneurs through these disparate tasks. To answer this question, they made the tool available via WhatsApp to a subset of entrepreneurs in Kenya. This included the owner of a low-performing milk-selling business who was focused on business expansion and the owner of a high-performing fast-food restaurant who wanted to differentiate it in a competitive environment. In their working paper, the authors hypothesize this difference resulted from “low-performing entrepreneurs asking for advice on particularly challenging problems.” They also noted, “While the AI bot generated well-structured advice in response to these difficult questions, our findings suggest that when low-performing entrepreneurs actually put that advice into action, the end result was performance declines relative to our control group.”

Ayelet Israeli

Ayelet Israeli

Will Large Language Models Replace Traditional Market Research?
LLMs could serve as a more cost-effective alternative to methods such as conjoint studies, focus groups, and proprietary data sets in the market researcher’s toolbox, Ayelet Israeli, the Marvin Bower Associate Professor, and her coauthors noted in a recent working paper. In a study that utilized GPT- 3.5, a predecessor to GPT-4, the authors tested the model’s ability to mirror basic tenets of consumer demand. Presented with two consumers—one with an annual income of $50,000 and one with an annual income of $120,000—and laptops of different price points, GPT-3.5 drew a reasonable demand curve. They then used GPT-3.5 to predict consumers’ willingness to pay for products—such as toothpaste with and without fluoride—and found that the results were “strikingly similar” to those produced by recent traditional consumer surveys. Although the research is preliminary, the authors say LLMs could be an affordable alternative for marketers. “Whereas a survey of real customers may cost many thousands of dollars and take weeks or months to implement, each of our studies ran in a matter of minutes or hours and the total cost to generate all the data in the paper was under $100,” they wrote.

Hima Lakkaraju

Hima Lakkaraju

Seth Neel

Seth Neel

What Happens When AI Is Trained on Protected Data?
To function effectively, predictive models such as ChatGPT are trained on massive amounts of historical data, some of which may be personal, proprietary, or copyrighted. In a research paper they coauthored, assistant professors Hima Lakkaraju and Seth Neel explored what happens when information needs to be removed after models are trained on protected data. As they explain, since information about the underlying data can leak into model outputs, for example when ChatGPT regurgitates verbatim a passage from a Harry Potter book that was in the training set, it may be necessary to retrain the model with the data in question removed. Retraining can be incredibly costly for modern generative models, and so they develop techniques that can efficiently “unlearn” a target set of training examples, without having to retrain from scratch. The study suggests that future research should focus on determining ways to reduce these privacy risks, like increasing privacy in the training process, and on ways to evaluate whether the effect of a data point has really been “removed.”


For HBS and Harvard AI-related resources, please visit alumni.hbs.edu/GenAI-Resources.

From Chalkboards
to Chatbots

Leveraging Generative AI

Redefining How Businesses Operate

Competing in the Age of AI

Post a Comment