Can Chat GPT3 get an Ivy league MBA? Research by Wharton Prof Christian Terwiesch Reveals

ChatGPT, a chatbot launched by Microsoft backed OpenAI, is creating waves across many knowledge industries. Its applications are being talked about in Book publishing, Legal Documentation and of course for writing Software Codes. Now, Wharton Professor and well-respected digital expert Christian Terwiesch has explored an interesting question – Can Chat GPT3 get a Wharton MBA?

In a 26-page paper, titled “Would Chat GPT3 Get a Wharton MBA?”, published in January 2023, copy of which was reviewed by MBAUniverse.com, Professor Terwiesch says that Chat GTP 3 “does an amazing job at basic operations management and process analysis questions including those that are based on case studies.” He goes on to say that “Not only are the answers correct, but the explanations are excellent.” Satisfied with the answers, he gave a “B or B- grade” to Chat GPT3, implying that it will pass a typical MBA exam. He suggests that as Chat GPT technology evolves, it may have an important bearing on management education. MBAUniverse.com explores key highlights from this intriguing paper by Prof Terwiesch.

But, first, let’s understand more about this technology.

What is ChatGPT?
ChatGPT, where GPT stands for Generative Pre-trained Transformer, is a chatbot launched by OpenAI in November 2022. OpenAI is an AI research and deployment company. It says that its mission is to ensure that artificial general intelligence benefits all of humanity. Investors in OpenAI include Microsoft, Reid Hoffman’s charitable foundation and Khosla Ventures. ChatGPT was launched as a prototype on November 30, 2022, and quickly got a lot of attention for its detailed responses and coherent answers across many domains of knowledge. To be sure, Chat GPT is different from Search Engine. ChatGPT is interactive while search engines passively provide article links. Microsoft is reportedly planning to launch a version of Bing that uses ChatGPT to answer search queries. However, as also pointed by Professor Terwiesch, the technology has flaws related to facts and calculations.

Prof Christian Terwiesch is the Andrew M. Heller Professor at the Wharton School of the University of Pennsylvania. He also holds a faculty appointment In Penn’s Perelman School of Medicine. His research on Operations Management and on Innovation Management has been published in many of the leading academic journals ranging from Management Science to The New England Journal of Medicine. Professor Terwiesch has authored/co-authored many books including “Matching Supply with Demand”, a widely used text-book in Operations Management, and also the foundation for MOOCs course on Coursera, enrolled by more than half a million students. His 2019 book “Connected Strategies” combines his expertise in the fields of operations, innovation, and strategy to help companies take advantage of digital technology leading to new business models and was a cover story in Harvard Business Review. He holds a doctoral degree from INSEAD and a Diploma from the University of Mannheim.

Let’s now come to the experiment conducted by Professor Terwiesch..

Would Chat GPT3 Get a Wharton MBA? Prof Christian Terwiesch Paper Says Yes….

In his preamble, Prof Christian Terwiesch gives credit to Chat GPT and says, “OpenAI’s Chat GPT3 has shown a remarkable ability to automate some of the skills of highly compensated knowledge workers in general and specifically the knowledge workers in the jobs held by MBA graduates including analysts, managers, and consultants.”

He clearly spells out that the purpose of his paper is to “document how Chat GPT3 performed on the final exam of a typical MBA core course, Operations Management.” Outlining the testing procedure he says, “Exam questions were uploaded as used in a final exam setting and then graded.”

According to Prof Christian Terwiesch, the “academic performance” of Chat GPT3 can be summarized as follows.

• First, it does an amazing job at basic operations management and process analysis questions including those that are based on case studies. Not only are the answers correct, but the explanations are excellent.
• Second, Chat GPT3 at times makes surprising mistakes in relatively simple calculations at the level of 6th grade Math. These mistakes can be massive in magnitude.
• Third, the present version of Chat GPT is not capable of handling more advanced process analysis questions, even when they are based on fairly standard templates.
• Finally, ChatGPT3 is remarkably good at modifying its answers in response to human hints. In the instances where it initially failed to match the problem with the right solution method, Chat GPT3 was able to correct itself after receiving an appropriate hint from a human expert.

Considering above performance, Prof Terwiesch says, “Chat GPT3 would have received a B to B- grade on the exam.” He further adds, “Chat GPT3 has important implications for business school education, including the need for exam policies, curriculum design focusing on collaboration between human and AI, opportunities to simulate real-world decision-making processes, the need to teach creative problem solving, improved teaching productivity, and more.”

Prof Terwiesch also went on to explore if Chat GPT can prepare questions for his next term exams. Based on the result, he concluded, “Given that ChatGPT has demonstrated some creativity in producing poetry and song lyrics, I tested if it would be able to produce new questions for future exams and the next edition of my book. The resulting questions were well worded and at times even humorous. However, they required substantial adjustments before becoming usable exam questions.”

Prof Terwiesch cautions B-school leaders, Faculty Members and MBAs that they should be mindful of the impact of technologies like CHAT GPT on their domain by noted the affect that electronic calculators and computers had on the corporate world, “Prior to the introduction of calculators and other computing devices, many firms employed hundreds of employees whose task it was to manually perform mathematical operations such as multiplications or matrix inversions. Obviously, such tasks are now automated, and the value of the associated skills has dramatically decreased. In the same way any automation of the skills taught in our MBA programs could potentially reduce the value of an MBA education.”

Stay tuned to MBAUniverse.com for more insights and updates on MBA Education.