Abstract:
Built on large language models (LLMs), silicon samples offer a promising approach for advancing behavioral research by simulating decision-making processes and replicating human decision-making in controlled experiments. This study evaluates LLMs' potential to replicate behavioral hypotheses in operations management (OM) contexts by conducting nine studies from the Management Science Replication Project. I develop an accessible framework where GPT-4o agents are assigned roles based on System 1 decision-making and participate in the experiments. Remarkably, the silicon samples replicated the core hypotheses in eight of the nine studies, with treatment effects closely matching the direction and magnitude of human decisions. Crucially, the natural language explanations generated by GPT agents offer insight into their decision logic, revealing whether observed choices align with the behavioral theories being tested. I also provide strong evidence that the agents' decisions are shaped by task structure and feedback across rounds, supporting an interpretation of in-situ reasoning rather than recall and reproduction of published results. These findings position silicon samples as a methodological tool that complements existing experimental, analytical, and simulation approaches in OM, while contributing evidence to broader debates on the use of LLM-based agents as research subjects. For researchers, silicon samples support scalable replication, diagnostic analysis of decision processes, and early-stage intervention testing. For decision makers, the approach enables rapid exploration of behavioral responses across complex operational settings prior to costly deployment.
Contact Emails:
scoco@ceibs.edu