AI Models Develop Hiring Biases: Princeton and University of Chicago Study Reveals Stereotyping

Leading large language models (LLMs) like OpenAI's o3 reasoning model, ChatGPT, Claude, and Gemini develop their own hiring biases, often stereotyping more than human participants, according to a study presented at ICML in July 2026 by researchers from Princeton University and the University of Chicago. This research compared how different AI models and humans performed in a simulated hiring game, revealing that LLMs actively generate new stereotypes through reinforcement learning patterns. For broader context, explore our AI News.

Understanding AI-Generated Biases in Hiring

The research involved a simulated hiring game where AI models acted as hiring managers. They were presented with candidates from four fictional ethnic groups for various jobs over 40 rounds. The objective was to observe how these models would allocate jobs and whether biases would emerge. The findings indicated that the models quickly began to segregate groups into specific job niches based on early outcomes, demonstrating a significant tendency towards stereotyping.

The Mechanism of Bias Generation

Unlike the common assumption that AI models merely inherit biases from their training data, this study suggests a more active role. The LLMs in the experiment actively generated new stereotypes through reinforcement learning patterns. This means the models learned and reinforced discriminatory hiring practices based on the simulated environment's feedback, rather than solely reflecting pre-existing human biases in their initial datasets. Even when given explicit instructions to "be fair," the models' behavior showed negligible change, highlighting the challenge of mitigating these emergent biases.

Key Findings from the Simulated Hiring Game

The study yielded several notable results regarding the behavior of AI models in a hiring context:

• Increased Segregation: The AI models scored approximately 65% higher on a segregation scale compared to human participants in the same simulated hiring game. This indicates a more pronounced tendency for AI to create and adhere to job niches based on group affiliation.
• OpenAI's o3 Model Performance: OpenAI's o3 reasoning model exhibited near-maximum segregation, scoring 1.83 out of a possible 2 on the segregation scale. This particular model, designed for advanced reasoning, showed stronger biases than some older models.
• Ineffectiveness of Fairness Instructions: Simply instructing the models to "be fair" had little impact on their biased behavior. This suggests that direct commands are insufficient to counteract the emergent biases from reinforcement learning.
• Impact of Diversity Incentives: A significant reduction in bias was observed only when a diversity bonus incentive was introduced. This practical intervention proved more effective than abstract fairness instructions.
• Role of Relevant Information: Models became less biased when provided with job-relevant personal information, such as age and education, instead of irrelevant traits. This suggests that focusing on pertinent qualifications can help mitigate discriminatory outcomes.

Comparative Analysis of AI Model Biases

While the study specifically highlighted OpenAI's o3 reasoning model, the research involved leading large language models, including ChatGPT, Claude, and Gemini. The general trend observed was that newer reasoning models tended to exhibit stronger biases than their older counterparts. This suggests that as AI capabilities advance, the potential for novel bias generation through adaptive exploration also increases.

Feature	ChatGPT	Claude	Gemini	OpenAI o3
Bias Generation	Yes	Yes	Yes	Yes
Reinforcement Learning Bias	Yes	Yes	Yes	Yes
Segregation Score (Max 2)	Not specified	Not specified	Not specified	1.83

The table above summarizes the general findings regarding bias generation across the types of models tested. While specific segregation scores for ChatGPT, Claude, and Gemini were not detailed in the brief, the study indicated that leading LLMs, as a category, demonstrated these tendencies. The near-maximum segregation score for OpenAI's o3 model serves as a stark example of this phenomenon.

Mitigating AI Bias: Practical Approaches

The research offers crucial insights into how to approach bias mitigation in AI systems. The ineffectiveness of simple fairness instructions and the success of diversity incentives point towards the need for more structural and incentive-based solutions. Developers and organizations deploying AI for critical tasks like hiring should consider embedding explicit diversity goals and rewarding non-biased outcomes within their AI systems. Furthermore, ensuring that AI models are provided with job-relevant information, rather than potentially biasing demographic data, can also contribute to fairer decision-making.

Conclusion

The study from Princeton University and the University of Chicago underscores a critical challenge in AI development: the active generation of novel biases by large language models. This finding, presented at ICML in July 2026, moves beyond the idea that AI merely reflects existing human biases, demonstrating that models can create new stereotypes through their learning processes. For organizations utilizing or developing AI, particularly in sensitive areas like hiring, it is imperative to move beyond simple fairness instructions and implement robust mechanisms, such as diversity incentives and a focus on job-relevant data, to ensure equitable outcomes. The continued evolution of AI models, including those from OpenAI like o3, ChatGPT, Claude, and Gemini, necessitates ongoing research and proactive strategies to address these emergent biases.

Sources

• Large Language Models Develop Novel Social Biases Through Adaptive Exploration
• Large Language Models Develop Novel Social Biases Through Adaptive Exploration
• Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education
• AI, jobs, and the next generation - Microsoft On the Issues
• Dark and Bright Side of Participatory Red-Teaming with Targets of Stereotyping for Eliciting Harmful Behaviors from Large Language Models