Analyzing reasoning and decision-making in large language models
William Thompson, Professor
Psychology
Applications for Fall 2024 are closed for this project.
Understanding the capabilities and limitations of conversational AI systems is a research priority with practical and theoretical implications in cognitive science and artificial intelligence. In this project, we are investigating the capabilities of modern LLMs on reasoning and decision-making tasks using experiment methods from cognitive science. The project includes the potential to join teams studying creative semantic extension, information-seeking strategies, and agent-oriented social reasoning.
Role: - Implementation of systematic experimental studies of model behavior
- Analysis and visualization of experimental datasets
- Model training and fine-tuning
- Analysis of model internal computational dynamics
Qualifications: Required: Strong experience programming in Python, capability to implement programs that systematically interact with commerical model APIs; data analysis experience.
Preferred: experience or willingness to acquire proficiency in model training and fine-tuning using Python frameworks such as Pytorch, tensorflow, and huggingface model ecosystems; interest in cognitive science and psychology.
Hours: 9-11 hrs
Digital Humanities and Data Science Social Sciences