The Science of Science: Human-AI (Large Language Models) Interaction in Conducting Research
Juliana Schroeder, Professor
Business, Haas School, Psychology
Closed. This professor is continuing with Fall 2023 apprentices on this project; no new apprentices needed for Spring 2024.
Research involving Large Language Models (LLM) is one of the hottest topics in social science. Nevertheless, a non-trivial amount of anecdotal evidence suggests that outputs by LLM could be biased, inconsistent, and unreliable. How can and should social science researchers leverage LLM in the research process (stimuli generation, response coding, etc.)?
We seek to address the following research questions:
How is the field currently utilizing LLMs in research processes? What opinions do researchers hold about LLM involvement in these processes? How can we develop a universal protocol enabling social science researchers to use LLMs in an unbiased way, ensuring external validity of research outputs?
In this project, we will:
(a) conduct a field survey among researchers;
(b) inform researchers about the potential benefits and drawbacks of using LLMs at each stage of the research process;
(c) provide recommendations for the appropriate use of LLMs.
Role: You will gain experience in nearly all aspects of the research process, including conducting literature review on prompt engineering guidelines, exploring prompt engineering in different research contexts, hybrid data collection (e.g., response coding with and without LLM using data from other ongoing projects), and questionnaire quality control. You will be directly supervised by Yunhao (Jerry) Zhang (postdoctoral fellow), but Professor Juliana Schroeder (Jerry’s postdoctoral advisor) will be heavily involved in the research process. You will get a chance to work with them closely based upon performance. NOTE: Project related tasks can be both in-person and remote.
Qualifications: We seek research apprentices who are interested in human-AI interaction in social science contexts (psychology / business / economics / …). Experience in Python coding and familiarity with Large Language Models' APIs are preferable (not required). We are especially interested in students who are considering applying to graduate school to do research in Psychology or any field related to Psychology! Please check our websites for full information about our research.
Hours: 9-11 hrs
Related website: https://www.yunhaojerryzhang.com/
Related website: http://www.julianaschroeder.com