Susan Brennan
[introductory/advanced] Where Are the Humans in Human-Centered AI?
Summary
Conversation is collaborative. Why isn’t AI?
Babies learn to interact before they learn words. Generative AI knows more words than you do, but not how to interact. Despite its startling virtuosity at producing paragraphs of fluent language, AI is far from adapt at conversational interaction. What does an AI agent need to learn, in order to converse?
And how can we work across disciplines to do convergent research, in order to solve problems that people actually have?
Syllabus
- Grounding meaning in human-human and human-AI communication
- Audience design, common ground, and partner-specific language use
- Coordination, initiative, entrainment, and repair
- Paralinguistic cues in human language use
Case studies:
- Do LLMs/LVLMs have an emergent Theory of Mind?
- (Using data from psycholinguistics experiments to test AI agents)
- Using AI to solve problems that people actually have
- (The role of stakeholders)
- How to do convergent research in human-AI interaction
- (An NSF-funded traineeship for graduate students)
References
Brennan, S. E. & Clark, H. H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory and Cognition, 22, 482-1493.
Brennan, S. E. and Williams, M. (1995). The feeling of another’s knowing: Prosody and filled pauses as cues to listeners about the metacognitive states of speakers. Journal of Memory and Language, 34, 383-398.
Cahn, J. E., & Brennan, S. E. (1999). A psychological model of grounding and repair in dialog. Proc. AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems (pp. 25-33). North Falmouth, MA: American Association for Artificial Intelligence.
Costello, T.H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science, 385, eadq1814 (2024). V. F. Reyna …
Jones, C. R., Trott, S., & Bergen, B. (2024). Comparing humans and large language models on an experimental protocol inventory for theory of mind evaluation (EPITOME). Transactions of the Association of Computational Linguistics.
Paige, A., Soubki, A., Murzaku, J., Rambow, O., & Brennan, S. E. (2024). Training LLMs to Recognize Hedges in Dialogues about Roadrunner Cartoons. In T. Kawahara, V. Demberg, S. Ultes, K. Inoue, S. Mehri, D. Howcroft, & K. Komatani, (Eds.), Proc. 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue (pp. 204-215). Kyoto, Japan: Association for Computational Linguistics.
Soubki, A., Paige, A., Murzaku, J., Rambow, O., & Brennan, S. E. (2025). Projecting Characters’ Knowledge from Utterances in Narratives: A Psycholinguistic Baseline for LLMs. ToM4AI, Advancing Artificial Intelligence through Theory of Mind, 39th Annual AAAI Conference on Artificial Intelligence, Philadelphia, PA.
Wang, Z., Li, W., Kaliosis, P., Brennan, S. E., & Rambow, O. (2025). LVLMs are Bad at Overhearing Human Referential Communication. Proc. 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025) (pp. 16770–16794). Association for Computational Linguistics. Suzhou, China.
Pre-requisites
None.
Short bio
Susan Brennan is SUNY Distinguished Professor at Stony Brook University in Cognitive Science, with affiliations in Psychology, Computer Science, and Linguistics. Her research uses eye-tracking, behavioral, and experimental methods to study psycholinguistics, communication (verbal, nonverbal, and multimodal), adaptive language use, and the human use of technology, including spoken dialogue systems. She currently directs the NSF Research Traineeship project, Detecting and Addressing Bias in Data, Humans, and Institutions, a collaboration among Departments of Psychology, Computer Science, Applied Math & Statistics, Linguistics, Political Science, Economics, Sociology, Neurobiology and Behavior, and Africana Studies, with AI and bias-related convergent research projects that include a collaboration with the Innocence Network. She holds a Ph.D. in cognitive psychology from Stanford University, an M.S. in visual studies from MIT (from what is now known as the MIT Media Lab), and a B.A. from Cornell University in cultural anthropology. She has worked in industry at Atari Research, Hewlett-Packard Labs, and Apple Computer.
