Abstract
Qualitative data analysis, a crucial subset of data analysis, extracts insights from unstructured data through systematic methods. It plays a significant role in diverse fields such as social science, psychology, and educational research. With the rapid evolution of Artificial Intelligence (AI) and Large Language Models (LLMs), leveraging AI, particularly LLMs, in qualitative analysis has gained considerable attention. The synergy between LLMs and human analysts in producing in-depth interpretations is becoming more and more important.
This PhD thesis, initiated before the advent of LLMs, delves into the then-niche area of AI-assisted qualitative analysis. It investigates the potential of AI in augmenting collaborative qualitative analysis, focusing on human-to-human and direct human-AI collaboration. The thesis is structured into two main sections: 1) developing innovative collaborative workflows for qualitative coding teams, integrating multiple humans and AI models, and 2) enhancing trust and reliance dynamics within human-AI teams by analyzing their interactions.
More specifically, I will begin by setting the context and motivations for this burgeoning field, followed by an overview of the core theoretical frameworks in qualitative analysis. Subsequently, I will introduce the design and development of two CQA systems, CoAIcoder and CollabCoder, along with the associated studies and evaluations. Lastly, I will present an empirical study examining the trust and reliance dynamics between humans and AI in a qualitative analysis context.