quallmer: A Large Language Model-Assisted Qualitative Analysis Toolkit in R

This article introduces quallmer, an R package that provides researchers with AI-assisted qualitative coding tools. It supports multiple data types such as text, images, PDFs, and tables, and includes built-in functions for reliability testing, validity verification, and audit trails. It aims to combine the semantic capabilities of LLMs with qualitative research methodologies, enabling efficient processing of large-scale data while maintaining academic rigor.

Traditional qualitative research relies on manual coding, which is time-consuming and prone to subjective bias. The development of LLM technology has opened up possibilities for innovation in qualitative research. The emergence of quallmer addresses the question of whether AI-assisted coding can balance efficiency and rigor, and it is specifically designed for R language users.

Key Features

• qlm_code(): Automatically codes data using an LLM-based codebook;
• qlm_codebook(): Creates a custom codebook with example templates;
• qlm_compare(): Evaluates inter-coder reliability (consistency between models, prompts, AI and humans);
• qlm_validate(): Verifies coding accuracy by comparing with gold standards;
• qlm_replicate(): Conducts sensitivity analysis to ensure result replicability;
• qlm_trail(): Generates a complete audit trail to support research credibility.

quallmer is built on the ellmer package and supports multiple LLM providers (OpenAI, Anthropic, etc.). It uses ellmer type specifications to define coding instructions. Its extensible framework adapts to domain-specific coding schemes and supports segmented analysis of long texts, making it suitable for scenarios like interviews and policy documents.

quallmer is suitable for:

• Social media data analysis: Sentiment tendency identification;
• Interview transcript coding: Theme/discourse pattern extraction;
• Policy document analysis: Conceptual framework/argumentation strategy identification;
• Multimodal data analysis: Image/PDF/table processing to support mixed-methods research.

Discussion points for AI-assisted qualitative analysis:

• Credibility: Balancing efficiency and rigor through audit trails;
• Researcher role: Shifting from coder to method designer/result validator;
• Transparency: Audit trails ensure process traceability;
• Bias ethics: Need for multi-model comparison and manual verification to mitigate LLM bias impacts.

Unique features of quallmer:

• Compared to commercial platforms: Open-source and controllable with adjustable parameters;
• Compared to general LLM APIs: Provides academic methodologies and quality control;
• Compared to other R packages: Fills the gap in LLM-assisted qualitative coding (other packages focus on quantitative analysis).

Future Developments

• Multilingual support;
• Visualization of coding results;
• Team collaboration features;
• Deepened integration with tidyverse;
• Advanced analytical functions (network/time series).

Conclusion

quallmer amplifies researchers' capabilities, allowing them to focus on core tasks (problem formulation/result interpretation). It is an important milestone in the digitalization of qualitative research and is expected to reshape the methodological landscape.