Research case
ChatGPT vs. crowd workers for text annotation (Gilardi et al., PNAS 2023)
Political scientists compared zero-shot ChatGPT labels with MTurk crowd workers on tweet and news annotation tasks (relevance, stance, topics, frames).
What is reported
The paper reports ChatGPT beat crowd workers on accuracy for most tasks, at a fraction of the per-annotation cost.
Before you use it
Findings concern the tested English datasets and model configuration. Check against your own human-coded sample before scaling.
Sources and review
Source checked · Not tested by this site
Checked the public abstract and bibliographic record. Results describe the authors' study, not a replication by this site.
License and reuse
No license has been recorded. Check the source’s license before reusing code, models or data.
A license note is not a license audit. Open source does not guarantee free hosting, model access or commercial reuse.