NLP

Subasa - Adapting Language Models for Low-resourced Offensive Language Detection in Sinhala

ARTICLE: Annotator Reliability Through In-Context Learning

Using LLMs to identify high-quality human annotators by checking if their labels are consistent with AI predictions—helping build better training data while preserving diverse …

Rater Cohesion and Quality from a Vicarious Perspective

Asking people to predict how others with different political views would label content reveals hidden biases and improves data quality for content moderation AI.

Harnessing Business and Media Insights with Large Language Models

Vicarious Offense and Noise Audit of Offensive Speech Classifiers: Unifying Human and Machine Disagreement on What is Offensive

We ran a massive experiment: 9 different AI content moderation systems analyzed 92 million YouTube comments about US politics. The results were shocking—different AI systems …

Subjective Crowd Disagreements for Subjective Data: Uncovering Meaningful CrowdOpinion with Population-level Learning

CrowdOpinion uses unsupervised learning to group similar content and predict the full range of human opinions about it, rather than forcing everyone into a single 'correct' …

Disagreement Matters: Preserving Label Diversity by Jointly Modeling Item and Annotator Label Distributions with DisCo

Findings from the Bambara - French Machine Translation Competition (BFMT 2023)

Improving Label Quality by Joint Probabilistic Modeling of Items and Annotators

Annotator Response Distributions as a Sampling Frame