framework
    Representative Problems in Authorship Attribution:
  1. Human-written Text Attribution (attributing unknown texts to human authors)
  2. LLM-generated Text Detection (detecting if texts are generated by LLMs)
  3. LLM-generated Text Attribution (identifying the specific LLM or human responsible for a text)
  4. Human-LLM Co-authored Text Attribution (classifying texts as human-written, machine-generated, or a combination of both)


Abstract

Accurate attribution of authorship is crucial for maintaining the integrity of digital content, improving forensic investigations, and mitigating the risks of misinformation and plagiarism. Addressing the imperative need for proper authorship attribution is essential to uphold the credibility and accountability of authentic authorship. The rapid advancements of Large Language Models (LLMs) have blurred the lines between human and machine authorship, posing significant challenges for traditional methods. We present a comprehensive literature review that examines the latest research on authorship attribution in the era of LLMs. This survey systematically explores the landscape of this field by categorizing four representative problems: (1) Human-written Text Attribution; (2) LLM-generated Text Detection; (3) LLM-generated Text Attribution; and (4) Human-LLM Co-authored Text Attribution. We also discuss the challenges related to ensuring the generalization and explainability of authorship attribution methods. Generalization requires the ability to generalize across various domains, while explainability emphasizes providing transparent and understandable insights into the decisions made by these models. By evaluating the strengths and limitations of existing methods and benchmarks, we identify key open problems and future research directions in this field. This literature review serves a roadmap for researchers and practitioners interested in understanding the state of the art in this rapidly evolving field.

Contributions

  • We provide a timely overview to discuss the challenges and opportunities presented by LLMs in the field of authorship attribution. By systematically categorizing authorship attribution into four problems and balancing problem complexity with practicality, we reveal insights into the evolving filed of authorship attribution in the era of LLMs.
  • We offer a comprehensive comparison of state-of-the-art methodologies, datasets, benchmarks, and commercial tools used in authorship attribution. This analysis not only improves the understanding of authorship attribution but also provides a valuable resource for researchers and practitioners to use as guidelines for approaching this direction.
  • We discuss open issues and provide future directions by considering crucial aspects such as generalization, explainability, and interdisciplinary perspectives. We also discuss the broader implications of authorship attribution in real-world applications. This holistic approach ensures that authorship attribution not only yields accurate results but also provides insights that are explainable and socially relevant.

Benchmarks

The table below is a summary of Authorship Attribution Datasets and Benchmarks with LLM-Generated Text. Size is shown as the sum of LLM-generated and human-written texts (with the percentage of human-written texts in parentheses). Language is displayed using the two-letter ISO 639 abbreviation. Columns P2, P3, and P4 indicate whether the dataset supports problems described in Problem 2, 3, and 4, respectively.

Name Domain Size Length Language Model P2 P3 P4
TuringBench News 168,612 (5.2%) 100 to 400 words en GPT-1,2,3, GROVER, CTRL, XLM, XLNET, FAIR, TRANSFORMER-XL, PPLM
TweepFake Social media 25,572 (50.0%) less than 280 characters en GPT-2, RNN, Markov, LSTM, CharRNN
ArguGPT Academic essays 8,153 (49.5%) 300 words on average en GPT2-Xl, text-babbage-001, text-curie-001, davinci-001,002,003, GPT-3.5-Turbo
AuTexTification Tweets, reviews, news, legal, and how-to articles 163,306 (42.5%) 20 to 100 tokens en, es BLOOM, GPT-3
CHEAT Academic paper abstracts 50,699 (30.4%) 163.9 words on average en ChatGPT
GPABench2 Academic paper abstracts 2.385M (6.3%) 70 to 350 words en ChatGPT
Ghostbuster News, student essays, creative writing 23,091 (87.0%) 77 to 559 (median words per document) en ChatGPT, Claude
HC3 Reddit, Wikipedia, medicine, finance 125,230 (64.5%) 25 to 254 words en, zh ChatGPT
HC3 Plus News, social media 214,498 N/A en, zh ChatGPT
HC-Var News, reviews, essays, QA 144k (68.8%) 50 to 200 words en ChatGPT
HANSEN Transcripts of speech (spoken text), statements (written text) 535k (96.1%) less than 1k tokens en ChatGPT, PaLM2, Vicuna-13B
M4 Wikipedia, WikiHow, Reddit, QA, news, paper abstracts, peer reviews 147,895 (24.2%) more than 1k characters ar, bg, en, id, ru, ur, zh davinci-003, ChatGPT, GPT-4, Cohere, Dolly2, BLOOMz
MGTBench News, student essays, creative writing 21k (14.3%) 1 to 500 words en ChatGPT, ChatGLM, Dolly, GPT4All, StableLM, Claude
MULTITuDE News 74,081 (10.8%) 200 to 512 tokens ar, ca, cs, de, en, es, nl, pt, ru, uk, zh GPT-3,4, ChatGPT, Llama-65B, Alpaca-LoRa-30B, Vicuna-13B, OPT-66B, OPT-IML-Max-1.3B
OpenGPTText OpenWebText 58,790 (50.0%) less than 2k words en ChatGPT
OpenLLMText OpenWebText 344,530 (20%) 512 tokens en ChatGPT, PaLM, Llama, GPT2-XL
Scientic Paper Scientific papers 29k (55.2%) 900 tokens on average en SCIgen, GPT-2,3, ChatGPT, Galactica
RAID News, Wikipedia, paper abstracts, recipes, Reddit, poems, book summaries, movie reviews 523,985 (2.9%) 323 tokens on average cs, de, en GPT-2,3,4, ChatGPT, Mistral-7B, MPT-30B, Llama2-70B, Cohere command and chat
M4GT-Bench Wikipedia, Wikihow, Reddit, arXiv abstracts, academic paper reviews, student essays 5,368,998 (96.6%) more than 50 characters ar, bg, de, en, id, it, ru, ur, zh ChatGPT, davinci-003, GPT-4, Cohere, Dolly-v2, BLOOMz
MAGE Reddit, reviews, news, QA, story writing, Wikipedia, academic paper abstracts 448,459 (34.4%) 263 words on average en GPT, Llama, GLM-130B, FLAN-T5 OPT, T0, BLOOM-7B1, GPT-J-6B, GPT-NeoX-2
MIXSET Email, news, game reviews, academic paper abstracts, speeches, blogs 3.6k (16.7%) 50 to 250 words en GPT-4, Llama2

Detectors

The image and tables below present an overview of commercial and open-source LLM-Generated Text Detectors.

detectors

Commercial Detectors

Detector Free Tier Paid Plan API Humanizer Website
GPTZero 10k words/mo $12.99/mo · 300k words gptzero.me
Winston 2k words trial $10/mo · 80k words gowinston.ai
Sapling 2k chars $12 · 100k chars sapling.ai
Pangram 4 checks/day $20/mo · 600 checks pangram.com
ZeroGPT 15k chars $7.99 · 100k chars zerogpt.com
Phrasly 6k words $10.99/mo · unlimited phrasly.ai
Smodin AI Detector 50k chars $12/mo smodin.io
Scribbr 500 words/check $19.95 · unlimited scribbr.com
QuillBot 1,200 words/scan $8.33 · unlimited quillbot.com
Draft & Goal 2k words $9.99/mo · 200k words detector.dng.ai
BrandWell 2,500 chars $199/yr · WriteWell plan brandwell.ai
Undetectable AI $5/mo · 10k words undetectable.ai
Isgen $8/mo · 350k words isgen.ai
Grammarly $12/mo · Pro plan grammarly.com
Plag.AI $14.95/mo · 10k words plag.ai
Plagiatkontroll $15.33/mo · 25k words plagiatkontroll.no
Originality.AI $12.95/mo · 200k words originality.ai
CopyLeaks $13.99/mo · 300k words copyleaks.com
GPT Radar $0.02 · 100 tokens gptradar.com
Turnitin's AI detector License required turnitin.com

Open-Source Detectors

Detector Method Repo
Binoculars Zero-shot GitHub
DetectGPT Zero-shot · probability curvature GitHub
Fast-DetectGPT Zero-shot · conditional probability curvature GitHub
GPT-2 Output Detector RoBERTa fine-tune GitHub
Hello-SimpleAI ChatGPT Detector RoBERTa · trained on HC3 Hugging Face
Desklib AI Text Detector DeBERTa-v3 · trained on RAID Hugging Face

BibTeX

@article{huang2025authorship,
  title={Authorship attribution in the era of llms: Problems, methodologies, and challenges},
  author={Huang, Baixiang and Chen, Canyu and Shu, Kai},
  journal={ACM SIGKDD Explorations Newsletter},
  volume={26},
  number={2},
  pages={21--43},
  year={2025},
  publisher={ACM New York, NY, USA}
}


framework
The example above compares Linguistically Informed Prompting (LIP) with other baselines that provide less linguistic guidance for the authorship verification task. The outputs of an LLM are categorized as either "Analysis" or "Answer." Only the LIP strategy correctly identifies that the two given texts were authored by the same individual. Texts highlighted in orange emphasize the differences across four levels of guidance. Texts highlighted in blue indicate the linguistically informed reasoning process, while blue text represents content referenced from the original documents.


Abstract

The ability to accurately identify authorship is crucial for verifying content authenticity and mitigating misinformation. Large Language Models (LLMs) have demonstrated exceptional capacity for reasoning and problem-solving. However, their potential in authorship analysis remains under-explored. Traditional studies have depended on hand-crafted stylistic features, whereas state-of-the-art approaches leverage text embeddings from pre-trained language models. These methods, which typically require fine-tuning on labeled data, often suffer from performance degradation in cross-domain applications and provide limited explainability. This work seeks to address three research questions: (1) Can LLMs perform zero-shot, end-to-end authorship verification effectively? (2) Are LLMs capable of accurately attributing authorship among multiple candidates authors (e.g., 10 and 20)? (3) How can LLMs provide explainability in authorship analysis, particularly through the role of linguistic features? Moreover, we investigate the integration of explicit linguistic features to guide LLMs in their reasoning processes. Our assessment demonstrates LLMs' proficiency in both tasks without the need for domain-specific fine-tuning, providing insights into their decision-making via a detailed analysis of linguistic features. This establishes a new benchmark for future research on LLM-based authorship analysis.

Contributions

  • We conduct a comprehensive evaluation of LLMs in authorship attribution and verification tasks. Our results demonstrate that LLMs outperform existing BERT-based models in a zero-shot setting, showcasing their inherent stylometric knowledge essential for distinguishing authorship. This enables them to excel in authorship attribution and verification across low-resource domains without the need for domain-specific fine-tuning.
  • We develop a pipeline for authorship analysis with LLMs, encompassing dataset preparation, baseline implementation, and evaluation. Our novel Linguistically Informed Prompting (LIP) technique guides LLMs to leverage linguistic features for accurate authorship analysis, enhancing their reasoning capabilities.
  • Our end-to-end approach improves the explainability of authorship analysis. It elucidates the reasoning and evidence behind authorship predictions, shedding light on how various linguistic features influence these predictions. This contributes to a deeper understanding of the mechanisms behind LLM-based authorship identification.

BibTeX

@inproceedings{huang2024authorship,
    title = "Can Large Language Models Identify Authorship?",
    author = "Huang, Baixiang  and  Chen, Canyu  and  Shu, Kai",
    editor = "Al-Onaizan, Yaser  and  Bansal, Mohit  and  Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-emnlp.26/",
    doi = "10.18653/v1/2024.findings-emnlp.26",
    pages = "445--460",
    abstract = "The ability to accurately identify authorship is crucial for verifying content authenticity and mitigating misinformation. Large Language Models (LLMs) have demonstrated exceptional capacity for reasoning and problem-solving. However, their potential in authorship analysis remains under-explored. Traditional studies have depended on hand-crafted stylistic features, whereas state-of-the-art approaches leverage text embeddings from pre-trained language models. These methods, which typically require fine-tuning on labeled data, often suffer from performance degradation in cross-domain applications and provide limited explainability. This work seeks to address three research questions: (1) Can LLMs perform zero-shot, end-to-end authorship verification effectively? (2) Are LLMs capable of accurately attributing authorship among multiple candidates authors (e.g., 10 and 20)? (3) Can LLMs provide explainability in authorship analysis, particularly through the role of linguistic features? Moreover, we investigate the integration of explicit linguistic features to guide LLMs in their reasoning processes. Our assessment demonstrates LLMs' proficiency in both tasks without the need for domain-specific fine-tuning, providing explanations into their decision making via a detailed analysis of linguistic features. This establishes a new benchmark for future research on LLM-based authorship analysis."
}