slopscore: a transparent linter for AI-slop writing patterns

2026-06-25

slopscore is a prose linter for AI-slop writing patterns: the vague, generic, over-smoothed, repetitive prose that is cheap to generate and hard to read. It reads a file and returns a 0 to 100 SlopScore, a per-dimension breakdown, and the exact phrases that triggered each finding. It scores writing patterns in the text. It says nothing about who wrote it.

Version 0.7.1 is out, MIT-licensed, on PyPI as slopscore-lint and documented at jman4162.github.io/slopscore.

pip install slopscore-lint
slopscore-lint scan post.md

What slopscore does

A scan reads essays, blog posts, Markdown, JSON fields, web pages, and the prose inside code, and answers one narrow question: where does this draft lean on generic, formulaic, low-specificity writing? The default scorer is rule-based, so every point in the score traces back to a named rule and a span in your text. You can pick a genre profile (blog, essay, academic, marketing, technical, social), set strictness, suppress a rule, or baseline a repository so existing findings do not block adoption. That makes it closer to a code linter than to a detector: you can disagree with a finding, and the tool shows the evidence so you can make the call.

Why authorship is the wrong question

The popular framing for machine writing is detection: did a human or an AI write this? That question is usually unanswerable from the text alone, and the detectors that claim to answer it are unreliable and biased. A widely cited study found that GPT detectors flag writing by non-native English speakers as machine-generated far more often than writing by native speakers (Liang et al., 2023). A false accusation can land on a careful writer, a student, or someone with a formal style.

Writing quality is the question you can act on. A sentence can be weak even when a person wrote it, and a paragraph can be specific and well argued even when an AI tool helped draft it. What matters for editing is the text in front of you: whether it makes concrete claims, cites evidence, repeats itself, or uses canned structure to sound deeper than it is. slopscore stays on that question and leaves authorship alone.

What it flags

The scan reports fourteen dimensions, thirteen of them slop patterns and one a human-writing signal. The patterns cover inflated significance, generic and low-specificity language, vague or over-attribution, unsupported broad claims, repeated sentence scaffolds, formulaic contrast and rule-of-three padding, shallow “-ing” analyses, copula avoidance, prompt residue, formatting artifacts, redundancy, and sameness of cadence. Each finding carries the phrase that triggered it, so a high score turns into a list of passages to reread rather than a verdict.

slopscore-lint scan post.md --suggest
slopscore-lint scan ./content --recursive --fail-on high
slopscore-lint scan post.md --format html -o report.html

How accurate it is

slopscore is good at the easy case and weak at the hard one, and the benchmark says so. On 128 taxonomy-graded examples of overt formulaic slop the rule scorer reaches an AUROC of 0.89. On 40 held-out real-world cases drawn from Wikipedia’s AI-Cleanup work it drops to 0.69, only a little above chance, and at a strict 1% false-positive setting it catches almost none of them.

Grouped bar chart: AUROC is 0.89 on overt formulaic slop (n=128) and 0.69 on real-world Wikipedia examples (n=40), with a dashed chance line at 0.5; the catch rate at a 1% false-positive rate is 0.65 on overt slop and 0.00 on the real-world set — The project’s own benchmark for the rule scorer. It separates overt formulaic slop from clean prose well and is much weaker on subtle real-world text. Use it to edit drafts; the score points you at passages to reread.

That is why the claims stay modest. slopscore is a linter for editing drafts, and the score is a prompt to look closer.

Fairness across language

A pattern detector can punish plain or non-native English, which is the failure mode behind the detector bias above. slopscore measures that risk directly. Its eval command reports the false-positive rate of each scorer on clean writing, split by language group.

Grouped bar chart of false-positive rate on clean writing by language group: the rule scorer is 0.00 for general, simple English, and non-native English; the learned model is 0.06 for general, 0.71 for simple English, and 0.33 for non-native English — False-positive rates on clean writing. The default rule scorer fires on none of the clean samples across all three groups. The opt-in learned model over-flags simple and non-native English, so the transparent rule scorer stays the default.

The tool also scores conservatively by default, abstains on text that is too short, and gates findings through corroboration so a single weak signal does not drive the score. None of that removes the risk. It makes the risk measurable.

Using it in CI and repos

slopscore runs locally, in a pre-commit hook, or in CI, and it ships a GitHub Action. For an existing repository, record a baseline first and then fail only on new findings, so the first run does not turn into a cleanup project.

slopscore-lint baseline ./content --recursive -o .slopscore-baseline.json
slopscore-lint scan ./content --recursive --baseline-file .slopscore-baseline.json --fail-on-new

It writes SARIF for code-scanning dashboards, plus console, JSON, Markdown, and HTML reports.

Prose inside code

Writing linters usually skip source files, which is where generated filler tends to settle: docstrings, comments, examples, and READMEs. slopscore extracts the prose from Python and JavaScript or TypeScript source, scans the docstrings and comments, and ignores the code.

slopscore-lint scan src/ --recursive
slopscore-lint scan README.md CHANGELOG.md --fail-on high

Who it is for

slopscore suits writers, maintainers, editors, and content teams who want a stricter review pass before publishing. It helps most when you draft with AI tools and want to remove the residue, when you keep a blog or docs site under CI, or when you review contributor documentation and want evidence spans instead of a vibe.

It is the wrong tool when the text is very short, when the genre is meant to be promotional, when the writing is mostly non-English, or when the goal is to prove who wrote something. It is also weak on subtle, polished, lightly edited prose, as the benchmark shows.

Try it

Try the interactive demo to scan text in your browser.

pip install slopscore-lint            # rule-based core
pip install "slopscore-lint[web]"     # + web page extraction
pip install "slopscore-lint[report]"  # + HTML reports
pip install "slopscore-lint[nlp]"     # + spaCy precision
pip install "slopscore-lint[all]"     # everything

The repo, the docs, and the PyPI package have the rest.

slopscore is beta software and intentionally conservative. The benchmark and fairness numbers are the project’s own evaluation on small labeled sets (128 and 40 examples), so read them as a sense of scale rather than a guarantee. The references are public.

This is an independent project I build on my own time using public sources and open tooling. The views are my own and do not represent any current or former employer.