Methodology

This dashboard tracks aggregate community compliance with core/AGENTS.md across Drupal core issue queue artifacts. It exists to evaluate a 3-month harm-reduction experiment: does publishing operator rules change observable LLM-prose patterns?

What's measured

Two tracks, different denominators by design:

• Stylometric detector — six features (burstiness, function-word vector cosine, phrase-tells v3 source-aware, Yule's K, hapax ratio, Flesch reading ease) computed against per-author baselines frozen on pre-2022 prose. Coverage: ~250 tier-1-baselined authors plus all 2026-active commenters at lower-confidence tiers.
• Rule conformance — pure regex / sentence-split checks on artifact text. §1 disclosure (compliant / modified-with-model-name / none / na) and five §2 sub-rules (≤180 words, ≤25 words/sentence, no forbidden words, no ##/### headers in <180-word comments, no formula phrasing). Coverage: every artifact (comment, MR note, issue summary, MR description) regardless of author.

Forbidden words list (§2.3)

Stem-prefix matching against this list, after fenced and inline code stripping. Trailing silent e dropped so conjugations match (leverage → matches leveraged, leveraging). No carve-outs.

data/forbidden-words.txt

Formula phrasing patterns (§2.5)

• whether X or Y — span ≤100 chars between keywords, excluding the whether or not idiom (empirically 19% of literal pre-period hits, structurally not the X/Y construction §2 targets).
• X rather than Y
• I believe …
• not only X but also Y — span ≤100 chars between keywords.

K-anonymity

K=10 for per-component, per-topic-tag, and MR-burst slices (component maintainership is publicly identifiable on drupal.org). K=5 elsewhere. Suppressed cells display in the JSON as "value": null, "suppressed": true with the underlying numerator and denominator preserved in the database for operator audit but not exposed in the public artifact.

Pre-period and burn-in

• Pre-period: T0 − 97d to T0 − 7d. Frozen reference; never recomputed unless an explicit re-render runbook is run.
• Burn-in: T0 to T0 + 7d. Shaded grey on charts; excluded from headline rolling-28d numbers.
• Treatment: T0 + 7d onward. Rolling 28d and cumulative-since-T0 windows.

AGENTS.md amendment policy

The dashboard pins the AGENTS.md ruleset to the version in effect at experiment T0. Mid-experiment amendments are visible to contributors but do not change what the dashboard measures — pre/post comparability requires a stable treatment definition.

Phrase-tells v3 source-aware disclosure

The phrase-tell list contains source-aware rules: markdown-format tells (**bold**, ## heading, [link](…)) fire only on source='comment' because Drupal.org strips Markdown rendering. GitLab notes (source='note') render Markdown natively, so MD usage is not a tell there. Comparing tell density across artifact types in the same chart reflects both LLM behavior and phrase-list scope; readers should compare each track to its own pre-period baseline rather than across tracks.

Live counts

M1 — comments rule-checked (latest day)

229

M2 — tier-1 cohort size (latest day)

469

M3 — suppressed cells (latest day)

2

Source code & data

• daily.json — every aggregate row the renderer reads; the source of truth for verifiers.
• forbidden-words.txt — the §2.3 list verbatim.
• AGENTS.md (T0 snapshot) — the ruleset this experiment measures.