Accuracy & False-Positive Metrics
How accurately Stoprop spots manipulation techniques and coordinated campaigns — and how rarely it false-flags clean news, fair opinions, or normal viral sharing. Every fraction below shows flagged / tested alongside its statistical margin.
No benchmark results published yet
Evaluation summaries appear here once benchmark results are generated.
Text Analyzer: Sensitivity Levels
Stoprop lets you choose between three sensitivity settings. Strict (the default) is tuned to keep false alarms on clean news reporting under 5%, while Balanced and Broad surface subtler rhetorical framing.
| Sensitivity | Precision | Recall | F1 | False positives on clean news | False positives on hard negatives |
|---|
Per-Technique Detection Breakdown
Measured across human-annotated news paragraphs (SemEval-2020 Task 11) and multi-technique test passages (Broad mode, excluding reported speech).
| Technique / Marker | Precision | Recall | F1 | Flagged | Reference spans |
|---|
Benchmark Split & False Positives on Clean Text
In strict scoring, persuasive words inside attributed quotes (reported speech) are not counted against the author; lenient scoring counts quoted speech too, matching the original SemEval-2020 annotation guidelines.
| Dataset | Scoring rule | Precision | Recall | F1 | Flagged | Reference |
|---|
Clean-text false-positive check
| Clean benchmark set | Paragraphs | Flagged (techniques) | Flags per 1,000 words |
|---|
Swarm Mode: Coordinated Campaign Benchmark
| Scenario | Type | Posts | Result | Outcome |
|---|
Viewpoint Neutrality & Language Coverage
Stoprop tests political and group symmetry on mirrored pairs where only the sides are swapped.
| Language | Benchmark status |
|---|
How these numbers are measured (and their limits)
- Language scope: Stoprop accepts English and other languages; quantitative accuracy benchmarks have been measured on English so far, while other languages use the same conservative Strict default.
- Reference datasets: evaluated on human-annotated news paragraphs from SemEval-2020 Task 11 (Da San Martino et al., 2020, arXiv:2009.02696) and controlled multi-technique passages, alongside clean-news and emotional/opinion hard-negative controls.
- Exact-quote verification: every highlighted passage is verified character for character against the input text; any unverified quote is automatically discarded before display.