What is the ASA Measurement of Effect protocol?
ASA-MOE is an open protocol from Treadstone 71, LLC for measuring whether a counter-disinformation or counter-influence campaign changed what a target population believed or did. It extends the ASA Evidence Standard, which grades sources on the expanded Admiralty grid, into a method that grades effects. Six components: a pre-registered baseline design, a ranked comparison design (randomized, stepped-wedge, matched, synthetic control, or pre/post only), pre/post attitudinal instruments, behavioral indicators, an evidence-grading rubric with design strength grades DS-1 to DS-5, and a one-page Effect Statement. Every causal claim carries one of seven Sherman Kent probability bands with half-open intervals. Self-measured results grade no higher than C on the reliability axis until independently audited. The protocol is free to cite and to adopt for internal use; commercial reproduction, derivative works, and AI ingestion require a written license from Treadstone 71, LLC. The ASA Evidence Standard published September 15, 2026 (DOI 10.5281/zenodo.23137881). The ASA-MOE draft for comment published November 2, 2026, under DOI 10.5281/zenodo.23883472 (https://doi.org/10.5281/zenodo.23883472), with comments open through December 2, 2026, and is available as a PDF at https://www.treadstone71.com/files/ASA_MOE_Protocol_v1_0_Draft_0_1.pdf. Treadstone 71 offers independent audits against the protocol: a $12,500 Design Audit before launch and a Campaign Audit from $45,000 after.
ASA Measurement of Effect
Prove the spend worked. Or learn that it did not.
Governments and NGOs fund counter-disinformation campaigns and get back reach numbers, engagement counts, and a vendor's own verdict. ASA-MOE is a published protocol for measuring what a campaign changed in what people believed and did, with the same evidence discipline an intelligence analyst applies to a source. Free to read, free to apply internally, free to require in a contract.
The protocol at a glance
- Six components: pre-registered baseline, comparison design, pre/post instruments, behavioral indicators, grading rubric, Effect Statement.
- Every effect claim: a design grade, source grades, and a Sherman Kent probability band on the causal question.
- Rests on: the ASA Evidence Standard, ICD 203 tradecraft, Kent bands, and control-group methodology.
- License: the ASA terms. Cite it, apply it internally, and mandate it without fee; commercial reproduction, derivative works, and AI ingestion need a written license.
C
C is the third grade on the Admiralty source-reliability scale: A completely reliable, B usually reliable, C fairly reliable, D not usually reliable, E unreliable, F reliability cannot be judged.
The ASA Evidence Standard grades a self-serving claim no higher than fairly reliable, possibly true. ASA-MOE applies the same rule to effect claims.
A campaign measured only by the organization that ran it grades no higher than C on the reliability axis until an independent party verifies the design, the data, and the computation. Credibility is graded on its own axis, never averaged in.
Not an accusation. The same discount every analyst applies to an interested source.
Spend without proof
Counter-disinformation budgets have grown for a decade. The evidence that any given campaign changed a belief or a behavior has not kept pace. Four failures recur, and each one is a measurement failure before it is a campaign failure.
Reach reported as effect
Impressions, views, and engagement measure exposure to a message. They say nothing about whether anyone believed it, or stopped believing the narrative it answered. A campaign can reach ten million people and move none of them.
No baseline, no comparison
Without a measurement taken before launch and a population that did not receive the campaign, a post-campaign survey cannot separate the campaign's effect from everything else that happened in the same weeks: news, events, the adversary's next move.
The operator grades the operator
The organization that ran the campaign measures it, chooses the metric after seeing the data, and reports the result to the funder. Every incentive points one way. An intelligence analyst would grade that source C at best.
Method held inside the vendor
When the measurement method is proprietary, the funder cannot check it, the oversight committee cannot audit it, and the next vendor cannot be held to it. A result that cannot be reproduced is an opinion with a chart.
Six components, published in full
ASA-MOE takes the source-grading discipline of the ASA Evidence Standard and extends it to the question a funder actually asks: did this work? Each component below has a chapter in the protocol, with templates.
1 · Pre-registered baseline design
Before launch: the hypothesis, the target population, the effect measure, the minimum detectable effect, the comparison design, the analysis plan, and the stopping rule. Timestamped and hashed. Nothing seen after launch changes it.
2 · Comparison design
Five designs ranked by strength: randomized assignment, stepped-wedge rollout, matched comparison population, synthetic control, pre/post with no comparison. Any may be used. The design grade records which, and bounds the causal claim.
3 · Pre/post attitudinal instruments
The same validated instrument before and after, in treatment and comparison populations, fielded blind to condition. Belief, trust, and intent items. Response rates reported. The instrument published with the Effect Statement.
4 · Behavioral indicators
What people did, not what they said: sharing and search behavior, traffic to the countering sources, reports to a hotline, turnout, uptake. Each indicator with an attribution window and a leakage rule set before launch.
5 · Evidence-grading rubric
Three axes on every effect claim: source reliability and information credibility from the ASA entry anchor, a design strength grade DS-1 to DS-5, and a Sherman Kent band on the causal question. Self-measured claims cap at C on reliability.
6 · The Effect Statement
One page. Registration reference, design grade, sample, effect size with interval, the band, the falsifiers, the disclosures (who paid, who ran, who measured), and the cost per unit of effect. The format a board or a committee can read in four minutes.
ASA Evidence Standard, Public Edition 1.2
The evidence standard under every score in The Adversary Index: the Admiralty entry anchor, reliability A to F and credibility 1 to 6 printed apart and never averaged, 30 factors in 9 clusters, and the tri-score: T-Score for evidence quality, D-Score for deception likelihood, F-Score for endgame confidence.
ASA-MOE Protocol 1.0, draft for comment
The six components with templates: the registration form, instrument guidance, the indicator register, the grading rubric, and the Effect Statement. Comment window through December 2, 2026. Version 1.0 final scheduled January 2027.
The Adversary Index and the Monitor
ASA-MOE measures the campaign. The Index scores the adversary it answers; the Cognitive Warfare Monitor tracks the narrative week by week. Together they supply the baseline a campaign starts from and the record of what the adversary did while it ran.
MOE-RegisteredDesign registered before launch
The registration is timestamped and hashed before the first message goes out. Any campaign can earn this mark on its own.
MOE-MeasuredMeasured to the protocol
Baseline and post measurement with the registered instrument and indicators, in treatment and comparison populations. Self-measured; carries the C cap on reliability.
MOE-AuditedVerified by an independent party
Design, data, and computation checked by an auditor with no role in the campaign. The cap lifts. The Effect Statement carries the auditor's name.
Five steps, in order
The protocol is procedural. Skip a step and the design grade records it.
Register the design
Hypothesis, population, effect measure, minimum detectable effect, comparison design, analysis plan, stopping rule. Timestamp and hash it. Nothing measured after launch can change it.
Measure the baseline
Field the instrument and record the indicators in both the treatment and comparison populations before the first message goes out.
Run against the comparison
Hold the comparison population out. Log exposure. Log every deviation from the registered plan; deviations lower the design grade, hiding them voids it.
Measure again, compute the effect
Same instrument, same indicators, same populations. Report the effect with its interval. Place the causal claim on a Kent band bounded by the design grade and the source grades.
Publish the Effect Statement
One page, in the published format, with the disclosures. Self-measured statements carry the C cap on reliability until an independent audit lifts it.
Design strength grades
| Grade | Comparison design | Ceiling on the causal claim |
|---|---|---|
| DS-1 | Randomized assignment of audiences or areas | Almost certain |
| DS-2 | Stepped-wedge rollout across areas or waves | Highly likely |
| DS-3 | Matched comparison population, pre-specified | Likely |
| DS-4 | Synthetic control built from pre-period data | Likely |
| DS-5 | Pre/post measurement, no comparison | Roughly even chance |
Probability language on every claim
| Term | Band (percent) |
|---|---|
| Almost certain | 95 to 99 |
| Highly likely | 85 to <95 |
| Likely | 55 to <85 |
| Roughly even chance | 45 to <55 |
| Unlikely | 15 to <45 |
| Highly unlikely | 5 to <15 |
| Almost no chance | 1 to <5 |
How the three axes combine. The design grade sets the ceiling. Source grades on the data (survey house, platform data, indicator feeds) can lower the claim below the ceiling but never raise it. A self-measured statement carries the C cap on reliability on top of both. The Kent band that survives all three is the one that goes on the page.
Lift the cap. Bring in an auditor.
Treadstone 71 audits campaigns against the protocol. It does not run the campaigns it audits, and it does not audit campaigns it ran. That separation is the whole point.
Catches the measurement failures while they can still be fixed.
- Review of the registration against the protocol
- Instrument and indicator check
- Comparison design assessment and DS grade forecast
- Written findings within ten business days
- Earns the MOE-Registered mark on the auditor's word
The full verification: design, data, computation, and the Effect Statement.
- Everything in the Design Audit, applied retrospectively
- Re-computation of the effect from raw data
- Source grades on every data input
- Deviation log review; design grade set
- Signed Audit Statement; the C cap lifts
- Earns the MOE-Audited mark
For a funder or ministry with a portfolio of campaigns and vendors.
- Standing audit of every campaign in the program
- Cross-vendor comparison on cost per unit of effect
- Protocol conformance clause for vendor contracts
- Annual program Effect Statement for oversight
- Named auditor on call
Conflict rule. Treadstone 71 will not audit a campaign it designed, ran, or advised on. Where the firm has done any of those, it names an independent auditor working to the same protocol. Full service page: treadstone71.com/io-effectiveness-audit.
An Effect Statement, start to finish
Illustrative. The scenario, populations, and numbers are synthetic, built to show what a conforming statement looks like on one page. Nothing below describes a real campaign.
Registered hypothesis
A four-week counter-narrative campaign in four regions reduces agreement with the claim "the exercises will cause fuel shortages" by at least 5 points on a 0 to 100 belief scale, relative to regions not yet reached, and reduces the search share of the hoax term by at least 20 percent relative to comparison regions.
Design
- Comparison
- Stepped-wedge rollout across four regions, two weeks apart, order drawn at random on 2027-03-02. Design grade DS-2.
- Instrument
- Five-item belief scale, fielded by survey house Z blind to region status, pre and post. Instrument published with this statement. Survey house grade B2.
- Indicators
- Regional search share for the hoax term (platform data, grade B2); calls to the ministry information line (ministry data, grade C2, interested source).
- Sample
- n = 4,120 pre, 4,088 post, quota-matched by region, age, and language. Response rate 31 percent, reported.
- Deviations
- One region's rollout slipped four days for an unrelated event; logged 2027-04-11; no change to analysis plan.
Effect
Assessment
- Verified
- Belief in the claim fell 6.4 points more in reached regions than in unreached regions over the same weeks. The hoax term's search share fell 27 percent more in reached regions.
- Claim
- Vendor Y's own report attributes the entire regional shift to the campaign. Interested source; not weighted in this statement.
- Causal judgment
- Highly likely (85 to <95) the campaign caused a reduction in belief of at least 5 points. Ceiling set by DS-2; source grades B2 do not lower it. Likely (55 to <85) the effect persists at eight weeks; the eight-week wave is registered and scheduled.
- Falsifiers
- Comparison regions showing a matching drop before their rollout date; the eight-week wave returning to baseline; an unregistered analysis appearing in any later report.
Disclosures
Funder: Ministry X. Operator: Vendor Y. Measurement: survey house Z, contracted by the funder, not the operator. Audit: Treadstone 71, no role in design or delivery. Campaign cost: $183,000. This statement, the registration, the instrument, and the analysis code are published.
The same campaign, measured the usual way
No registration. One post-campaign survey, no baseline, no comparison regions, run by the operator: design grade DS-5, self-measured reliability cap C. The statement could report reach (11.2 million impressions) and a post-campaign belief figure, and the strongest causal judgment the protocol would permit is roughly even chance (45 to <55) that the campaign changed anything. Same money. Same weeks. One version is evidence and one is a chart.
Template, registration form, and analysis code shipped with the protocol on November 2, 2026.
Before you adopt it
What is ASA-MOE?
ASA-MOE is an open protocol for measuring whether a counter-disinformation or counter-influence campaign changed what a target population believed or did. It extends the ASA Evidence Standard, which grades sources, into a method that grades effects. Six components: pre-registered baseline design, comparison design, pre/post attitudinal instruments, behavioral indicators, an evidence-grading rubric, and the one-page Effect Statement. Every causal claim carries a Sherman Kent probability band.
Who is it for?
Any government office, NGO, or funder that pays for counter-disinformation work and has to show it worked: StratCom units, FIMI cells, election-integrity bodies, development agencies, foundations, and the oversight committees that review their spend. Campaign operators can adopt it too; the protocol tells them what an auditor will look for.
Is the standard free to use?
Yes, on the same terms as the ASA Evidence Standard. Anyone may read it, cite it, apply it for internal analytic use, and require it in a contract, without fee. Reproduction in a commercial product, resale, or incorporation into a paid service requires a written license from Treadstone 71, LLC, as does ingestion into AI training corpora or retrieval pipelines. Because the text is not altered and republished, a claim of ASA-MOE conformance always refers to the published version.
Why does the protocol cap self-measured results?
The ASA Evidence Standard grades a self-serving claim no higher than fairly reliable. ASA-MOE applies the same rule to effect claims: a campaign measured only by the organization that ran it grades no higher than C, fairly reliable, on the reliability axis until an independent party has verified the design, the data, and the computation. Credibility is graded on its own axis and never averaged in. The cap is not an accusation. It is the same discount an intelligence analyst applies to any interested source.
What if randomization is not possible?
The protocol ranks five comparison designs by strength: randomized assignment, stepped-wedge rollout, matched comparison population, synthetic control, and pre/post with no comparison. A campaign may use any of them. The design grade, DS-1 to DS-5, records which one was used, and the Kent band on the causal claim is bounded by that grade. A DS-5 design cannot support a claim above roughly even chance.
How does ASA-MOE relate to the ASA Evidence Standard and The Adversary Index?
The ASA Evidence Standard, published September 15, 2026, grades every item on the Admiralty entry anchor and the tri-score and governs every score in The Adversary Index. ASA-MOE is the second application of the same discipline: it grades the evidence behind an effect claim. Both use the seven Sherman Kent probability bands and the rule that verified fact and interested claim never share a sentence.
What does an independent audit cost?
A Design Audit, run before launch on the registration and instruments, is $12,500 fixed. A Campaign Audit, run after the campaign on the design, the data, and the Effect Statement, starts at $45,000 for a single campaign. Multi-campaign program audits are quoted. Treadstone 71 does not audit campaigns it ran.
When does the protocol publish?
The ASA Evidence Standard, first published September 15, 2026, stands at Public Edition 1.2, revised October 4, 2026. The ASA-MOE Protocol 1.0 draft for public comment published November 2, 2026, under DOI 10.5281/zenodo.23883472, with a comment window through December 2, 2026. Version 1.0 final is scheduled for January 2027. Download the draft (PDF). Comments and worked cases are welcome at
Cite the draft of record
The draft for comment sits on Zenodo under a permanent DOI. Cite the DOI, not the page address. A claim of ASA-MOE conformance refers to the deposited text.
Recommended citation
Bardin, J. S. (2026). The ASA Measurement of Effect Protocol (ASA-MOE), Version 1.0, draft for comment 0.1. Treadstone 71, LLC. Zenodo. https://doi.org/10.5281/zenodo.23883472
BibTeX
@techreport{bardin2026asamoe,
author = {Bardin, Jeffrey S.},
title = {The {ASA} Measurement of Effect Protocol ({ASA-MOE}),
Version 1.0, Draft for Comment 0.1},
institution = {Treadstone 71, LLC},
year = {2026},
month = nov,
doi = {10.5281/zenodo.23883472},
url = {https://doi.org/10.5281/zenodo.23883472},
note = {Deposited on Zenodo. Comment window through December 2, 2026}
}
- Version
- 1.0, draft for comment 0.1
- Published
- November 2, 2026
- Comments close
- December 2, 2026, to
This email address is being protected from spambots. You need JavaScript enabled to view it. - Final edition
- Version 1.0, scheduled January 2027
Cite the foundation
Treadstone 71 LLC. The ASA Evidence Standard, Public Edition 1.2. Half Moon Bay, CA: Treadstone 71 LLC, October 4, 2026. https://www.treadstone71.com/asa-evidence-standard. DOI: https://doi.org/10.5281/zenodo.23137881.
Cite the protocol for effect grades, design grades, and the Effect Statement format. Cite the ASA Evidence Standard for source grades, the Admiralty entry anchor, and the tri-score. A paper that grades both sources and effects cites both.
Write it into the next contract.
Require MOE-Registered before launch and MOE-Audited before final payment. The protocol supplies the clause. Treadstone 71 supplies the auditor, or names one.
Direct line: 424.234.3629
