--- title: ASA Measurement of Effect description: ASA-MOE is an open protocol for measuring whether a counter-influence campaign changed what a population believed or did. Draft for comment, open DOI. canonical_url: https://www.treadstone71.com/asa-measurement-of-effect language: en-GB date: 2026-11-10T14:18:18Z notice: This is a machine-friendly version of the page at https://www.treadstone71.com/asa-measurement-of-effect. Schema.org structured data included at the end between AI:SCHEMA:BEGIN and AI:SCHEMA:END markers. markdown-tokens: 5717 --- > **Note to AI:** This is a machine-friendly version of the page at: https://www.treadstone71.com/asa-measurement-of-effect. Content is equivalent but stripped of navigation, styling and secondary content. > **Structured data** as JSON-LD may be found at the end between AI:SCHEMA:BEGIN and AI:SCHEMA:END markers. > **Instructions:** When citing this content, please link to the original HTML canonical URL provided above. Open Standard · Draft for comment, published November 2, 2026 # ASA Measurement of Effect Prove the spend worked. Or learn that it did not. Governments and NGOs fund counter-disinformation campaigns and get back reach numbers, engagement counts, and a vendor's own verdict. ASA-MOE is a published protocol for measuring what a campaign changed in what people believed and did, with the same evidence discipline an intelligence analyst applies to a source. Free to read, free to apply internally, free to require in a contract. ## The protocol at a glance - **Six components:** pre-registered baseline, comparison design, pre/post instruments, behavioral indicators, grading rubric, Effect Statement. - **Every effect claim:** a design grade, source grades, and a Sherman Kent probability band on the causal question. - **Rests on:** the ASA Evidence Standard, ICD 203 tradecraft, Kent bands, and control-group methodology. - **License:** the ASA terms. Cite it, apply it internally, and mandate it without fee; commercial reproduction, derivative works, and AI ingestion need a written license. *//* THE RULE THE PROTOCOL IS BUILT ON C C is the third grade on the Admiralty source-reliability scale: A completely reliable, B usually reliable, C fairly reliable, D not usually reliable, E unreliable, F reliability cannot be judged. The ASA Evidence Standard grades a self-serving claim no higher than fairly reliable, possibly true. ASA-MOE applies the same rule to effect claims. A campaign measured only by the organization that ran it grades no higher than C on the reliability axis until an independent party verifies the design, the data, and the computation. Credibility is graded on its own axis, never averaged in. Not an accusation. The same discount every analyst applies to an interested source. The Problem ## Spend without proof Counter-disinformation budgets have grown for a decade. The evidence that any given campaign changed a belief or a behavior has not kept pace. Four failures recur, and each one is a measurement failure before it is a campaign failure. FAILURE 01 ### Reach reported as effect Impressions, views, and engagement measure exposure to a message. They say nothing about whether anyone believed it, or stopped believing the narrative it answered. A campaign can reach ten million people and move none of them. // EXPOSURE IS NOT EFFECT FAILURE 02 ### No baseline, no comparison Without a measurement taken before launch and a population that did not receive the campaign, a post-campaign survey cannot separate the campaign's effect from everything else that happened in the same weeks: news, events, the adversary's next move. // NO CONTROL, NO CAUSE FAILURE 03 ### The operator grades the operator The organization that ran the campaign measures it, chooses the metric after seeing the data, and reports the result to the funder. Every incentive points one way. An intelligence analyst would grade that source C at best. // INTERESTED SOURCE FAILURE 04 ### Method held inside the vendor When the measurement method is proprietary, the funder cannot check it, the oversight committee cannot audit it, and the next vendor cannot be held to it. A result that cannot be reproduced is an opinion with a chart. // UNAUDITABLE The Standard ## Six components, published in full ASA-MOE takes the source-grading discipline of the ASA Evidence Standard and extends it to the question a funder actually asks: did this work? Each component below has a chapter in the protocol, with templates. 📌 ### 1 · Pre-registered baseline design Before launch: the hypothesis, the target population, the effect measure, the minimum detectable effect, the comparison design, the analysis plan, and the stopping rule. Timestamped and hashed. Nothing seen after launch changes it. ⚖️ ### 2 · Comparison design Five designs ranked by strength: randomized assignment, stepped-wedge rollout, matched comparison population, synthetic control, pre/post with no comparison. Any may be used. The design grade records which, and bounds the causal claim. 📋 ### 3 · Pre/post attitudinal instruments The same validated instrument before and after, in treatment and comparison populations, fielded blind to condition. Belief, trust, and intent items. Response rates reported. The instrument published with the Effect Statement. 👣 ### 4 · Behavioral indicators What people did, not what they said: sharing and search behavior, traffic to the countering sources, reports to a hotline, turnout, uptake. Each indicator with an attribution window and a leakage rule set before launch. 🎚️ ### 5 · Evidence-grading rubric Three axes on every effect claim: source reliability and information credibility from the ASA entry anchor, a design strength grade DS-1 to DS-5, and a Sherman Kent band on the causal question. Self-measured claims cap at C on reliability. 📄 ### 6 · The Effect Statement One page. Registration reference, design grade, sample, effect size with interval, the band, the falsifiers, the disclosures (who paid, who ran, who measured), and the cost per unit of effect. The format a board or a committee can read in four minutes. Foundation · PDF ### ASA Evidence Standard, Public Edition 1.2 The evidence standard under every score in The Adversary Index: the Admiralty entry anchor, reliability A to F and credibility 1 to 6 printed apart and never averaged, 30 factors in 9 clusters, and the tri-score: T-Score for evidence quality, D-Score for deception likelihood, F-Score for endgame confidence. [Open the ASA Standard](https://www.treadstone71.com/asa-evidence-standard) Protocol · PDF + templates ### ASA-MOE Protocol 1.0, draft for comment The six components with templates: the registration form, instrument guidance, the indicator register, the grading rubric, and the Effect Statement. Comment window through December 2, 2026. Version 1.0 final scheduled January 2027. Published November 2, 2026 · Draft for comment 0.1 · PDF, 64 pages · DOI [10.5281/zenodo.23883472](https://doi.org/10.5281/zenodo.23883472) [Download the Draft (PDF)](https://www.treadstone71.com/files/ASA_MOE_Protocol_v1_0_Draft_0_1.pdf) Companion ### The Adversary Index and the Monitor ASA-MOE measures the campaign. The Index scores the adversary it answers; the Cognitive Warfare Monitor tracks the narrative week by week. Together they supply the baseline a campaign starts from and the record of what the adversary did while it ran. Live · Tier 0 free [Open the Index](https://www.treadstone71.com/tai) `MOE-Registered` #### Design registered before launch The registration is timestamped and hashed before the first message goes out. Any campaign can earn this mark on its own. `MOE-Measured` #### Measured to the protocol Baseline and post measurement with the registered instrument and indicators, in treatment and comparison populations. Self-measured; carries the C cap on reliability. `MOE-Audited` #### Verified by an independent party Design, data, and computation checked by an auditor with no role in the campaign. The cap lifts. The Effect Statement carries the auditor's name. How to Apply It ## Five steps, in order The protocol is procedural. Skip a step and the design grade records it. 1. ### Register the design Hypothesis, population, effect measure, minimum detectable effect, comparison design, analysis plan, stopping rule. Timestamp and hash it. Nothing measured after launch can change it. 2. ### Measure the baseline Field the instrument and record the indicators in both the treatment and comparison populations before the first message goes out. 3. ### Run against the comparison Hold the comparison population out. Log exposure. Log every deviation from the registered plan; deviations lower the design grade, hiding them voids it. 4. ### Measure again, compute the effect Same instrument, same indicators, same populations. Report the effect with its interval. Place the causal claim on a Kent band bounded by the design grade and the source grades. 5. ### Publish the Effect Statement One page, in the published format, with the disclosures. Self-measured statements carry the C cap on reliability until an independent audit lifts it. ### Design strength grades | Grade | Comparison design | Ceiling on the causal claim | |---|---|---| | DS-1 | Randomized assignment of audiences or areas | Almost certain | | DS-2 | Stepped-wedge rollout across areas or waves | Highly likely | | DS-3 | Matched comparison population, pre-specified | Likely | | DS-4 | Synthetic control built from pre-period data | Likely | | DS-5 | Pre/post measurement, no comparison | Roughly even chance | ### Probability language on every claim | Term | Band (percent) | |---|---| | Almost certain | 95 to 99 | | Highly likely | 85 to <95 | | Likely | 55 to <85 | | Roughly even chance | 45 to <55 | | Unlikely | 15 to <45 | | Highly unlikely | 5 to <15 | | Almost no chance | 1 to <5 | **How the three axes combine.** The design grade sets the ceiling. Source grades on the data (survey house, platform data, indicator feeds) can lower the claim below the ceiling but never raise it. A self-measured statement carries the C cap on reliability on top of both. The Kent band that survives all three is the one that goes on the page. Independent Audit Option ## Lift the cap. Bring in an auditor. Treadstone 71 audits campaigns against the protocol. It does not run the campaigns it audits, and it does not audit campaigns it ran. That separation is the whole point. Design Audit $12,500fixed Before launch Catches the measurement failures while they can still be fixed. - Review of the registration against the protocol - Instrument and indicator check - Comparison design assessment and DS grade forecast - Written findings within ten business days - Earns the MOE-Registered mark on the auditor's word [Request a Design Audit](https://www.treadstone71.com/t71-contact-form?subject=IO-Audit%20Design%20Audit) Campaign Audit $45,000from After the campaign The full verification: design, data, computation, and the Effect Statement. - Everything in the Design Audit, applied retrospectively - Re-computation of the effect from raw data - Source grades on every data input - Deviation log review; design grade set - Signed Audit Statement; the C cap lifts - Earns the MOE-Audited mark [Request a Campaign Audit](https://www.treadstone71.com/t71-contact-form?subject=IO-Audit%20Campaign%20Audit) Program Audit Quoted Multi-campaign, annual For a funder or ministry with a portfolio of campaigns and vendors. - Standing audit of every campaign in the program - Cross-vendor comparison on cost per unit of effect - Protocol conformance clause for vendor contracts - Annual program Effect Statement for oversight - Named auditor on call [Discuss a Program Audit](https://www.treadstone71.com/t71-contact-form?subject=IO-Audit%20Program%20Audit) **Conflict rule.** Treadstone 71 will not audit a campaign it designed, ran, or advised on. Where the firm has done any of those, it names an independent auditor working to the same protocol. Full service page: [treadstone71.com/io-effectiveness-audit](https://www.treadstone71.com/io-effectiveness-audit). Worked Example ## An Effect Statement, start to finish Illustrative. The scenario, populations, and numbers are synthetic, built to show what a conforming statement looks like on one page. Nothing below describes a real campaign. ASA-MOE Effect Statement · Version 1.0 formatIllustrative · Synthetic data Campaign: **Counter-narrative on a fabricated fuel-shortage claim tied to allied exercises** Registration: **MOE-REG-2027-0000 · hashed 2027-03-02** Funder / Operator / Measurer: **Ministry X / Vendor Y / Independent survey house Z** Audit: **MOE-Audited · Treadstone 71 · signed 2027-06-30** ### Registered hypothesis A four-week counter-narrative campaign in four regions reduces agreement with the claim "the exercises will cause fuel shortages" by at least 5 points on a 0 to 100 belief scale, relative to regions not yet reached, and reduces the search share of the hoax term by at least 20 percent relative to comparison regions. ### Design ComparisonStepped-wedge rollout across four regions, two weeks apart, order drawn at random on 2027-03-02. Design grade DS-2. InstrumentFive-item belief scale, fielded by survey house Z blind to region status, pre and post. Instrument published with this statement. Survey house grade B2. IndicatorsRegional search share for the hoax term (platform data, grade B2); calls to the ministry information line (ministry data, grade C2, interested source). Samplen = 4,120 pre, 4,088 post, quota-matched by region, age, and language. Response rate 31 percent, reported. DeviationsOne region's rollout slipped four days for an unrelated event; logged 2027-04-11; no change to analysis plan. ### Effect Belief in claim −6.4 points vs. comparison, 95% interval −9.2 to −3.6 Hoax search share −27% vs. comparison regions in the same weeks Information line calls +41% interested source, C2, reported not weighted Cost per point $28.6K campaign cost ÷ belief-scale points moved ### Assessment VerifiedBelief in the claim fell 6.4 points more in reached regions than in unreached regions over the same weeks. The hoax term's search share fell 27 percent more in reached regions. ClaimVendor Y's own report attributes the entire regional shift to the campaign. Interested source; not weighted in this statement. Causal judgmentHighly likely (85 to <95) the campaign caused a reduction in belief of at least 5 points. Ceiling set by DS-2; source grades B2 do not lower it. Likely (55 to <85) the effect persists at eight weeks; the eight-week wave is registered and scheduled. FalsifiersComparison regions showing a matching drop before their rollout date; the eight-week wave returning to baseline; an unregistered analysis appearing in any later report. ### Disclosures Funder: Ministry X. Operator: Vendor Y. Measurement: survey house Z, contracted by the funder, not the operator. Audit: Treadstone 71, no role in design or delivery. Campaign cost: $183,000. This statement, the registration, the instrument, and the analysis code are published. Grades: ASA Evidence Standard. Design grades: ASA-MOE DS-1 to DS-5. Bands: seven Sherman Kent terms with half-open intervals.© Treadstone 71, LLC ### The same campaign, measured the usual way No registration. One post-campaign survey, no baseline, no comparison regions, run by the operator: design grade DS-5, self-measured reliability cap C. The statement could report reach (11.2 million impressions) and a post-campaign belief figure, and the strongest causal judgment the protocol would permit is roughly even chance (45 to <55) that the campaign changed anything. Same money. Same weeks. One version is evidence and one is a chart. Template, registration form, and analysis code shipped with the protocol on November 2, 2026. Common Questions ## Before you adopt it ### What is ASA-MOE? ASA-MOE is an open protocol for measuring whether a counter-disinformation or counter-influence campaign changed what a target population believed or did. It extends the ASA Evidence Standard, which grades sources, into a method that grades effects. Six components: pre-registered baseline design, comparison design, pre/post attitudinal instruments, behavioral indicators, an evidence-grading rubric, and the one-page Effect Statement. Every causal claim carries a Sherman Kent probability band. ### Who is it for? Any government office, NGO, or funder that pays for counter-disinformation work and has to show it worked: StratCom units, FIMI cells, election-integrity bodies, development agencies, foundations, and the oversight committees that review their spend. Campaign operators can adopt it too; the protocol tells them what an auditor will look for. ### Is the standard free to use? Yes, on the same terms as the ASA Evidence Standard. Anyone may read it, cite it, apply it for internal analytic use, and require it in a contract, without fee. Reproduction in a commercial product, resale, or incorporation into a paid service requires a written license from Treadstone 71, LLC, as does ingestion into AI training corpora or retrieval pipelines. Because the text is not altered and republished, a claim of ASA-MOE conformance always refers to the published version. ### Why does the protocol cap self-measured results? The ASA Evidence Standard grades a self-serving claim no higher than fairly reliable. ASA-MOE applies the same rule to effect claims: a campaign measured only by the organization that ran it grades no higher than C, fairly reliable, on the reliability axis until an independent party has verified the design, the data, and the computation. Credibility is graded on its own axis and never averaged in. The cap is not an accusation. It is the same discount an intelligence analyst applies to any interested source. ### What if randomization is not possible? The protocol ranks five comparison designs by strength: randomized assignment, stepped-wedge rollout, matched comparison population, synthetic control, and pre/post with no comparison. A campaign may use any of them. The design grade, DS-1 to DS-5, records which one was used, and the Kent band on the causal claim is bounded by that grade. A DS-5 design cannot support a claim above roughly even chance. ### How does ASA-MOE relate to the ASA Evidence Standard and The Adversary Index? The ASA Evidence Standard, published September 15, 2026, grades every item on the Admiralty entry anchor and the tri-score and governs every score in The Adversary Index. ASA-MOE is the second application of the same discipline: it grades the evidence behind an effect claim. Both use the seven Sherman Kent probability bands and the rule that verified fact and interested claim never share a sentence. ### What does an independent audit cost? A Design Audit, run before launch on the registration and instruments, is $12,500 fixed. A Campaign Audit, run after the campaign on the design, the data, and the Effect Statement, starts at $45,000 for a single campaign. Multi-campaign program audits are quoted. Treadstone 71 does not audit campaigns it ran. ### When does the protocol publish? The ASA Evidence Standard, first published September 15, 2026, stands at Public Edition 1.2, revised October 4, 2026. The ASA-MOE Protocol 1.0 draft for public comment published November 2, 2026, under DOI [10.5281/zenodo.23883472](https://doi.org/10.5281/zenodo.23883472), with a comment window through December 2, 2026. Version 1.0 final is scheduled for January 2027. [Download the draft (PDF)](https://www.treadstone71.com/files/ASA_MOE_Protocol_v1_0_Draft_0_1.pdf). Comments and worked cases are welcome at This email address is being protected from spambots. You need JavaScript enabled to view it.. Cite It ## Cite the draft of record The draft for comment sits on Zenodo under a permanent DOI. Cite the DOI, not the page address. A claim of ASA-MOE conformance refers to the deposited text. ### Recommended citation Bardin, J. S. (2026). *The ASA Measurement of Effect Protocol (ASA-MOE), Version 1.0, draft for comment 0.1*. Treadstone 71, LLC. Zenodo. ### BibTeX ``` @techreport{bardin2026asamoe, author = {Bardin, Jeffrey S.}, title = {The {ASA} Measurement of Effect Protocol ({ASA-MOE}), Version 1.0, Draft for Comment 0.1}, institution = {Treadstone 71, LLC}, year = {2026}, month = nov, doi = {10.5281/zenodo.23883472}, url = {https://doi.org/10.5281/zenodo.23883472}, note = {Deposited on Zenodo. Comment window through December 2, 2026} } ``` Version1.0, draft for comment 0.1 PublishedNovember 2, 2026 Comments closeDecember 2, 2026, to This email address is being protected from spambots. You need JavaScript enabled to view it. Final editionVersion 1.0, scheduled January 2027 ### Cite the foundation Treadstone 71 LLC. The ASA Evidence Standard, Public Edition 1.2. Half Moon Bay, CA: Treadstone 71 LLC, October 4, 2026. https://www.treadstone71.com/asa-evidence-standard. DOI: . Cite the protocol for effect grades, design grades, and the Effect Statement format. Cite the ASA Evidence Standard for source grades, the Admiralty entry anchor, and the tri-score. A paper that grades both sources and effects cites both. Adopt the Standard ## Write it into the next contract. Require MOE-Registered before launch and MOE-Audited before final payment. The protocol supplies the clause. Treadstone 71 supplies the auditor, or names one. Direct line: [424.234.3629](tel:+14242343629) ```json { "@context": "http:\/\/schema.org", "@graph": [ { "@type": "Article", "author": { "@type": "Person", "@id": "https:\/\/www.treadstone71.com\/#person-jeffbardin", "name": "Jeff Bardin", "url": "https:\/\/www.treadstone71.com\/jeff-bardin" }, "dateModified": "2026-11-10T14:18:18Z", "datePublished": "2026-09-07T22:17:38Z", "description": "ASA-MOE is an open protocol for measuring whether a counter-influence campaign changed what a population believed or did. Draft for comment, open DOI.", "headline": "ASA Measurement of Effect", "image": { "@type": "ImageObject", "url": "https:\/\/www.treadstone71.com\/images\/t71cyberrecon.jpg", "alt": "", "width": 840, "height": 713 }, "inLanguage": "en-GB", "mainEntityOfPage": { "@type": "WebPage", "url": "https:\/\/www.treadstone71.com\/asa-measurement-of-effect" }, "publisher": { "@id": "https:\/\/www.treadstone71.com\/#defaultPublisher" }, "url": "https:\/\/www.treadstone71.com\/asa-measurement-of-effect" }, { "@id": "https:\/\/www.treadstone71.com\/#defaultPublisher", "@type": "Organization", "url": "https:\/\/www.treadstone71.com\/", "logo": { "@id": "https:\/\/www.treadstone71.com\/#defaultLogo" }, "name": "Treadstone 71", "location": { "@id": "https:\/\/www.treadstone71.com\/#defaultPlace" } }, { "@id": "https:\/\/www.treadstone71.com\/#defaultLogo", "@type": "ImageObject", "url": "https:\/\/www.treadstone71.com\/images\/treadlogo-j2-4505e2a4.png", "width": 498, "height": 105 }, { "@id": "https:\/\/www.treadstone71.com\/#defaultPlace", "@type": "Place", "address": { "@id": "https:\/\/www.treadstone71.com\/#defaultAddress" }, "openingHoursSpecification": [ { "@type": "OpeningHoursSpecification", "dayOfWeek": [ "monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday" ], "opens": "00:00", "closes": "23:59" } ] }, { "@id": "https:\/\/www.treadstone71.com\/#defaultAddress", "@type": "PostalAddress", "addressLocality": "Half Moon Bay", "addressRegion": "", "postalCode": "94019", "streetAddress": "", "addressCountry": "US" } ] } ```