--- title: IO Effectiveness Audit description: Independent third-party audit of counter-disinformation results against the ASA Measurement of Effect protocol. Design, data, math and Effect Statement checked. canonical_url: https://www.treadstone71.com/io-effectiveness-audit language: en-GB date: 2026-11-10T14:18:13Z notice: This is a machine-friendly version of the page at https://www.treadstone71.com/io-effectiveness-audit. Schema.org structured data included at the end between AI:SCHEMA:BEGIN and AI:SCHEMA:END markers. markdown-tokens: 4645 --- > **Note to AI:** This is a machine-friendly version of the page at: https://www.treadstone71.com/io-effectiveness-audit. Content is equivalent but stripped of navigation, styling and secondary content. > **Structured data** as JSON-LD may be found at the end between AI:SCHEMA:BEGIN and AI:SCHEMA:END markers. > **Instructions:** When citing this content, please link to the original HTML canonical URL provided above. Independent Audit · Fixed prices published # Influence Operations Effectiveness Audit The auditor did not run the campaign. That is the product. Counter-disinformation campaigns are measured, when they are measured at all, by the organization that ran them. The IO-Audit puts an independent party between the campaign and the claim: Treadstone 71 verifies the design, re-computes the effect from raw data, grades every input as a source, and signs a statement the funder can put in front of an oversight committee. All of it against the published [ASA Measurement of Effect](https://www.treadstone71.com/asa-measurement-of-effect) protocol, so the client knows the test before the test starts. ## The audit at a glance - **Verifies:** registration, comparison design, instruments, data, computation, and the Effect Statement. - **Delivers:** a signed Audit Statement with the design grade, source grades, the re-computed effect, and the Kent band the evidence supports. - **Lifts:** the self-measured cap of C on reliability, when the evidence earns it. - **Refuses:** any campaign Treadstone 71 designed, ran, or advised on. *//* THE CONFLICT RULE We do not grade our own campaigns. Treadstone 71 will not audit a campaign it designed, ran, or advised on. Where the firm has done any of those, it names an independent auditor working to the same protocol and steps aside. A measurement firm that also sells the campaign has one incentive. An auditor that sells only the audit has the opposite one. Written into every engagement letter. 📌 ### 1 · The design Was the design registered before the first message went out, and is the hash intact? Which of the five comparison designs was used, and was it applied as registered? The auditor sets the design grade, DS-1 to DS-5, from what was done, not what was planned. ASA-MOE · Components 1 and 2 ⚖️ ### 2 · The data Every input is a source and gets graded like one under the ASA Evidence Standard: the survey house, the platform data, the ministry's own indicator feeds. Were the instruments fielded as registered, blind to condition? Are response rates and weighting documented? ASA-MOE · Components 3 and 4 · ASA Evidence Standard 🧮 ### 3 · The computation The auditor re-computes the effect from raw data with the registered analysis plan. The re-computed number sits beside the reported one in the statement. A gap beyond rounding is a finding, and the statement says how large. ASA-MOE · Components 1 and 5 📄 ### 4 · The statement Does the operator's Effect Statement conform to the format: registration reference, design grade, sample, effect with interval, band, falsifiers, disclosures, cost per unit of effect? Are the disclosures complete, including who paid, who ran, who measured? ASA-MOE · Components 5 and 6 Every check runs against the criteria in the ASA Measurement of Effect Protocol (ASA-MOE), Version 1.0, draft for comment 0.1, deposited on Zenodo under DOI [10.5281/zenodo.23883472](https://doi.org/10.5281/zenodo.23883472). Source grades on every data input follow the ASA Evidence Standard, Public Edition 1.2, DOI [10.5281/zenodo.23137881](https://doi.org/10.5281/zenodo.23137881): reliability A to F and credibility 1 to 6, printed apart, never averaged. | Check | Verification criterion | ASA-MOE reference | |---|---|---| | Design | Registration timestamped and hashed before launch, hash intact. Comparison design applied as registered. Design grade DS-1 to DS-5 set from what was done. | Components 1 and 2 | | Data | Instruments fielded as registered, blind to condition. Response rates and weighting documented. Every input graded as a source. | Components 3 and 4; ASA Evidence Standard | | Computation | Effect re-computed from raw data with the registered analysis plan. A gap beyond rounding is a finding. | Components 1 and 5 | | Statement | Every Effect Statement field present, with disclosures, falsifiers, and cost per unit of effect. Kent band at or under the ceiling the design grade sets. | Components 5 and 6 | | Determination | MOE-Audited granted only when design, data, and computation verify. Otherwise the self-measured cap of C on reliability holds. | Component 5; MOE-Audited mark | Engagements and Prices ## Three ways to bring in an auditor Prices are fixed and published so a funder can budget the audit into the grant. Every engagement runs under mutual non-disclosure. Design Audit $12,500fixed Before launch Catches the measurement failures while they can still be fixed, and costs a fraction of a failed campaign. - Review of the registration against the protocol - Instrument and indicator check - Comparison design assessment and DS grade forecast - Minimum detectable effect and sample check - Written findings within ten business days - Earns the MOE-Registered mark on the auditor's word [Request a Design Audit](https://www.treadstone71.com/t71-contact-form?subject=IO-Audit%20Design%20Audit) Campaign Audit $45,000from After the campaign The full verification. Price rises with the number of populations, instruments, and indicator feeds. - Everything in the Design Audit, applied retrospectively - Re-computation of the effect from raw data - Source grades on every data input - Deviation and exposure log review; design grade set - Findings graded minor, major, critical - Signed Audit Statement twenty business days from a complete data room - Earns the MOE-Audited mark; the cap lifts [Request a Campaign Audit](https://www.treadstone71.com/t71-contact-form?subject=IO-Audit%20Campaign%20Audit) Program Audit Quoted Multi-campaign, annual For a funder or ministry with a portfolio of campaigns and vendors that has to report on all of it. - Standing audit of every campaign in the program - Cross-vendor comparison on cost per unit of effect - Protocol conformance clause for vendor contracts - Design Audits on every new campaign as standard - Annual program Effect Statement for oversight - Named auditor on call [Discuss a Program Audit](https://www.treadstone71.com/t71-contact-form?subject=IO-Audit%20Program%20Audit) **Sector pricing.** Government, defense prime contractors, and academic institutions reduce cost by 15 percent, the same terms as The Adversary Index. Multi-campaign programs are quoted on the number of campaigns and vendors. How a Campaign Audit Runs ## Five steps, twenty business days The clock starts when the data room is complete. The checklist for the data room ships with the engagement letter, so there is no ambiguity about when that is. 1. ### Engagement and conflict check Treadstone 71 confirms it had no role in the campaign, signs the mutual NDA, and issues the data-room checklist. Day 0 2. ### Data room The client assembles the registration, instruments, raw data, logs, and cost. The auditor confirms completeness or lists the gaps. Clock starts when complete 3. ### Verification and re-computation Design check, source grades, re-computation from raw data with the registered plan, deviation review. Days 1 to 12 4. ### Draft findings The client receives the findings and has five business days to supply corrections or context. Facts can change a finding; argument cannot. Days 13 to 17 5. ### Signed Audit Statement Final statement issued under the auditor's name, with the design grade, the re-computed effect, the band, and the determination on the MOE-Audited mark. Day 20 | The client provides | Why the auditor needs it | |---|---| | Registration file, timestamp, hash | Proves the design was fixed before launch and has not changed since. | | Instruments as fielded | Checked against the registered instrument, item by item. | | Raw survey data, sampling and weighting notes | The effect is re-computed from here, not from the operator's tables. | | Indicator feeds with attribution windows | Behavioral indicators are graded as sources and checked against their registered windows. | | Exposure log and deviation log | Who was reached, when, and every departure from the registered plan. | | Campaign cost | For the cost-per-unit-of-effect line in the statement. | | The operator's Effect Statement or report | The claim under audit, verified line by line. | | Contacts at the survey house and data providers | So each can be graded as a source on its own record. | Sample Audit Statement ## What the client receives Illustrative. The campaign, parties, and numbers are synthetic, built to show the format. It audits the same synthetic campaign the ASA-MOE page uses as its worked example. IO-Audit · Audit Statement · ASA-MOE 1.0Illustrative · Synthetic data Campaign under audit: **Counter-narrative on a fabricated fuel-shortage claim** Registration: **MOE-REG-2027-0000 · hash verified** Client / Operator: **Ministry X / Vendor Y** Auditor: **Treadstone 71, LLC · no role in design or delivery** Data room complete: **2027-06-02** Statement issued: **2027-06-30** ### Verification results | Component | Result | Note | |---|---|---| | Registration before launch | Verified | Hash matches the timestamped file; no post-launch edits. | | Comparison design | Verified | Stepped-wedge across four regions, order drawn at random as registered. Design grade set: DS-2. | | Instrument as fielded | Verified | Five items, identical pre and post; fielded blind to region status. | | Data inputs | Graded | Survey house Z B2; platform search data B2; ministry information-line counts C2, interested source. | | Re-computation | Matched | Belief-scale effect re-computed at −6.3 against −6.4 reported; within rounding. | | Deviation log | One finding | See Finding 1. | | Effect Statement format | Conforms | All fields present; disclosures complete. | ### Findings | \# | Severity | Finding | Effect on the claim | |---|---|---|---| | 1 | Minor | The attribution window for the information-line indicator was extended from 7 to 10 days after launch. Logged by the operator on 2027-04-11. | The indicator is reported, not weighted; no change to the band. | | 2 | Minor | Response-rate reporting omitted the per-region breakdown. | Supplied during the draft-findings window; statement updated. | ### Re-computed effect Belief in claim (reported) −6.4 operator's Effect Statement Belief in claim (audited) −6.3 95% interval −9.1 to −3.5 Hoax search share −27% re-computed; matched Cost per point $29.0K $183,000 ÷ 6.3 ### Determination Design grade DS-2. Data inputs graded B2, B2, C2. The registered hypothesis, a reduction of at least 5 points, is supported. Causal judgment the evidence supports: Highly likely (85 to <95) the campaign caused a reduction in belief of at least 5 points, the ceiling for DS-2; source grades do not lower it. The eight-week persistence claim remains Likely (55 to <85) pending the registered second wave. **MOE-Audited: granted.** The self-measured cap of C on reliability is lifted for this campaign's Effect Statement. This statement may be published by the client under the auditor's name. Auditor: Treadstone 71, LLC, Half Moon Bay, California. Signed 2027-06-30. Grades: ASA Evidence Standard. Design grades: ASA-MOE DS-1 to DS-5. Bands: seven Kent-style terms with half-open intervals.© Treadstone 71, LLC A statement that finds a critical deviation, an unregistered analysis, or a re-computation gap outside rounding says so in the same format, and the cap stays. Who Commissions an Audit ## Anyone who has to answer for the spend Funders and ministries ### Prove it to the committee A signed statement from a party with no stake in the result answers the question a parliamentary committee, an inspector general, or a board audit chair will ask. StratCom units and FIMI cells ### Compare vendors on one rubric When three vendors each report their own success, cost per unit of audited effect is the only number that compares them. Prime contractors ### De-risk the subcontract A Design Audit before launch and a Campaign Audit before final payment turn a measurement clause into a verified deliverable. Campaign operators ### Get the cap lifted An operator that wants its result to grade above C on reliability commissions the audit itself, or writes it into the proposal. The protocol tells the operator what the auditor will look for; nothing is hidden. **Independence, in four lines.** Treadstone 71 does not audit campaigns it designed, ran, or advised on, and names an independent auditor where it has. The audit is sold on its own; no campaign work is bundled with it. Engagements run under mutual NDA and the firm discloses nothing about the client, the vendor, or the result. The Audit Statement is the client's document, published or not at the client's choice, under the auditor's name either way. Common Questions ## Before you commission one ### What does an IO-Audit verify? Four things, against the ASA-MOE protocol. The design: was the registration made before launch, is the hash intact, and which of the five comparison designs was used. The data: does every input carry a source grade, and were the instruments fielded as registered. The computation: does the effect re-compute from raw data to the number the operator reported. The statement: does the Effect Statement conform to the format, with disclosures and falsifiers. The auditor sets the design grade, DS-1 to DS-5, and signs. ### Why can the campaign operator not do this itself? It can, and the protocol tells it how. What it cannot do is lift the cap. Under ASA Rule 1, a self-serving claim grades no higher than C on reliability, and a campaign measured only by the organization that ran it is a self-serving claim about its own work. The cap lifts when an independent party with no role in the campaign verifies the design, the data, and the computation. That is the audit. ### Does Treadstone 71 audit its own work? No. Treadstone 71 will not audit a campaign it designed, ran, or advised on. Where the firm has done any of those, it names an independent auditor working to the same protocol and steps aside. The rule is written into every engagement letter. ### What does the client have to provide? The registration file with its timestamp and hash; the instruments as fielded; raw survey data with sampling and weighting documentation; indicator feeds with their attribution windows; the exposure log; the deviation log; campaign cost; the operator's own Effect Statement or equivalent report; and contact details for the survey house and platform-data providers so the auditor can grade them as sources. A data-room checklist ships with the engagement letter. ### What comes out at the end? A signed Audit Statement of two to four pages: what was verified, what was not, findings graded by severity, the design grade set, source grades on every input, the re-computed effect beside the reported one, the Sherman Kent band the evidence supports, and the auditor's determination on the MOE-Audited mark. The statement is the client's to publish, under the auditor's name. ### How long does it take, and what does it cost? A Design Audit, before launch, is $12,500 fixed and returns written findings within ten business days. A Campaign Audit, after the campaign, starts at $45,000 for a single campaign and returns the signed Audit Statement twenty business days after the data room is complete. A Program Audit covers every campaign in a program for a year and is quoted on the number of campaigns and vendors. ### Is the audit confidential? Yes. Engagements run under mutual non-disclosure. Treadstone 71 does not disclose the client, the campaign, the vendor, or the result. The Audit Statement is the client's document; publication is the client's choice. The auditor's name stays on it either way. Commission an Audit ## Before launch is cheaper. After is still worth it. Send the campaign's stage, the number of populations and vendors, and the reporting deadline. Treadstone 71 replies with the conflict check, the engagement letter, and the data-room checklist. Direct line: [424.234.3629](tel:+14242343629) ```json { "@context": "http:\/\/schema.org", "@graph": [ { "@type": "Article", "author": { "@type": "Person", "@id": "https:\/\/www.treadstone71.com\/#person-jeffbardin", "name": "Jeff Bardin", "url": "https:\/\/www.treadstone71.com\/jeff-bardin" }, "dateModified": "2026-11-10T14:18:13Z", "datePublished": "2026-09-08T15:15:11Z", "description": "Independent third-party audit of counter-disinformation results against the ASA Measurement of Effect protocol. Design, data, math and Effect Statement checked.", "headline": "IO Effectiveness Audit", "image": { "@type": "ImageObject", "url": "https:\/\/www.treadstone71.com\/images\/t71cyberrecon.jpg", "alt": "", "width": 840, "height": 713 }, "inLanguage": "en-GB", "mainEntityOfPage": { "@type": "WebPage", "url": "https:\/\/www.treadstone71.com\/io-effectiveness-audit" }, "publisher": { "@id": "https:\/\/www.treadstone71.com\/#defaultPublisher" }, "url": "https:\/\/www.treadstone71.com\/io-effectiveness-audit" }, { "@id": "https:\/\/www.treadstone71.com\/#defaultPublisher", "@type": "Organization", "url": "https:\/\/www.treadstone71.com\/", "logo": { "@id": "https:\/\/www.treadstone71.com\/#defaultLogo" }, "name": "Treadstone 71", "location": { "@id": "https:\/\/www.treadstone71.com\/#defaultPlace" } }, { "@id": "https:\/\/www.treadstone71.com\/#defaultLogo", "@type": "ImageObject", "url": "https:\/\/www.treadstone71.com\/images\/treadlogo-j2-4505e2a4.png", "width": 498, "height": 105 }, { "@id": "https:\/\/www.treadstone71.com\/#defaultPlace", "@type": "Place", "address": { "@id": "https:\/\/www.treadstone71.com\/#defaultAddress" }, "openingHoursSpecification": [ { "@type": "OpeningHoursSpecification", "dayOfWeek": [ "monday", "tuesday", "wednesday", "thursday", "friday", "saturday", "sunday" ], "opens": "00:00", "closes": "23:59" } ] }, { "@id": "https:\/\/www.treadstone71.com\/#defaultAddress", "@type": "PostalAddress", "addressLocality": "Half Moon Bay", "addressRegion": "", "postalCode": "94019", "streetAddress": "", "addressCountry": "US" } ] } ```