Platform Accountability Index · v0.1 — design
Measuring what the platforms did
Reporting content to a platform asks it to act. Recording what it then actually did is a separate piece of work, and it is work Civic Shield intends to publish. Nothing on this page has been measured. This is the method, published before the first figure exists.
The measurement
What the Index will measure
Platforms publish a great deal about the volume of content they act on. Far less is available about what happened to a specific, properly formed notice: how long it took, what was done, whether it survived an appeal, and whether the content came back [1]. Those are the things Civic Shield intends to count.
The unit of measurement will be the notice Civic Shield sent and the outcome that followed it, drawn from our own records at both ends. Every measurement is to be broken down by platform, country, language and content category, and no figure will be published without the size of the sample it came from.
-
Time to first response
The interval between a properly formed notice leaving Civic Shield and the first substantive answer from the service. Measured from our own outbound record and our own clock, not from a figure a service reports about itself.
Denominator · every notice sent, including the ones never answered -
Action rate
The share of notices that resulted in the content being removed, restricted, labelled or geo-blocked — counted against everything that was sent, not against everything that happened to be answered.
No response is counted as no action, never as a missing value -
Appeal outcomes
What happened when a decision was contested: upheld, reversed, or never resolved. Reversals will be reported in both directions, including the appeals Civic Shield loses, because a measurement that only records the other side being wrong is not a measurement.
Open at the cut-off is reported as open -
Re-upload rate
Whether content that was removed reappeared on the same service inside a defined observation window. A removal that does not hold is a different outcome from a removal, and merging the two would flatter everyone involved.
Observation window fixed before the period opens -
Consistency across languages and countries
The same category of content, notified in different languages and from different European contexts, and whether the outcome differs. This is the comparison a single market invites and the one that is hardest to make from the inside.
Breakdowns · platform · country · language · category
Why this organisation can commit to it
Independence is a method requirement, not a boast
A measurement is only worth reading when the people producing it have nothing riding on the answer. That is a plain statement about measurement, and it is the reason this page exists at all.
Civic Shield holds no platform partnership, no commercial relationship with any service it would measure, no advertising or platform revenue, and no regulatory designation. It is not a trusted flagger under Article 22 of the Digital Services Act and does not describe itself as one [1]. There is no access, contract, seat or status that an unwelcome result could cost it, which is why it can commit to publishing the result whatever it turns out to be.
This is a description of how Civic Shield is built. It is not a claim about anyone else’s work. Organisations that work directly with platforms do things this organisation cannot do and should not attempt, and the field needs both kinds. The point is only that the two jobs are different, and that this one is the job Civic Shield is choosing.
Methodology commitments
The rules we are agreeing to before we have any results
These are the constraints that make the difference between a measurement and an advertisement. They are written down now, when we do not yet know which of them will be inconvenient.
- The sample is fixed before the result is seen.
- The sample, the observation window and the metric definitions will be registered and published before a measurement period opens. Notices will not be selected into the Index after their outcome is known, and the denominator is always the full set sent.
- Every figure carries an interval and a count.
- A proportion drawn from a small number of notices is mostly noise. Each published figure will carry an uncertainty interval and the count it was computed from, and where the sample is too small to support a comparison the Index will say so instead of drawing the comparison anyway.
- Appeals are counted, not assumed.
- An appeal is a separate event with its own outcome and its own clock [1]. Cases still open at the cut-off will be reported as open rather than resolved in whichever direction suits the story, and the cut-off date is stated on the edition.
- Right of reply, before publication.
- Each service measured will receive the figures and the method that produced them ahead of publication, with a stated window to respond. Its response will be published with the edition. Where it demonstrates an error, the figure is corrected before publication and the correction is recorded.
- Versioning, and the cost of changing the method.
- Each edition is numbered and dated. Changing a metric definition breaks comparison with earlier editions, so it raises the methodology version, is described in the changelog, and the affected series is marked as not comparable rather than quietly restated.
- Corrections are published, not applied.
- A figure found to be wrong is corrected on the page that carried it, with the date and what changed, and the version is raised. A withdrawn figure is marked withdrawn and left visible. Removing a mistake without a trace is indistinguishable from removing an inconvenient result.
- A person decides what is strong enough to publish.
- Automation may prepare the counts. Whether a set of counts is strong enough to be published as a finding is a human decision, taken by a named reviewer and logged like every other decision in the record — the same rule that governs every escalation described in the methodology note.
- The underlying method is available to be attacked.
- Definitions, sampling frame, coding rules and the counting code will be published with the edition, so that a disagreement can be about the arithmetic rather than about whether to trust us.
Alongside it
Aggregate statistics — and nothing but aggregates
The Index measures platform behaviour. Beside it, Civic Shield intends to publish aggregate statistics about the material itself. Aggregates only: the statistics are a shape, never a specimen.
What the statistics will contain
- Counts.
- How much was recorded in a period, against a stated collection scope.
- Trends.
- Movement over time on a consistent definition, with any change of definition marked as a break in the series rather than smoothed over.
- Categories.
- The taxonomy, including protected criticism as a first-class outcome rather than a discard.
- Languages.
- Which languages the material was in, which is also the fairest available test of where a moderation system is weakest.
- Platforms and source types.
- Where it was found, including the open web — sites, blogs, newsletters and feeds that no notice-and-action mechanism covers.
What they will never contain
- No post content.
- No antisemitic text, image, symbol, quotation, paraphrase or screenshot. Reprinting is distribution, not documentation.
- No usernames.
- No handles, display names, profile links, account identifiers or post URLs.
- No individuals, at any level of aggregation.
- The unit is a category, a language, a country and a period. It is never a person, and there is no drill-down that arrives at one.
- No cell small enough to identify anyone.
- A suppression threshold will be published with the schema and applied before anyone sees the numbers. Cells below it are suppressed, and the suppression itself is shown rather than hidden.
- No live counter.
- Figures are published as dated, versioned editions. A number that updates itself is a number nobody can cite.
The regulatory hook
Where this connects to European law
The Digital Services Act does not require platforms to be perfect. It requires the largest services to identify the systemic risks arising from the design and operation of their services, to mitigate them, and to report on both [1]. Those obligations are assessed against evidence, and much of the available evidence about what a service does with what it receives is produced by that service.
- Notice and action — Article 16. The mechanism a notice goes into. The Index measures what comes back out of it.
- Internal complaint-handling — Article 20. The appeal route. A reversal rate is a direct measurement of how well it works.
- Systemic risk assessment and mitigation — Articles 34 and 35. Where independent outcome data, broken down by language and country, is most obviously relevant.
- Transparency reporting — Article 42. Platform self-reporting, against which an outside measurement can be compared.
- Researcher data access — Article 40. A route that may become available to Civic Shield once it is a qualifying legal entity. It is not available today, and the Index is deliberately designed so that it never depends on being granted.
Civic Shield holds no enforcement power and seeks none. The Index will be a measurement, published for regulators, journalists, courts, community bodies and the public to use or to argue with. It is not a finding of illegality about any company, and it will not be written as though it were.
What the Index is not
It is not a register of people. Civic Shield does not publish blacklists, shaming indexes or scoreboards of individuals, and that commitment is unchanged by this page. The Index measures companies against their own published rules and their obligations in law. No person is named, ranked, scored or profiled in it.
It is not a prediction. No forecast, no projection and no modelled estimate will be presented as an observation. What is counted is what happened.
It is not a campaign. The Index is not built to produce a particular answer about a particular company, and an edition that finds a service responding well will say so as plainly as one that finds the opposite.
It does not exist yet. Nothing on this page has been measured, and no edition has been published.
What has to be true before the first edition
The Index cannot run ahead of the organisation. Each item below blocks it, and none of them is a formality.
-
A legal entity, and everything else on the roadmap
The registration, the legal opinions and the data protection impact assessment listed on the roadmap all sit in front of this. Nothing operational starts before they close.
Blocks · every measurement -
Notices actually sent
There is nothing to measure until Civic Shield has sent something and waited for an answer. The Index depends on the founding programme having run; it is what that programme makes possible, not a parallel activity.
Blocks · the first period -
A fixed taxonomy and an evaluated classifier
Categories that move mid-period make every breakdown meaningless. The taxonomy and the error rates come first, as set out in the methodology note.
Blocks · category and language breakdowns -
A sample large enough to say anything
If the founding programme produces too few notices to support a published comparison, the first edition will publish the method, the raw counts and the reason no comparison is being drawn. That outcome is planned for. It is not a result to be hidden by publishing a comparison anyway.
Determines · what the first edition can claim -
The method registered before the window opens
Definitions, sampling frame and cut-off dates published in advance of the measurement period they govern. This page is the first instalment of that.
Blocks · publication of any figure
Changelog
-
v0.1 — 4 August 2026
First publication. Design and commitments only. No measurement has been taken, no platform has been assessed, and no data exists.
References
Sources
- Regulation (EU) 2022/2065 — Digital Services Act. Article 16 (notice and action), Article 20 (internal complaint-handling), Article 22 (trusted flaggers, cited only to state that Civic Shield does not hold that status), Articles 34 and 35 (systemic risk assessment and mitigation), Article 40 (data access for vetted researchers), Article 42 (transparency reporting).
- Regulation (EU) 2016/679 — General Data Protection Regulation, Recital 26: the principles of data protection do not apply to anonymous information or to personal data rendered anonymous such that the data subject is no longer identifiable.
Next step
The founding year is what makes the first edition possible.
Funders, regulators, journalists and researchers can request the due-diligence pack and a 30-minute briefing. The most useful thing you can do before the method is used is to tell us where it is wrong.