Short answer
A platform obligation ai image review rarely starts with detection. It starts with notice. In most regulatory frameworks the duty to act is triggered by a report from a user, a court order, or a flagged advertising placement, not by the platform noticing something on its own. Detection is how you handle scale once the obligation exists, not the trigger itself.
Why teams get the sequence wrong
Moderation stacks are usually built around models first and process second. That works for spam, where volume is the problem. It works badly for synthetic imagery, because the platform's legal position depends on what it knew and when. A detector that scores an image at 0.9 has told you nothing about whether you had a duty to look.
The three triggers that create an obligation
- User notice: a report from a depicted person, a rights holder, or any user. Most regimes require a timely response once notice is given, and the clock usually starts the moment the report lands rather than when a reviewer opens it.
- Regulatory designation: certain categories, such as election communication during a defined window or content involving minors, carry elevated duties regardless of notice.
- Contractual commitment: platform terms and advertiser agreements often create obligations broader than the law requires. These are the ones that surface in disputes, because the platform wrote them itself.
What to record before you decide
- The timestamp of first notice, distinct from the timestamp of first review.
- The image as delivered, not a screenshot taken later. Re-encoding removes the evidence a reviewer would need.
- Which signals were consulted and what each returned, including the ones that disagreed.
- The decision and its stated basis, in language a later reviewer can audit.
Where detection helps and where it misleads
Detection is genuinely useful for triage: it lets you route a queue and prioritise the cases a human should open first. It is not useful for establishing whether the content was synthesized, because a high score means the file does not resemble a camera original, which is also true of edited authentic images. A missing provenance record is not proof of fabrication; it is a statement about the transport path. Where the score and the provenance record conflict, escalate to human review rather than resolving it in the model's favour. Read the methodology for how these layers are combined, or look at sample reports to see how conflicting signals are presented.
If you need to see provenance, metadata, and detector output side by side on a file you already hold, start a free review and read all three before writing a decision.
What this workflow cannot settle
It cannot tell you whether an image is authentic. It cannot reconstruct edits made before the first copy you received. And it cannot substitute for a legal reading of the rules that apply to your jurisdiction, your content category, and the commitments you published in your own terms.