Why a second chance beats a hard block for open-text checks

Every platform that pays people for open-text answers has to decide what to do when an answer looks wrong. The usual answer is a hard block: the response is discarded, the participant is flagged, sometimes the account is closed. We made a different choice for the open-text check that runs before a participant reaches a study. A participant whose answer fails gets one more attempt, with a hint. Only a second failure closes the session. This post explains what the check looks for, why the second attempt is not leniency, and what we have learned from building it.

What the open-text check is for

Buyers of survey data reverse completions for a short list of reasons that has barely changed in years: rushed answers, contradictions, straightlining, and open-text fields that are empty, copied or generated. The last group has grown fastest. Chat assistants make it trivial to produce a fluent paragraph that says nothing about the respondent, and an answer pasted from somewhere else looks just as fluent.

Our check sits between the match and the redirect. A participant who has been matched to a study answers a short block that mirrors what buyers later look for: a free-text question read for effort, a consistency pair, an attention item, and inputs that a human never sees but automation fills in. The block is scored in the request, deterministically, without an external service, and the score decides whether the participant proceeds to the study.

What counts as a hard signal

Most of the block is measured and weighed rather than enforced, because most of it is a matter of degree. Three things are not a matter of degree:

• A hidden input with a value. The field was not visible to a person. Something else filled it in.
• Assistant text. An answer that refers to itself as an assistant, that frames its own output as a suggestion ("you could write this:"), or that carries the structure of a chat reply rather than of a person answering a question.
• A copied answer. Free text that is a near-duplicate of another session's answer, above a length where coincidence is implausible.

Each of these is a strong sign that the answer was not written by the participant in that moment. None of them is proof.

Why not simply block

The argument for a hard block is efficiency: a suspicious answer costs nothing to discard, and a buyer never sees it. The argument against it is what a hard block does to the honest minority inside every suspicious pattern, and what that costs the platform.

Honest people trigger strong signals

A participant who drafted an answer in a notes app and pasted it in produces a paste. A participant who answered a similar question yesterday and remembers what they wrote produces a near-duplicate of their own text. A browser extension that auto-fills forms fills hidden fields. These are not edge cases invented for the sake of argument; they turn up in review every week. A hard block treats each of them as fraud, and the participant never learns why the surveys stopped coming.

Silent blocks teach nothing

The person operating an automation learns nothing from a block either, but they also lose nothing: they open a new session. The honest participant, by contrast, loses a survey and often their trust in the platform. A hard block is therefore asymmetric in the wrong direction. It punishes the people who would have corrected course and barely inconveniences the ones who would not.

Reversals are the real cost

What buyers actually penalise is the completion that reaches them and turns out to be worthless. A check that catches that before the redirect saves the reversal; a check that additionally throws away honest completions saves nothing extra and costs inventory. The goal is to stop bad answers reaching the study, not to remove people.

What the second chance does

On a hard signal the block is shown once more, with a hint that says what a good answer looks like: written by you, in your own words, about the question asked. That is all. The participant is not told which signal fired, and the hint is the same for every signal, so there is nothing to reverse-engineer. An honest participant reads it, writes two sentences, and proceeds. Automation and copy-paste produce the same pattern again, and the second failure closes the session before the study is reached. The study never sees the completion, the buyer never reverses it, and the participant's session ends with a neutral screen.

Two design choices matter here. The second chance is per participant and per session; it never becomes a block on a traffic source, a supplier or an affiliate, because a source is a mix of people and a source-level block punishes the honest ones in the mix. And the decision made at the time is never rewritten later. When the scoring rules improve, past answers are re-scored so the new rules can be calibrated on the whole cohort, but a participant who passed on the day stays passed.

What we watch

A second chance is only better than a block if it actually separates the two groups. We measure the share of second attempts that pass, the reversal rate of completions that passed on the first attempt against those that passed on the second, and the reversal rate of everything the check let through against everything it stopped, once buyers have had time to reconcile. Those numbers are reviewed on a schedule, and the rules only tighten when the numbers say the second attempt is being used to launder bad answers rather than to fix honest ones. We do not publish the internal rates; we do publish the principle, because participants deserve to know how they are judged.

The honest limit

A second chance does not stop a determined operator who reads the hint and adapts. Nothing at the answer level does; that is what device and behaviour signals, per-session limits and buyer feedback are for. What the second chance does is remove the collateral damage from the one check that touches every honest participant most directly: the one that reads their words. That is worth a second attempt.