What Happens to the People Who Drop Out of a Trial

Answer first: stopping the drug and leaving the study are two different events

FDA's draft guidance on developing weight-reduction drugs makes this distinction the first practical recommendation in its statistical section. The protocol and informed consent form, it says, should clearly differentiate between treatment discontinuation — meaning discontinuation of intervention — and study withdrawal, meaning withdrawal of consent to continued participation in study procedures, including data collection.

The guidance then takes a position on where the line should sit. The only grounds for study withdrawal, discontinuing the collection of outcome information, should be withdrawal of subject consent to continued collection of data.

That is a stronger statement than it looks. It means a person who stops taking the drug is expected to keep being measured. Their later weight is still data. It is only when someone actively withdraws consent to be measured that the trial loses them.

Two supporting recommendations follow from it. To help ensure collection of data after early treatment discontinuation, the guidance says, the patient consent form and investigator training should include material emphasizing the scientific importance of data recorded after treatment discontinuation. And the protocol should include options to ascertain key outcome information in subjects who discontinue study treatment and are unable or unwilling to continue all study visits, including returns only for the visit at which primary and key secondary endpoints are evaluated.

The technical name for the question, and the answer the guidance prefers

The word the guidance uses is estimand. Study protocols and statistical analysis plans, it says, should clearly prespecify how intercurrent events and missing data will be handled during the trial and how they will be accounted for in the statistical analyses, and sponsors should consult with FDA regarding these issues during trial design.

The guidance defines intercurrent events in a footnote: events occurring after treatment initiation that affect either the interpretation or the existence of the measurements associated with the clinical question of interest — for example discontinuation of assigned treatment, use of prohibited medications, use of alternative or additional medications, and corrective surgery. Missing data, another footnote says, include withdrawal of informed consent for collection of additional data, missed clinical visits, and loss to follow-up.

Its recommended default is stated plainly. Although FDA is open to considering alternative estimands, the guidance says, it generally recommends inclusion of analyses using the treatment policy estimand, in which all subjects, regardless of intercurrent events, continue to be measured at each prespecified clinical visit unless consent to collect such data is explicitly withdrawn, and in which all such measurements are included in the statistical analyses. For other strategies addressing specific intercurrent events, sponsors should justify that the estimand addresses a meaningful clinical question of interest and can be estimated with plausible assumptions.

The plain-language version of a treatment policy estimand is that the analysis asks what happened to people assigned to a treatment, not what happened to people who stayed on it. Those are different questions with different answers, and the second one flatters a drug that is hard to keep taking.

Some data goes missing anyway, and there are rules for that too

The guidance is realistic about this. Despite the best precautions, it says, some data will inevitably be missing, and how the statistical analyses will account for it should be prespecified in the statistical analysis plan. Missing data should be imputed in a fashion consistent with what the values, with their corresponding uncertainty, would likely have been had they been collected.

Its recommendation is multiple imputation of missing data based on data retrieved from subjects who discontinued treatment — retrieved dropout — calculated according to study treatment and study month of an intercurrent event. For continuous endpoints it says multiple imputations can be aggregated using Rubin's method; for categorical endpoints, particular imputation models should be discussed with FDA during study planning.

There is also a recommendation about how the case report forms record why someone left. To help inform the imputation process, the guidance says, medical reasons for treatment discontinuation or study withdrawal should be recorded on the case report forms and data sets — its example is recording "nausea" rather than "patient decision" or "investigator decision".

That example is the whole issue in miniature. "Patient decision" hides the reason. "Nausea" is a fact about the drug.

The test that asks how much of the result depends on assumptions

The guidance describes an analysis whose purpose is to find the breaking point of a conclusion. The use of retrieved data to impute missing data adds uncertainty, it says, because the outcomes of subjects who support continued collection of data after intercurrent events may differ from the outcomes of subjects who withdraw from the study.

To assess sensitivity to that uncertainty, it recommends tipping point analyses that vary assumptions about the missing data. These should be two-dimensional — allowing assumptions about missing outcomes on the two treatment arms to vary independently — and should include scenarios where dropouts on treatment have worse outcomes than dropouts on placebo. The goal, the guidance says, is to evaluate the plausibility of the assumed expected values for missing outcomes in each treatment arm under which the conclusions change, meaning under which there is no longer evidence of a treatment effect.

It also draws a distinction that is easy to blur. Sensitivity analyses are distinct from supplementary analyses, which target a different estimand or a different estimator of the same estimand. Supplementary analyses may provide additional insight but do not directly test the missing data assumptions of the primary analysis.

A tipping point analysis is, in effect, a statement of how much would have to be wrong about the people who left before the result stops holding. It answers a question a single headline percentage never does.

Where a reader can actually see this

None of this is invisible. Two federal reporting rules put the numbers in documents.

During a trial, 21 CFR 312.33 requires a sponsor's annual report to state, for each study, the total number of subjects initially planned, the number entered to date, the number whose participation was completed as planned, and the number who dropped out for any reason. It also requires a list of subjects who dropped out in association with any adverse experience, whether or not thought to be drug related.

After a trial, the federal results-reporting rule at 42 CFR 11.48 requires a participant flow table documenting the progress of human subjects through the trial, by arm, including the number who started and the number who completed. It separately requires, for each outcome measure, the number of participants analyzed by arm and — where that number differs from the number assigned to the arm — an analysis population description giving a brief description of the reasons for the difference.

That last requirement is the one to look for. A gap between the number assigned and the number analyzed is exactly where a dropout problem becomes visible, and the rule requires the responsible party to say in writing why the gap exists.

So the reading sequence for any trial result is: how many were assigned, how many were analyzed, and what does the record say about the difference. A percentage that arrives without those three numbers is not yet interpretable, however precise it looks.