Research · 6 min read
What a Trial Has to Publish When It Reports Results
The federal rule does not ask for a summary. It specifies tables — who started, who finished, what was measured, every serious adverse event, every other event above five percent — plus the protocol and the statistical analysis plan.
Key takeaways
- 42 CFR 11.48 specifies required tables — participant flow, baseline characteristics, outcomes with statistical analyses, and adverse events — not a free-text summary.
- Wherever the number of participants analyzed differs from the number assigned to an arm, the rule requires a written description of the reasons.
- Every outcome measure must be labeled primary, secondary, other pre-specified, or post hoc, and the protocol and statistical analysis plan must be filed alongside.
- Required statistical reporting is either a p-value with the procedure used, or an estimation parameter, estimated value and confidence interval if calculated.
- Three safety tables are required: all serious adverse events, non-serious events exceeding 5 percent frequency in any arm, and all-cause mortality — each with numbers affected and at risk by arm.
- The record must disclose any agreement restricting the principal investigator's ability to discuss or publish the results after the primary completion date.
Answer first: results reporting is a list of required tables, not a narrative
The federal rule at 42 CFR 11.48, headed "What constitutes clinical trial results information?", sets out what a responsible party must provide. It is organized as six groups of required elements, and each of the first four is described as information for completing a table.
Those four are participant flow, demographic and baseline characteristics, outcomes and statistical analyses, and adverse event information. The remaining two are a copy of the protocol and the statistical analysis plan, and administrative information.
This structure is the reason a registry record can be compared against a company's own description of its trial. The company writes the press release. The rule specifies the tables. When those two disagree, the disagreement is visible without any specialist knowledge.
Which trials this applies to is set by 42 CFR 11.42, which requires results information for applicable clinical trials whose studied product is approved, licensed or cleared by FDA and also for those whose product is not — for trials with a primary completion date on or after January 18, 2017, the required contents are those specified in 11.48.
Who started, who finished, and who was actually analyzed
The participant flow requirement asks for information documenting the progress of human subjects through the trial, by arm, including the number who started and completed. It also requires pre-assignment information: a description of significant events occurring after enrollment and before assignment of subjects to an arm.
The baseline table must cover demographic and baseline measures collected by arm or comparison group and for the entire population who participated. The rule names the measures that must be described: age, sex or gender, race, ethnicity if collected under the protocol, and any other measure assessed at baseline that is used in the analysis of the primary outcome.
Then comes the requirement that does the most work for a careful reader. For the baseline table, if the overall number of baseline participants differs from the number assigned to the arm and overall, the responsible party must give an analysis population description — a brief description of the reasons for the difference. The same requirement repeats for outcomes: the number of participants analyzed must be reported by arm, and if it differs from the number assigned, the reasons must be described.
In other words the rule anticipates that these numbers will not match, and requires the mismatch to be explained rather than smoothed over.
Outcomes have to be labeled by type, including post hoc
For each outcome measure, the rule requires a description that includes the name of the measure, the metric used to characterize it, the time points at which it was assessed, and the unit of measure. It also requires an outcome measure type: whether the outcome is primary, secondary, other pre-specified, or post hoc.
That fourth category is the interesting one. A post hoc outcome is one identified after the data existed. The rule does not forbid reporting one; it requires the record to say what it is. A page quoting a striking finding from a trial can therefore be checked against the record's own label for that finding.
The statistical analysis requirements are equally specific. A statistical analysis must be submitted if it was pre-specified in the protocol or statistical analysis plan and performed on the outcome data; or was made public by the sponsor or responsible party before results information was submitted for the primary outcomes; or was conducted on a primary outcome in response to a request made by FDA before that submission.
For each such analysis the record must identify the arms compared and the type of statistical test conducted, and — for a non-inferiority or equivalence test — describe the analysis including at minimum the power calculation and the non-inferiority or equivalence margin. It must then give one of two things: a statistical test of hypothesis, meaning the p-value and the procedure used; or a method of estimation, meaning the estimation parameter, the estimated value, and the confidence interval if calculated.
So both quantities a reader might want are contemplated by the rule. Which one appears depends on what analysis was done.
The adverse event tables, and the five percent line
The rule requires information describing how adverse events were collected: the specific time frame over which the information was collected, a brief description of how the trial's definitions differed from the rule's own if they did, and the type of collection approach — whether systematic or non-systematic.
That last element matters more than its length suggests. A systematically collected event was asked about. A non-systematically collected one was volunteered. Two trials of the same drug can produce different-looking safety tables for that reason alone, and the rule requires each record to say which it used.
Then it requires three tables. A table of all serious adverse events grouped by organ system, with the number and frequency of each event by arm or comparison group. A table of all adverse events other than serious ones that exceed a frequency of 5 percent within any arm, again grouped by organ system with number and frequency by arm. And a table of all-cause mortality, with the number and frequency of deaths due to any cause by arm.
Each table must also carry a total number affected and a total number at risk, by arm, so a frequency can be checked against its own denominator. The five percent threshold is the reason a label or a registry record will not list every discomfort anyone reported: below that line, a non-serious event does not have to appear in this table at all.
The protocol, the analysis plan, and one disclosure about the investigator
The rule requires a copy of the protocol and the statistical analysis plan, if the plan is not included in the protocol, together with all amendments approved by a human subjects protection review board before submission that apply to all trial facility locations. Each document must carry the official title, the registry number if available, and the document's date on its cover page. Names, addresses and other personally identifiable information may be redacted, as may trade secret and confidential commercial information as those terms are defined in the Freedom of Information Act and the Trade Secrets Act.
Publishing the plan alongside the results is what makes the pre-specified and post hoc labels checkable rather than assertions.
The administrative group contains one element that has nothing to do with statistics and is worth knowing about. The record must indicate whether the principal investigator is an employee of the sponsor and, if not, whether there exists any agreement between the sponsor or its agent and the principal investigator — other than one solely to comply with privacy law — that restricts in any manner the investigator's ability, after the primary completion date, to discuss the results at a scientific meeting or any other public or private forum, or to publish information concerning the results in a scientific or academic journal.
The statute behind all of this sits at 42 U.S.C. 282(j). Its registration requirements are separate from results and are also itemised: descriptive information including a brief title and summary intended for the lay public, the primary purpose, the study design, the study phase for a drug trial, the primary disease or condition, the intervention, the study start date, the expected completion date, the target number of subjects, and outcomes including primary and secondary outcome measures — plus recruitment information including eligibility criteria, gender, age limits, whether healthy volunteers are accepted, and overall recruitment status.
Read together, the statute and the rule mean that for a covered trial there should exist a public document stating what the trial set out to measure before it ran, and a second stating what it found. Comparing those two is the check that the whole scheme exists to make possible.
Sources
- 42 CFR 11.48 — What constitutes clinical trial results information?The six groups of required results elements; the participant flow and pre-assignment requirements; the baseline measures named including age, sex or gender, race and ethnicity if collected; the analysis population description requirement wherever the number analyzed differs from the number assigned; the outcome measure description elements and the four outcome measure types including post hoc; the three circumstances requiring a statistical analysis to be submitted; the statistical analysis overview elements including the non-inferiority or equivalence margin and power calculation; the choice between a statistical test of hypothesis with p-value and a method of estimation with confidence interval; the adverse event collection elements including time frame and systematic or non-systematic approach; the three required adverse event tables and the 5 percent threshold; the total number affected and total number at risk elements; the requirement to file the protocol and statistical analysis plan with permitted redactions; and the required disclosure of any agreement restricting the principal investigator's ability to discuss or publish results.
- 42 CFR 11.42 — For which applicable clinical trials must clinical trial results information be submitted?That results information is required both for applicable clinical trials whose studied product is approved, licensed or cleared by FDA and for those whose product is not, and that for trials with a primary completion date on or after January 18, 2017 the required contents are those specified in § 11.48.
- 42 U.S.C. 282 — Director of National Institutes of Health, subsection (j), Expanded clinical trial registry data bankThe itemised registration content required by the statute, including a brief title and brief summary intended for the lay public, primary purpose, study design, study phase for a drug trial, study type, primary disease or condition, intervention name and type, study start date, expected completion date, target number of subjects, outcomes including primary and secondary outcome measures, and recruitment information including eligibility criteria, gender, age limits, whether healthy volunteers are accepted and overall recruitment status.
Frequently asked questions
What has to be reported when a covered trial reports results?
42 CFR 11.48 requires participant flow, demographic and baseline characteristics, outcomes and statistical analyses, adverse event information, a copy of the protocol and statistical analysis plan, and administrative information. The first four are specified as tables with named elements.
Does the record have to say whether a finding was planned in advance?
Yes. Each outcome measure must carry an outcome measure type identifying it as primary, secondary, other pre-specified, or post hoc. The protocol and statistical analysis plan must also be filed, which is what makes those labels verifiable.
Are p-values or confidence intervals required?
One or the other, depending on the analysis. For each required statistical analysis the record must give either a statistical test of hypothesis — the p-value and the procedure used — or a method of estimation, meaning the estimation parameter, the estimated value and the confidence interval if calculated.
Does every side effect have to be listed?
No. The rule requires a table of all serious adverse events, a table of non-serious adverse events that exceed a frequency of 5 percent within any arm, and a table of all-cause mortality. Non-serious events below that five percent line do not have to appear in that table.
Why does it matter whether adverse events were collected systematically?
Because a systematically collected event was asked about and a non-systematically collected one was volunteered, which can make two trials of the same drug produce differently shaped safety tables. The rule requires the record to state which collection approach was used.
Does the record say anything about the investigator's freedom to publish?
Yes. The administrative information must indicate whether the principal investigator is an employee of the sponsor and, if not, whether any agreement exists that restricts the investigator's ability, after the primary completion date, to discuss the results publicly or to publish them in a scientific or academic journal.