The Files Behind the Paper: Why Supplementary Materials Now Decide Peer Review Outcomes

There was a time when the manuscript was the review and everything else was garnish. That era is over. Across disciplines, editors now assign reviewers to examine supplementary files with the same rigor applied to the article itself, and in data-intensive fields the files often carry more evidentiary weight than the prose. A scholar who spends months refining an argument and an afternoon assembling the appendix has the priorities exactly backward.

Why the Review Has Moved Into the Files

Three forces converged to elevate supplementary materials. First, the reproducibility movement — accelerated by high-profile replication failures across psychology, economics, and biomedical research — pushed journals to demand the artifacts needed to verify claims independently. Second, funders such as the National Institutes of Health and the National Science Foundation now require data management and sharing plans, which make deposit in recognized repositories a condition of the grant, not a courtesy. Third, journals discovered that transparent supplementary packages reduce post-publication disputes and retraction risk.

The practical consequence: a reviewer who cannot open your data, run your code, or trace your extended methods may treat the entire manuscript as unverifiable. In the most competitive outlets, that judgment is fatal.

What Reviewers Actually Look For

Supplementary review is not vague. Reviewers tend to check for a short list of concrete properties.

Accessibility. Files must be deposited in a recognized repository — Dryad, Zenodo, ICPSR for social science data, GitHub paired with a versioned archive for code — with a persistent identifier such as a DOI. Links to personal or institutional web pages do not satisfy most journals' policies and frequently fail within a few years.

Documentation. A dataset without a codebook is a liability. Every file should have a README explaining its contents, structure, software requirements, and the order of operations needed to reproduce results.

Integrity. Reviewers increasingly spot-check whether tables in the manuscript can be regenerated from the deposited code. Discrepancies between the paper and the supplementary record are treated as integrity concerns, not typos.

Consent and ethics. Human-subjects data must demonstrate that sharing complies with the approved IRB protocol. Depositing identifiable data without documented consent is among the most serious errors a supplementary package can contain.

The Mistakes That Trigger Desk Rejection

Editors screen submissions quickly, and certain supplementary failures end the process before review begins.

• Data described as "available upon request," which most major journals now explicitly disallow.
• Code files with no version information, dependencies, or run instructions.
• Appendices containing the actual analytical results rather than supporting material, blurring the boundary of the peer-reviewed article.
• File formats that reviewers cannot open with standard tools.
• Supplementary content uploaded after acceptance, leaving it unreviewed and unverified.

Each of these is avoidable with a day of preparation — which is precisely why their presence signals carelessness to an editor.

Building the Package as You Go

The strongest supplementary portfolios are assembled during the research, not reconstructed afterward. Establish a project folder with separate, clearly named directories for raw data, processed data, analysis scripts, and output figures from the first day of the project. Write the README as the work proceeds. Choose the repository early and confirm it issues DOIs and accepts your data types.

This habit pays dividends beyond the immediate submission. A well-documented package becomes the methodological backbone for follow-up papers, the demonstration of rigor requested in grant applications, and evidence of open-science practice that hiring and promotion committees increasingly ask to see.

A Career Asset, Not an Afterthought

Scholars sometimes regard supplementary materials as bureaucratic overhead. The better framing is reputational infrastructure. A reviewer who runs your code and reproduces your headline result becomes an advocate for the paper. A reader who extends your dataset cites your work. A committee evaluating your research integrity has concrete evidence of your standards.

Peer review has always been a judgment about trustworthiness. The difference now is that the judgment extends past the polished prose into the files behind it. Scholars who treat those files as part of the argument — documented, deposited, and verifiable — give reviewers every reason to extend that trust. Scholars who treat them as an afterthought give reviewers every reason to hesitate.