Why are we still publishing static PDFs? On peer-review and what Science could have been

science
thoughts
Published

November 21, 2026

Modified

November 21, 2026

On peer-review

Peer-review is often espoused as one of the most important parts of science. We don’t trust works that haven’t gone through peer-review, and sometimes we trust them too much just based on the fact that it has passed peer-review.

While I’m still far from being a graybeard scientist, I’ve both gone through the peer-review process and been a peer-reviewer a number of times over the years.

In my opinion, the current peer-review process has a number of problems that often go unnoticed, and compound on one another.

Because peer-reviews are typically done by time-starved researchers, the quality of the peer-review is often an issue. I’ve encountered many reviewers who ask only surface-level questions that hardly concern the technical content of the work, or are entirely LLM-generated. Worse, sometimes a single superficial review is enough to get the paper published.

To solve the reviewer shortage, journals often dangles incentives that target early-career researchers. This can be in the form of giving out certificates or giving out reviewer of the year awards. But this incentivize early-career researchers to accept peer-review requests even when they are unfamiliar with the topic, or even when they lack the time for it. This reduces the quality of the peer-review, or pushes them to delegate it to an LLM.

Meanwhile, the “publish or perish” pressure in academia turns authors into “yes-men” to the peer-reviewer’s comments/arguments/suggestions just to get their papers accepted, further eroding the integrity of the peer-review process.

I’ve also encountered reviewers who use peer-review as an opportunity to get their papers cited. For example, telling the authors that they did not properly frame their work with known results in the literature, and then providing said literature that sneakily is the peer-reviewer’s own works. This introduces a conflict of interest where the reviewer now benefits from the paper getting published.

These are just a few examples and there are of course various other problems, e.g., the editor’s and reviewer’s bias towards big names or prestigious affiliations, or rushed reviews because the journal advertised how fast one can get their papers published.

Science needs to evolve

But what’s the alternative?

I often hear people say that

“Yes, peer-review is imperfect, but it’s the best that we have.”

But is this true? Is this really the “best that we have”?

One of the most important technological advancements in the 21st century is the internet, which allows us to collaborate and share ideas to one another all around the world almost instantaneously.

How has scientific papers and publishing utilized this technology?

Well, a paper can easily be submitted to a journal from anywhere on Earth, which in turn can easily be sent to peer-reviewers on the other side of the Earth, and can easily be downloaded anywhere the moment it is published.

In my opinion, this is not good enough. Science should have evolved with the advent of the internet.

Imagine that when you submit a paper for “publication”:

  • All the digital contents (codes, data, etc.) are uploaded into a single place that is publicly available.
  • Anyone can inspect these contents.
  • Anyone can submit issues/comments that are visible to the public, that the authors can engage and discuss on.
  • Anyone can contribute by submitting changes that are visible to the public, that the authors can consider to accept to the “published” work or not.
  • The full history of what was changed and how the paper evolved is recorded.

There’s full transparency on the scientific work, issues/reviews, suggested changes etc., and at the same time, this piece of scientific work is not static, it is evolving as long as people continue to contribute to it1.

1 Importantly, authors still control what is accepted, and community upvoting/reputation could act as a filter for low-quality contributions.

How can we achieve this?

In fact, this has already been achieved for software. I’m of course referring to public repositories (i.e., Github, Codeberg, GitLab etc.) that rely on the most popular collaborative tool invented in the 21st century: Git2.

2 I know of research groups that already are using Git internally to collaborate on papers.

3 Which is how it already works anyway, where anyone who made contributions to the paper should be acknowledged as a co-author.

This of course changes the very nature of a “published” scientific work. A scientific work can continue to evolve over time, and anyone who contributed changes are listed as a contributor3.

Your “worth” as a scientist is then not just the number of publications or citations you have, but also the amount of open-source science you contributed to, similar to how it is for software engineers.

This is a huge change to the concept of a scientific paper and what it means to be a scientist, and thus probably would never happen.

But by contrast, this is what we currently have:

  • You submit (and pay the journal) to publish a paper, wait a couple of months for a few comments from peer-reviewers that LLMs probably could have also come up with.
  • Once the work passed peer-review it is (typically) put behind a paywall, remains static, and can become outdated for future readers.
  • There’s no way to comment on or discuss the work at the same place that the work is published.
  • Someone who spots a mistake keeps it to themselves as there’s no way to raise these issues to everyone who access the work.
  • Issues emailed to the authors are kept hidden from everyone else, and are subject to the authors agreeing and finding time to send an update to the journal.

These, in my opinion, are hindering scientific progress. Just look at the amount of bug fixes that are patched in open-source software. I wish we had that for scientific papers4.

4 It is also worth noting that open-source AI research works are already skipping the peer-review process by “publishing” to arXiv along with a public repository, since the rapid development in the field of AI means that it’s probably outdated by the time the peer-review process completes.

Did we missed the window?

I used the phrase “could have been” in the title as I think that perhaps we have already missed the window for this revolution.

Agentic AIs are rapidly beginning to take over scientific works, and a collaborative Git-style workflow built might never get the chance to take root, or would not amount to much anyway.

Yet, at the same time, perhaps this way of doing science could enable AI to easily contribute and collaborate with one another, just as it has done for software engineering, making AI-driven science trustworthy, more systematic, and more auditable by both humans and AIs alike.

Either way, keeping science locked behind paywalls in static PDFs that are given the “stamp of approval” by a few peer-reviewers, is a relic we would all benefit from retiring.

Back to top