2-5%

Manuscript under review: “The Hacker News Front Page, 27 July 2026.” Approximately thirty co-authors. None of them appear to have met.

Summary of submission

The authors submit thirty findings under a single cover and ask that the collection be read as one body of work. I have read it that way. I am aware this is not how the material was intended to be taken, and I would gently suggest that this is the problem.

Read as one paper, the manuscript is methodologically incoherent. Two sections contain a control group. One contains an apology for its own graphs. Several contain no observable quantity of any kind, and in at least one case the entire contribution is a claim about a period of time that has not happened yet. The authors do not appear to notice that these are different genres of writing. They have been set side by side at identical font size, which I take to be an editorial decision rather than an accident.

My concerns follow.

Methods (sections 3 and 11)

Section 3 is the strongest work in the submission and I want that on the record before I say anything else.

The author has sixteen solar panels, wired to the inverter as two banks of eight. He washed one bank. He then reported the ratio of power output between the two banks across the day, which cancels out cloud cover, sun angle, panel temperature and every other thing that would otherwise eat the effect he was hunting. That is a control group, built in a garden, out of hardware he already owned, by a man with a hose. His finding is that washing solar panels is worth about 2-5%, or £60-£150 a year, decaying to zero over a few years.

That is a humiliating number to publish. It is precisely the number that makes the whole afternoon look not worth having had. He published it anyway, and then went further: when the power ratio dropped after the first panel was washed, he wrote “I’m unsure why” and left it in. He did not resolve it, and he did not quietly drop the interval that embarrassed him.

Section 11 shares the instinct. Its author wanted to know how a heavily publicised machine-driven rewrite was actually going, so he counted. On 9 July there were 1,277 open pull requests from the bot doing the rewriting. On 27 July there were 2,475. Merging to main with the full check suite runs about forty minutes, sometimes up to an hour and a half, from which he gets 86 days of continuous pipeline to clear the current backlog. The last release tag is dated 12 May. The announced cost was $165,000 of API calls across eleven days in May, and at a comparable burn he puts the running total near $800,000.

Then, in the discussion appended to his own section, he volunteers that his graphs count Rust files touched per commit rather than commits, which inflates them, and that he has not gone back to fix it because he judges that the number would serve his point either way.

I want to note what happened there. The author degraded his own evidence, in public, unprompted, at no benefit to himself, in the middle of an argument he was winning. Nothing in this manuscript compelled him to do it. He did it because that is what you do.

Sections 3 and 11 are the same paper. One is about dust and one is about pull requests and neither of them is really about that. Both are: here is how I got the number, here is where the number is soft, here is the number. Add the author of section 8, who dates every step of an unanswered vulnerability disclosure down to the day the internal APIs quietly closed and nobody ever wrote back, and there are three people on this page who understand that a finding is a method plus a result, and that the method is the load-bearing half.

Statistical treatment

Everywhere else, the manuscript’s discussion discards the manuscript’s own measurements.

Section 14 reports that removing a frontend framework took a JavaScript bundle from 615kb to 530kb. That is a result. It has a before, an after and a unit. The hundred-odd comments appended to section 14 do not engage with it. They consist almost entirely of professional biography. The phrase “I have been writing frontends since early 2000” appears repeatedly, from the same commenter, deployed as an argument. The section offered a results table and the discussion answered with curricula vitae.

Section 6 is worse, and worse in an instructive way. Section 6 is a status page: a vendor reporting elevated error rates on its own model. Fifty-odd comments follow, in which working practitioners characterise that model’s reliability. One reports that it could not complete a single one of about five tasks without either getting it partly wrong or introducing a regression, credits it for catching and fixing those regressions competently, and says he forcibly downgraded to the previous version. Another finds it “more reliable than even myself at times,” and then adds, in the same comment and without apparent alarm, that “this huge error bar in delivered quality is one of the biggest issues with LLMs.” He has named the problem he is standing inside and does not appear to notice that he is standing inside it.

Sit with the width of that error bar. The reported outcomes for one product, among people who use it daily and professionally and for money, span from better than me to zero clean runs out of about five. Nobody ran a test. Nobody held anything constant. Nobody washed eight panels and left eight dirty. The aggregate measurement available to a reader after fifty comments is worse than no measurement, because it has the shape of data.

The one section where the reviewers behaved like scientists is section 22, on the pollination of magnolias by beetles. Its commenters correct each other constantly, cheerfully, and on substance: that all mosquitoes drink nectar and only the females of some species drink blood, that seeds are not gametes. Every correction is checkable and every one gets made. Section 22 is also the only section on the page where no participant stood to gain a dollar from the outcome. I have marked it as the control.

Major concern: the instrument and the lobby

Section 20 announces a large open-weights model release. The first substantive response is not about the model. It is about the fact that once third-party providers begin serving it, the settled market price will reveal roughly what it costs to serve something of that size, which finally yields an independent figure against which to check whether the closed labs are subsidising their token pricing. The commenter’s phrase is “clear of conflict datapoint.” Handed a frontier model, the community’s first instinct was to pick it up and use it as a ruler.

Section 9 reports that federal disclosures for the first half of 2026 show one AI company nearly doubling its lobbying spend to $2.22m and another nearly tripling its own to $3.53m. The most-upvoted reaction is that these sums are astonishingly small. Pocket change, at this scale.

The authors file these as unrelated findings. They are one finding, and it is the one the manuscript most needs to state plainly and never does.

A ruler has appeared. It cost nothing and anyone may pick it up. Simultaneously, the parties who would be measured by it are spending low single-digit millions, which they correctly regard as trivial, on the question of who is permitted to hold rulers and what they may be pointed at. I would reject a draft that called this hypocrisy. It is a rational allocation of capital, and it follows directly from the asymmetry the rest of the manuscript demonstrates: 2-5% can be wrong, and “transformative” cannot. A measurement is a commitment. An announcement is a position that never has to settle. If your product’s value rests on a claim about a future quarter, precision is not an asset you have failed to acquire. It is a liability you are correctly declining to take on.

The man with the hose has more skin in the game than anyone in section 9. He published a figure that made his own afternoon look silly, and it cost him the afternoon. The parties in section 9 have three and a half million dollars riding on never doing that, and at that price they are getting a bargain.

Limitations, declared and undeclared

Section 21 is a three-dimensional visualisation of a database engine rendered as a city, and it is a delight. Its authors describe it, in their own words, as an early and unreviewed prototype that likely contains errors, and they ask to be sent corrections. The author goes further, volunteering unprompted that the thing began as a weekend curiosity about what a model could do, and posting the original prompt he used; another commenter independently confirms the provenance from the commit trailer. All of this is in the open. The reader knows exactly how much weight the artifact will bear, because the artifact says so.

Section 4 proposes a general theory of software bugs and declines to define its terms. The sharpest objection in its thread is that the author never actually defines anything, which makes it impossible to separate the empirical science from the data visualisation. A second commenter goes further and cleaner: the theory concerns programs found by search that look correct on small inputs, not programs written by someone who reasoned about the problem, and those are two different situations sharing one word.

These sections are the entire argument in miniature and they sit seventeen apart. One volunteers its own error bars unasked and earns trust in proportion. The other refuses to specify what it is claiming, is therefore unfalsifiable, and is therefore worth nothing to me as a referee however pleasant it is to read. The difference is not intelligence. Both are the work of clever people. The difference is willingness to say a thing that could turn out to be false.

Recommendation

Reject.

The manuscript cannot be published as a unified body of work, because it is not one. What it is: three or four people who went and checked something, bound together with a much larger volume of writing about a future that has not been asked to arrive on any particular date.

I would urge the editors to extract sections 3, 8, 11 and 21 and publish them exactly as they stand. They require no revision. Their authors have already done the only thing I would have asked of them, which is to state a number, state how they got it, and state where it is weak.

To the remaining authors: I am not asking for more rigour. Rigour is expensive and most of you do not need it. I am asking for one sentence. Tell me what would have to happen for you to be wrong. If you cannot write that sentence about your own work then you have not produced a finding, you have produced a mood, and I have fifty comments’ worth of moods already and they average to nothing.

Section 3’s author washed eight panels, left eight dirty, and got 2-5%. It is the highest-quality claim in the submission. It is also the smallest. I do not believe that is a coincidence, and I would like the revised manuscript to account for why not.