One thing that sucks about #PeerReview being so broken and a vector of domination rather that cooperation is that, in the best case, they can be skillshares as much as anything else. In some code reviews I have given and received, I have taught and learned how to do things that I or the other person wished they knew how to do, but didnt.

That literally cant happen in the traditional model of review, where reviews are strict, terse, and noninteractive. Traditional review also happens way too late, when all the projected work is done. Collaborative, open, early review literally inverts the dreaded "damn reviewers want us to do infinity more experiments" dynamic. Instead, wouldnt it be lovely if during or even before you do an experiment, having a designated person to be like "hey have you thought about doing it this way? If not i can show you how"

The adversarial system forces you into a position where you have to defend your approach as The Correct One and any change in your Genius Tier experimental design must be only to validate the basic findings of the original design. Reviewers cannot be considered as collaborators, and thus have little incentive to review with any other spirit than "gatekeeper of science."

If instead we adopted some lessons from open source and thought of some parts of reviews as "pull requests" - where fixing a bug is somewhat the responsibility of the person who thinks it should be done differently, but then they also get credit for that work in the same way that the original authors do, we could
a) share techniques and knowledge between labs in a more systematic way,
b) have better outcomes from moving beyond the sole genius model of science,
c) avoid a ton of experimental waste from either unnecessary extra experiments or improperly done original experiments,
d) build a system of reviewing that actually rewards reviewers for being collegial and cooperative

Also - how amazing is it to like... be able to TALK ABOUT WORK thats under review instead of having it be some secret because of the absolutely LUDICROUS idea of getting "scooped" which has absolutely NO PLACE in science. Again in the best case, we could imagine an "it takes a village" kind of review where we can talk about it in public, other people can wander in and out, there are some people assigned to ensure some minimal checklist of requirements but we can benefit from anyones input.

How much richer would review be from there being robust, critical and cooperative, public discussion THROUGHOUT the lifecycle of the experiment? By the time something reaches its 1.0 status of publication, it could have been seen in detail by a dozen people in the field and have their contributions creditable alongside the work. It almost makes TOO MUCH SENSE

Think about the difference between opening an issue to report a bug vs. Submitting some review on pubpeer about a critical flaw in a published paper.

One is welcome, at worst a "friend telling you about food stuck in your teeth" correction that you cant see but other people can. (Sure some people raise issues like assholes, but solving people being assholes isnt really in scope). The other is potentially career ending, always confrontational even if we want it to be constructive, the amount of times it has ended up well is a vanishingly small fraction of all criticisms like that.

Whats different? Papers are like software in that they always have some bugs somewhere, are embedded in a moving field that can shift underneath it, they make some claims on reality and either deliver or dont. They are different in that papers arent expected to be amended, often thats literally impossible. Review is "done" when a paper is published. Review happens at the "end" of a paper rather than throughout its life. You cant deprecate a paper - "this was good at the time, but we have updated our understanding." All the things that make it so someone suggesting something you could fix being an act of kindness and collaboration are forbidden by the traditional review and publishing. (A lot more subtlety but im longposting on a microblogging medium not writing an encyclopedia rn)

A recent example im thinking of is this issue I raised with @linkml that was the underlying cause of a decent number of hard to diagnose bugs:
https://github.com/linkml/linkml/issues/1839

Did that invalidate some claims made in the docs and published lit re: importing and extensibility much like errata in a paper? Yes! Was it a huge deal, fatal to the reputation of the project? Absolutely not!!! In fact a noisy bug tracker is a sign of HEALTH. there isnt a reply there because I discussed it in person with the devs later, and theres a relatively obvious fix to be made, but I meant it in cooperation, as a compliment - I care enough about your work to investigate why it isnt working - and it was received as such. Im working on a patch now. Colleagues working together to make something work!

If youve ever criticized a paper before, you know that simply is not the way it goes. I have a lot of people who I love dearly who do a lot of work I think is fatally flawed, but I wouldn't dare criticize them because it could put their career in danger. They arent bad or stupid people, but their lab does things in a particular way, etc. And so everyone loses out: the work is worse, the work is stuck in time and becomes irrelevant within discipline and misleading outside of it, the labs never improve, the working conditions never improve, we create protective silos around ourselves to survive.

People always jump all over my ass when I talk about open review to say "think of the ECRs who cant raise criticism" or "think of how the powerful labs would abuse that power" and I am always like dog I know, what im talking about is an ANTIDOTE to that problem which is made much WORSE by closed review.

Why does criticism have to be something you'll get retaliation for instead of welcomed as a contribution? Why is it possible for senior researchers to end peoples careers in private 3-party reviews instead of having to show their whole entire ass in public? Open review isnt just keeping everything the same but copy pasting the reviews on the website at the end - its an invitation to change the cultural coordinates of review itself from something broken and shitty to something with a plausible chance of being healthy.