Hacker News

I'm not sure that the system was ever that near to perfection: for example, John Maddox of Nature didn't like the advent of pre-publication peer review, but that presumably had something to do with it limiting his discretion to approve and desk-reject whatever he wanted. But in any case it (like other aspects of the cozy interwar and then wartime scientific world) could surely never have survived the huge scaling-up that had already begun in the post-war era and created the pressure to switch to pre-publication peer reivew in the first place.

Peer review existed before 1951 in the US at least. See for example Einstein’s reaction to negative reviews when he tried to publish in Physical Review in 1935 https://paeditorial.co.uk/post/albert-einstein-what-did-he-t...

> coincidentally founded by Ghislaine Maxwell's father

A crazy world we live in where Robert Maxwell's daughter is more notorious than he is.

>Pergamon Press in 1951 (coincidentally founded by Ghislaine Maxwell's father)

perhaps a bit off-topic, but what is coincidental about this and/or what is the relevance of Ghislaine Maxwell here?

>scientists focused on quality not quantity.

I know a PhD professor doing post doc or something, and he accepted a scientific study just because it was published in Nature.

He didn't look at methodology or data.

From that point forward, I have never really respected Academia. They seem like bottom floor scientists who never truly understood the scientific method.

It helped that a year later Ivys had their cheating scandals, fake data, and academia wide replication crisis.

Some "fun" reading on the subject of Mr. Maxwell:

https://sarahkendzior.substack.com/p/red-lines

tl;dr He is the bridge that uncomfortably links Biden's former Secretary of State, Antony Blinken, to Jeffrey Epstein and Mossad. Hence, *gestures at the last couple of weeks and years*. Dude was just, like, Fraud Central, apparently.

> Do you want issues of Nature and cell to be replication studies?

Of course I do! Not all of course, and taking (subjectively measured) impact into account. "We tried to replicate the study published in the same journal 3 years ago using a larger sample size and failed to achieve similar results..." OR "after successfully replicating the study we can confirm the therapeutic mechanism proposed by X actually works" - these are extremely important results that are takin into account in meta studies and e.g. form the base of policies worldwide.

> Do you want issues of Nature and cell to be replication studies?

More than anything. That might legitimately be enough to save science on its own.

I know you got a ton of responses already but not caring about replicability just invalidates science as a method. If we care only about first to publish we end up in the current situation where we don't even know that we know is actually even remotely correct.

All because journals prefer novelty over confirmation. It's like a castle of cards, looks cool but not stable or long-term at all.

> Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies.

Actually, yes, I do. The marginal cost for publishing a study online at this point is essentially nil.

If you're a reader within the field, then you are the one person in the world who should be most interested in negative replication studies.

Even if that negative study could save you one, two, three+ years of work for the same outcome (which you then also can't really do anything with)? Shouldn't there BE funding for replication studies? Shouldn't that count towards tenure? Part of the problem is that publications play such a heavy role in getting tenure in the first place.

I'm sure you can more narrowly tune your email alerts FFS.

"Original research" isn't worth much unless replicated, which is the entire problem being discussed in this thread. Replicating studies are great though because they tell you if the original research actually stands and is valid.

> Replicating work is far more difficult than a lot of original work.

Only if the original work was BS. And what, just because it's harder, we shouldn't do it?

> Do you want issues of Nature and cell to be replication studies?

Hell yeah. We’re all trying to get that Nature paper. Imagine if you could accomplish that by setting the record straight.

Realistically, everyone will say “yes” to the “do you want” question because if you’re not a reader or a subscriber you benefit from the readers reading replication studies.

I believe people will enthusiastically say yes but that they do not routinely read that journal.

>Also who's funding you for replication work? Do you know the pressure you have in tenure track to have a consistent thesis on what you work on?

This is partly why much of today's science is bs, pure and simple.

> Replicating work is far more difficult than a lot of original work.

I don’t regularly read scientific studies but I’ve read a few of them.

How is it possible that a serious study is harder to replicate than it is to do originally. Are papers no longer including their process? Are we at the point where they are just saying “trust me bro” for how they achieved their results?

> Do you want issues of Nature and cell to be replication studies?

Not issues of Nature but I’ve long thought that universities or the government should fund a department of “I don’t believe you” entirely focused on reproducing scientific results and seeing if they are real

Economics has the Journal of Comments and Replications in Economics: https://jcr-econ.org/

We already have archival journals. What's missing is funding and any prospect of career advancement.

And funding dedicated to replication studies.

Is there a viable career path for researchers who choose to focus on replication instead of novel discoveries? I assume replications are perceived as less prestigious, but it's also important work.

Of course this is slightly messy too. Fraudsters are probably always incorrect, of course they could have stolen the data. But being incorrect doesn't mean your intentionally committing fraud.

That would be great if journals bothered publishing replication studies. But since they don't, researchers can't get adequate funding to perform them, and since they can't perform them, they don't exist.

We can't look for failed replication experiments if none exist.

that approach is accurate, but not scalable.

the effort to publish a fraudulent study is less (sometimes much less) than the effort to replicate a study.

Yeah, but this happens all the time.

>>95% of the time, the fraudsters get off scot-free. Look at Dan Ariely: Caught red-handed faking data in Excel using the stupidest approach imaginable, and outed as a sex pest in the Epstein files. Duke is still giving him their full backing.

It’s easy to find fraud, but what’s the point if our institutions have rotten all the way through and don’t care, even when there’s a smoking gun?

Is it that easy?

Machine Learning papers, for example, used to have a terrible reputation for being inconsistent and impossible to replicate.

That didn't make them (all) fraudulent, because that requires intent to deceive.

Once you move from abstract to practical - like say having legislators or regulators make rules based on The Science, or relying personally on more facts than you have time to independently verify - yes you do need to have trustworthy people.

Are you going to build a competitor to CERN?

There are many things that cannot be feasibly verified empirically without access to rare resources.

Can ordinary civilians feasibly measure, for example, global trends in mean temperature without relying on the data of others?

>It's career-ending..

That is where being good at politics come into play. And if you are good at it, instead of being career-ending, fraud will put you in the highest of the positions!

No one wants a "plant" who cannot navigate scrutiny!

> The good news is that it does not match any of the years of experience in my field.

I worked for exactly one academic, and he indulged in impossible-to-detect research fraud. So in my own limited experience research fraud was 100%.

It was a biology lab, and this was an extremely hard working man. 18 hours per day in the lab was the norm. But the data wasn't coming out the way he wanted, and his career was at stake, so he put his thumb on the scale in various ways to get the data he needed. E.g. he didn't like one neural recording, so he repeated it until he got what he wanted and ignored the others. You would have to be right in the middle of the experiment to notice anything, and he just waved me off when I did.

This same professor was the loudest voice in the department when it came to critiquing experimental designs and championing rigor. I knew what he did was wrong, because he taught me that. And he really appeared to mean it, but when push came to shove, he fiddled, and was probably even lying to himself.

So I came away feeling that academic fraud is probably rampant, because the incentives all align that way. Anyone with the extraordinary integrity to resist was generally self-curated out of the job.

Undoubtably climate science is the exception and immune from fraudulent data generation and sometimes complete forgery

It's the tax-payer funded business model, the NGO trap. Subsidies, grants, tax-breaks, credit, deductions, exemptions, etc. A whole class of profiteers live in this sector. Even though academia funding isn't strictly categorized as an NGO, it still fits/foots the bill. Public funding of private gains is the oldest trick in the book. Ask any capitalist, they know. And I'm not saying I'm against public funding, but this is often codified into a mafia of sorts when enough money flows through.

The real problem here is the fundamental lack of democratic control over our agencies. That our political organization is intensely lagging behind our productive organization. That our whole political will involves TRUSTING strangers to not be corrupt instead of directly democratizing these processes as much as possible.

But besides that, you cannot remove history from historical analysis. The reason socialism countries struggled in the beginning wasn't an inherent flaw in its organization, but the fact that they were under constant war war by capitalist countries through out their existence. Also keep in mind that most socialist countries did NOT have a whole section of the world where-from to extract riches through murder (S.America, Africa, Middle east, etc), like western capitalist countries had. This is convenient for you to ignore. Maybe because you don't know, or don't care about the super-exploitative history of these places and how they tie into western capitalism. But they are inherent to western wealth and these countries' whole history is struggle against this exploitation.

Not to mention that most of the countries on earth are capitalists and are very very very poor.

To add: Socialism has nothing to do with "clamping down" on X or Y industry, as you hypothetically claim would happen. Socialism is almost exclusively about removing the need to generate capital from production. It unleashes production from its historical ball and chain that is profiteering.

In a single sentence: Instead of production being held back by capitalists generating wealth we can produce for our own needs. It is self sustaining production.

Central planning is not fallacious. Your problem is with corruption, not democratic central planning. The US Govt is a pro-capitalist entity that pro-capitalists try to distance themselves from (ironically). So using them as an example isn't saying anything at all.

Central planning is not "allow a small group of people to decide things", as happens in the US Govt. Central planning is to take into account all sources of information on production to plan said production democratically.

This will always beat the highly highly inefficient speculation of capitalism. Where trillions vanish on a whim and cause of a tweet, where crisis occur every 8-10 years, and where its whole trade market is built to hide that it is mostly insider trading. Again, your problem is with corruption not democratic central planning.

And the way to deal with corruption is to create more democratic bodies where avg people hold real power. I don't see you asking for that either. We call that socialism.

Except with AI, a fraudulent gatekept world would still be a smaller percentage of fraud than what is coming. Infinite scale fraud.

The soviets may have rigged a few studies; but the democratized world now faces almost all studies being rigged.

There is a saying in my native language that goes something like "If you mix poison and milk, the milk will turn poisonous, instead of poison becoming milk (aka beneficial)".

I guess, to convert it into this context, we can say that if you mix the high minded and infantile (which I think is what Internet and social media did), the high minded becomes infantile, instead of the other way around.

there is no 'sin of maximal inclusivity here', the gate is broken, but primarily because it was largely an honor system before, and no one has the motivation or resources to really dig into a lot of these papers.

in no sense was it corrupted by the desire to include a larger population in journal publications.