How to separate creative genius from creative mistakes? Not with peer-review. It is a consensus filter.
Classical peer review is a form of scientific gatekeeping (it’s good to see that term recognized in official literature). Unpaid anonymous peer review is useful at filtering out some low quality papers, it is also effective at blocking the controversial ones which later go on to be accepted elsewhere and become cited many times, the paradigm changers.
And the more controversial the topic, presumably, the worse the bias is. What chance would anyone have of getting published if, hypothetically, they found a consequential mathematical error underlying the theory of man-made global warming? Which editors would be brave enough to even send it out for review and risk being called a “denier”? Humans are gregarious social beings, and being in with the herd affects your financial rewards, as well as your social standing. Even high ranking science journal editors are afraid of being called names.
Mark Peplow discusses a new PNAS paper in Nature:
Using subsequent citations as a proxy for quality, the team found that the journals were good at weeding out dross and publishing solid research. But they failed — quite spectacularly — to pick up the papers that went to on to garner the most citations.
“The shocking thing to me was that the top 14 papers had all been rejected, one of them twice,” says Kyle Siler, a sociologist at the University of Toronto in Canada, who led the study1. The work was published on 22 December in the Proceedings of the National Academy of Sciences.
There is no formalized sure-fire system to find and reward the creative genius needed for the big leaps in science. Their work must be impeccable logical, but it is an art to cut through human biases to recognise that genius. And art cannot be mandated or controlled. We should never place much confidence in a formalized process, especially one that’s unpaid and anonymous, to spot the papers that will be the most cited 50 years from now.
But the team also found that 772 of the manuscripts were ‘desk rejected’ by at least one of the journals — meaning they were not even sent out for peer review — and that 12 out of the 15 most-cited papers suffered this fate. “This raises the question: are they scared of unconventional research?” says Siler. Given the time and resources involved in peer review, he suggests, top journals that accept just a small percentage of the papers they receive can afford to be risk averse.
For the record:
Siler and his team tapped into a database of manuscripts and reviewer reports held by the University of California, San Francisco, that had been used in previous studies of the peer-review process.
Anyone who thinks “peer review” is somehow part of the scientific method does not know what science is.
h/t to the brilliant Matthew.
REFERENCES
Peplow, Mark (2014) Peer review — reviewed, Top medical journals filter out poor papers but often reject future citation champions. Nature,doi:10.1038/nature.2014.16629 [Discussion of Siler et al]
Siler, K., Lee, K. & Bero, L. Proc. Natl Acad. Sci. USA http://dx.doi.org/10.1073/pnas.1418218112 (2014).
So initially the peer review process was to assist in editing out any poorly researched scientific papers, a type of quality control?, if so when did peer review develop (according to some) into a necessary part of the scientific method?
I’m guessing just after the hyping up of the CAGW hypothesis but I would be interested in earlier examples.
102
It is often overlooked that a main purpose of peer review is to prevent the conversation being clogged by repetition.
Practitioners in an area have a pretty good idea of what has been said before, so a couple of them review a new paper and if it is not original it is not published. Beyond that, yes some basic quality control is applied, looking for obvious errors. But no, they don’t check real hard, and they aren’t expected to. For instance reviewers almost never run the software involved (although I hazily recall that in some medical areas this is now a requirement) or directly inspect the data. Non-originality is a stronger barrier to being published than quality.
290
Unless there is a ‘consensus’. Thus is the true scientific method lacking…in my opinion as a layman.
121
You mean papers that duplicate or falsify the result of other paper- those are not published as they are not original. Hilariously that they think its any good at weeding out the dross given the Lancet has published absolutely faulty papers and many other journals.
80
Another overlooked element in the peer review process is that there is no standard for evaluation of papers. Every editor is free to set his own or leave it up to reviewers. Evidence of this is that such a large percentage of innovative papers were rejected in one place but accepted in another. Peer review could be rehabilitated if the scientific societies were to establish standards and rubrics for the journals to follow. Publication of results in an online repository would be enlightening for authors, other researchers, and the public. Anonymity of reviewers still could be protected while the sunshine of open access would inhibit the raw politics that corrupt the process now. This isn’t a radical idea; evaluation rubrics are used commonly in education to assess performance. Why not apply it to a most important problem sorely in need of improvement?
51
Anonimity of the reviewers is a big part of the problem IMHO.
Why should the reviewers have Anonimity?
31
I can see a point in anonymity before and during the peer review process, but after the paper is published, I can see no reason why these names should not be published with the paper.
This might brings some basic honesty back into the system.
70
The anonymity is for the protection of the reviewer, to encourage them to be objective without facing repercussions. Academia is a snakepit of festering grudges and faction fighting – more scheming liars than dreaming spires.
40
It sounds like what the science community needs to create, is an equivalent to a patent office.
A single office with paid mathematicians and wordsmiths to check the contents of the paper is valid (and like the patent office, not validating if it’s true, let the citation count handle that).
30
Non-originality is a stronger barrier to being published than quality.
This comment is very true. Peer reviewed journals rely on people – or rather universities – subscribing to them. If there are few original articles that get quoted elsewhere, then there will be few subscribers. That is unless the subject like the pop charts – hit after hit, with no originality. Then the University can archive editions older than 2-3 years.
30
I’m sorry. How does the counter-claim ” … a main purpose of peer review is to prevent… repetition.” address the original claim that unusually good (and presumably ORIGINAL) work, as evidenced by the number of subsequent citations, are being prevented from publication?
If the counter-claim is true, then the filter that is supposed to prevent repetition is preventing originality, while apparently the work that confirms, if not repeats, old work is flowing around or through the supposed filter.
On the evidence it would seem to be easier to suppose peer review is NOT intended to filter old work, but IS INTENDED to confirm old work. The “peers” are then privileged players in a position to repel challengers rather than an audience for new and novel ideas and proofs.
91
Dr. Evans. That is not so in many engineering fields. Many regulators in county/state/federal flood control, bridge/culvert design, 100 year flood plain delineation, structural analyses for large projects, etc. now require the original field and remotely sensed data, model input files and model output files, model name , version, and authentication of version so that regulator employees can reproduce the submitted results. Most that I am familiar with will reject the request if they cannot replicate the analyses and result. But, most of these people are engineers and the submitting engineers are registered and must include an original stamp of registration on all copies submitted.
This may sound severe and that it suppresses innopvation. This is true to an extent, but original research gradually changes the standards, data, models, etc. The important point is that the roads, structures, buildings, bridges, etc. are much safer because of the process of replicating submitted designs and analyses.
50
Good!
30
Back in the distant past, one of my hats was as a structural design engineer. I studied Mechanical Engineering subjects; machine components, vibration and noise, control systems, electronics and electrics, materials science, thermodynamics and fluid mechanics, numerical analysis, alternative energy systems so I was obviously the person “most” suited.
All my calculation sheets were prepared ostensibly for independent verification. My engineering boss used to do most of the designs for customers and I handled the “overflow”; except when things got a bit to tricky for him and the customer wanted information from outside of his comfort zone. One of those occasions was (in August 1986) when a public servant (Engineer) queried how the maximum footing capacity from a pressure distribution under a slab footing had been calculated by my boss; using his tried and true recipe. The public servant asked for proof that the assumptions were appropriate.
And they weren’t as the public servant knew! Don’t ask if you don’t already know the answer. 😉
And so I set forth, deriving equations from first principles including integrals and second-order, partial differential equations; eventually arriving at the simultaneous solution of two polynomials. I asked my boss to check my work — maths isn’t my strong point — but his eyes glazed over. So he spoke to some consulting engineers and they suggested that we submit a series of “standard” designs with ratings for different soils so that they could check them. Those equations had “many” solutions; a lot of them not real. I rolled out a proven equation solver FORTRAN subroutine and plugged in the parameters corresponding to standard footing dimensions.
The parcel prepared for the consulting engineers included 22 pages of abbreviated calculations and their solution. The consultants phoned me shortly after receiving the parcel, commenting on the “novel approach”. And he wasn’t being sarcastic. The consultants routinely used a commercial computer program to suck-it-and-see; if a particular footing was adequate to support a particular load. Such computing expenses were out of the reach of small to medium fabrication engineers at the time.
In summary; things were being done as they “always” had been. Then somebody noticed a problem and asked a penetrating question. The solution wasn’t easy to find, nor easy to confirm because of the established mind-sets of those involved.
00
“Non-originality is a stronger barrier to being published than quality.”
This is only part of the story.
It often also depends on WHO is being original, and for what purpose. Often the only people who are allowed to publish original ideas are gatekeepers, and only if it suits their particular cause or agenda, which is a complex system determined by flows within social value systems and identified deficiencies within market forces.
My experience within both the public sector, universities and private industry tells me resistance to publication of controversial and innovative ideas is very often all about social hierarchy. It’s not so much about ‘what’ is being said as ‘who’ is saying it. The main reason that ground-breaking ideas are sometimes not published within academic journals is the same as occurs in internal office politics within private companies, it may upset those higher up on the social scale who didn’t think of it first, and/or those who may not like doing it that way because that is not their background and training.
Think of it, you’ve spent your whole career perfecting a technique or process or idea, with lots of training and emotional investment, which has given you social standing and enabled you to become an expert in the field, which is then nullified by an inexperienced upstart who discovers something which you never even thought of, and which makes most of what you have done and currently do either obsolete or irrelevant. What’s more, you cant quite get your had around it, because it isn’t the way you have been trained. In developing markets its a well known issue, and it’s called ‘creative destruction’, and it’s also one of the main reasons for the lack of social and economic development in developing counties, the existing and often rigid social hierarchies do not necessarily benefit from new ideas and innovation, in fact they may be downright negative and destructive for them.
In these instances, it isn’t ‘originality’ that is the problem, but a social hierarchy which does not accept something which would upset change within the social hierarchy.
This is also traditionally very obvious when one looks at, for example, what the church tried to suppress throughout the middle ages; people were branded ‘heretics’ and such like when research and ideas threatened the church’s social dominance and control, in other words, when such ideas led to ‘creative destruction’ of the church’s control of the way people thought and behaved, and moreover also threatened the social relevance and importance of those within the church who had invested their entire lives to something which was now making their existence less important and relevant than they thought it was. Originality was, and is, actually discouraged in such a system.
110
thingadonta, an insightful comment,
as was a comment a serf recollects by Dr Evans
waay back,terwit ‘should the facts change,I change,
what do you do, Sir?’
Engineers live closer to the littoral,’London Bridge
falling down,’ than some other seekers after truth,
climate scientists fer instance or Joseph Stiglitz
economists predicting fuchure trends in cloudy mucho
interactive zones.
As fallible humans we’re prone ter digging in, and
ivy-walled and tenured academia lends itself ter terf
centred-ness. Serfs sense this …Gate-keeping – bad.
Open debate and testing of conjectures – good.
30
beththeserf- The quote is attributed to Keynes:
“When my information changes, I alter my conclusions. What do you do, sir?”
The irony is, Keynes held fast to a set of illogical propositions when they became popular.
20
As an economist, I am curious. I know how Keynes has been corrupted, but did he defend the corruption (I have not read that he did – but instead held true to his original contention).
00
When governments brought in Keynes recommendations, the resultant data on growth, inflation, etc were not consistent with the model, the model thus was pretty well falsified. Governments did not want to relinquish the powers that Keynes said were needed. They said they had to stimulate more -money printing. Hoi polloi are impressed by government ‘doing something’. Money printing does not work, but Keynes did not retract, he or at least his followers convoluted and contrived more ingenious money printing techniques. The story of Keynes is a tragedy showing how fame and fortune corrupted one of the most brilliant minds of the age. It is not that Keynes’ ideas were altered beyond recognition, but that he would not reconsider when the data were in, he was on a roll.
00
There are parallels between Keynesianism and the CO2 theory:
– Both assist the growth of government.
– Funding of economics PhDs by govt and banks meant that from the 1950s on you could really only get a PhD in economics if you were a Keynesian. All the plum govt consultancies and roles in economics went only to Keynesians. Thus almost the whole economics profession has been Keynesian since the 1950s. Likewise the CO2 theory in climate from the 1980s or 1990s.
– Keynesianism was considered crackpot or silly before 1930. Likewise the CO2 theory used not to get much attention (not that you’d know that now).
– Both theories create economics goods (money or carbon emissions certificates) out of nothing, for speculation and sale by the finance industry, and which the general population has to provide real goods and services to obtain.
40
@Lucky – I am fully aware of how governments corrupted Keynes teachings, but did he promote the corruption? That is the question. In my studies, I have not found that he did. Now perhaps he was not vocal enough in clarifying his theories and rejecting the corruption. But I did not find where he changed his theories to conform to the corruption. That is not to say he did not, just that I have not found where he did.
00
@Dr. Evans
All you say is true – for the corruption of Keynes. I am not a Keynesian (being more of the Chicago school, which now is the Austrian camp), but I have yet to see where his theories actually got a true test. I think they would fail (the corruption is extending the time limit from “short term” to extended). So even the corruption did not work, which does provide evidence that his theories would not work. Yet for his theories to work, government would have to “live within its means” most of the time. An accomplishment that has not occurred. If government could, then his theories could get an honest real world test.
And the biggest similarity between Keynesian Economics and Climate change is that government corrupts. Government is not funding research, it is paying for a product. And it only continues paying as long as the product is what it wants.
00
Yes, that’s certainly a major factor thingadonta. It’s strong, and can prevent people invested in the old ways from seeing the obvious, when others with little investment see it immediately.
60
I’ve suspected that peer review runs much like the president’s press secretary, who parrots the party line the president wants emphasized, regardless of anything else.
I admit that I’ve oversimplified the situation. But really, isn’t that the basic mechanism at work here? In the case of Josh Earnest it’s keep the line Barack Obama wants. In the case of the peer review process it’s keep the party line, the view that keeps the funding and the recognition coming.
It seems like a classic case of what I’ve mentioned before — sooner or later an organization begins to serve only itself. Now the process by which science has been conducted for a long time has begun to serve itself instead of science.
140
My thoughts too Roy, in politics a trick to make yourself appear knowledgeable to people is to use keywords of a subject using your own bent on them, what you end with is an opinion given without substance, hell even unqualified uneducated people can sound clever with the right amount of training. 😉
Watch any MSM political interview and you’ll see a repeated robotic mantra never straying from the safety of those confines, they present like a live legal contract paranoid of being found libel if an original thought occurred.
60
Darwin’s theory is one. The fossil evidence is not there but not admitted.
00
The IPCC gets it right and the IPA gets it horribly wrong!
357
And please.. tell us just ONE thing the IPCC have got right ?
311
Griss, my friend. Don’t respond to the flamer. What does the IPA have to do with the topic of this thread? Nothing!
Oh, the calamity of the inanity. As an active member, the comment defies logic.
181
Please point to one of these “controversial “champion” papers” in this
240
Once YOU have read it, maybe you will see the reality passed your brain-washed insanity.
Go on, purchase a copy.. read it and learn.
I dare you. 🙂
222
I thought did me a favour SF. But alas, not.
I have no idea why my Commonwealth Bank credit card is not a valid one. Any ideas?
10
You could have accessed a site designed to scrape your credit or debit card details, perhaps? In which case SillyFilly has been a naughty horse — a very naughty horse, indeed.
50
SF and the logic in the same sentence? They don’t even inhabit the same planet.
191
SF always defies logic. !!
It doesn’t even know what logic is.
And certainly cannot support any tantrum/rants it throws.
132
Inflame the flamer? Got an interesting email from Gina a few hours ago. That gal just keeps kicking goals. I suppose you will read it in the MSM in the next few days.
Can horses read…?
81
You can lead a horse to a source, but you can’t make it think.
111
You can take a sauce to a horse, and make it much more palatable.
30
Horseradish?
20
Dogfood?
20
Or as it appears on the menu in a French restaurant; Cheval.
20
According to Wikipedia:
10
I mean sillydilly can ye not put something worthwhile towards the discussion? Maybe,on reflection, you can see some problems with the peer-review system;common to all the fields of post-modern enquiry.
211
Not in Jo’s context!
227
So you’re personalising the experience and refusing to cooperate at any level, because this is Jo Noava’s blog?
71
“So you’re personalising the experience and refusing to cooperate at any level”
Is this Jacqui Lambie? 🙂
81
Oops, wrong thumb. Apologies!
00
Silly Filly —
Silly, I do hope you won’t mind if I ask you to cite an instance where that statement is true and back it up with data, citation of research or whatever it takes to make your statement worth making.
Can you do it? I keep waiting…
231
And Silly Filly is silent just like so many others I’ve challenged to do more than point a finger and say, “You’re wrong and I’m right.”
One more player at the table with an empty, busted hand and a loud mouth.
20
The way to get a paper past AGW peer review is to make sure the abstract appears to support the AGW hypothesis.
The AGW peer reviewers aren’t very bright and probably will not read much of the actual paper (because its too hard), so if you can hide your conclusions somewhere within the actual paper, you can sneak it through the gatekeepers.
How do I know this…
…. sorry, but that’s a secret 😉
212
In ‘climate science’, I would impose these rules on the peer review process – no exceptions:
1. No research papers to be hidden behind paywalls.
2. All data to be available at the time of publication in an easily readable/extractable form.
3. Full and comprehensive details provided on how the data was processed and the conclusions reached.
4. Those doing the peer review to not have published articles with any of the authors of the research paper.
5. Full details provided of who provided funding for the research paper and how much was provided.
6. if ‘homogenised’/manipulated data is used, there must be clear comparison tables between this data and the raw original data, with an explanation as to how/why this was done.
7. Research papers directly and indirectly funded by obviously biased activist organisations such as Greenpeace, Friends of the Earth and the David Suzuki Foundation to be excluded from consideration. Here, great care will be needed, as there are some huge, low profile, greenie organisations in the USA which will finance anything on climate, as long as the conclusions point to imminent Thermageddon.
If these simple and perfectly reasonable rules had been followed, do you think we would have ever heard of Michael Mann and most of the other high profile names in the climate alarmist movement?
At least this would be a start
453
Thumbs-up from me Peter.
41
Is that because you think he is not saying what actually occurs?
115
Gee Aye, you must skip read the posts. Jo has posted several threads this last year alone where studies have demonstrated failure of pear review to do exactly those things.
Here’s one such thread.
http://joannenova.com.au/2014/06/failure-of-peer-review-meaningless-statistical-significance-needs-fixing-says-doctor-journal/
61
Peter, I object to the rejection of papers on grounds of who funded them. It is enough to state who funded them. Piont 1 is about journal funding not science funding. I agree with your other points, 2-6, and I wonder how anybody could possibly disagree?
120
I agree re: funding – where the money comes from doesn’t alter the facts: that’s an argument alarmists use; they should be judged on their merits based around the scientific method.
131
Besides, there are various Foundations and Charitable Trusts that act as a
money laundering, a common focus, for donations, for all sorts of purposes. The Tides Foundation, being just one example.20
Not bad except for No:7, it should read:
7. Research papers directly and indirectly funded by obviously biased activist organisations such as The Galileo Movement, Heartland Institute, Global Warming Policy Foundation, Institute of Public Affairs, Strategic and Public Policy Institute, NIPCC, and most so called climate sites utilising the term “Climate Science Coalition” to be excluded from consideration. Here, great care will be needed, as there are some huge, low profile, denialist organisations in the USA which will finance anything on climate, as long as the conclusions point to imminent Coldageddon.
And then conclude:
If these simple and perfectly reasonable rules had been followed, do you think we would have ever heard of any high profile names in the climate denialist movement?
245
If a “theory” is wrong, you’ll have dissent. Dissent doesn’t have to be funded to be effective.
I do wish you’d look further afield and see that your logic goes both ways. You’re not more correct because you believe what Paul Nurse believes. But you could easily be lead astray by doing that.
I accept that you believe “there are some huge, low profile, denialist organisations”. Do you also accept that there are huge businesses and organisation taking money FOR the global warming cause too?
151
Greg is absolutely correct. I run such an organisation. You can tell that, from my penchant for white Persian cats.
30
‘…conclusions point to imminent Coldageddon.’
Evidence suggests there is only a couple of years left in global warming, please prove me wrong.
31
Alfred Wegener’s theory of Continental Drift was long rejected by most geologists and his papers were commonly prevented from publication by peer review. Nowadays continental drift is accounted for by Plate Tectonics and forms part of conventional geological opinion.
280
John,
It was peer review that eventually come round to support him to form a consensus view, you cant have it both ways.
115
Rubbish.
It was discovery of more and more evidence supporting his theory.
171
Frank:
Peer review had nothing to do with Wegener’s hypothesis of Continental Drift (1915), which was proven wrong.
He wasn’t able to provide a mechanism for the movement, which created a lot of peer rejection, even though there was visible evidence which we now easily accept in the light of modern knowledge.
Yes, there was a general growth of evidence over the years but still no mechanism. It was The Nuclear Test Ban Treaty which accidentally provided the impetus, the investment and the equipment which made the discovery of plate tectonics inevitable. There was wide-spread installation of seismometers around the world to monitor Treaty adherence. These instruments showed that earthquakes, volcanoes, and other active geological features mostly aligned along distinct belts around the world. Plate tectonics was born and those belts defined the edges of tectonic plates. The whole point of the installations was to pinpoint exactly where an explosion occurred.
Other strong evidence was the discovery of magnetic `stripes’ of opposing magnetic polarity alternating across the ocean floors. So the ocean floors were moving, not the continents.
The evidence for Plate Tectonics was overwhelming.
Wegener is credited for being first with the idea of moving chunks of the earth’s surface but Continental Drift and Plate Tectonics are quite different. The first has no mechanism, and the second is all mechanism and well supported by strong evidence.
22
Alfred Wegener was a German polar researcher, geophysicist and meteorologist.
The claim that Wegener’s theory of Continental Drift had no mechanism is not true. Wegener used his observations of the movement of sea ice off the Greenland coast to develop his ideas of continental drift. He noticed that the movement of tabular ice (continents) created surface pressure ridges with counter balancing roots (fold-mountains) and that the ice floes separated with extensional fractures (rift valleys).
NASA video of massive Arctic ice fracture
Wegener also speculated on sea-floor spreading and the role of the mid-ocean ridges, stating: the Mid-Atlantic Ridge … zone in which the floor of the Atlantic, as it keeps spreading, is continuously tearing open and making space for fresh, relatively fluid and hot sima [rising] from depth. However, he did not pursue these ideas in his later works.
Alfred Wegener: Continental Drift Theory
10
Rubbish, it was the discovery of the Atlantic mid ocean ridge and subsequent facts.
81
No Frank. It was facts, new discoveries and scientific discussion that eventually broke through the “accepted” ideas.
You should try entering the scientific discussion sometime.
Maybe post a link to a certain paper, but that you seem to be having extreme difficulty finding.
122
Round and round we go. Frank doesn’t understand science and it appears nothing is ever going to change that. You’d have more success teaching science to my dog, at least she learns.
82
You contradict yourselves —-again.
The system that came up with the ‘accepted ideas’ is the same system that corrected itself, that’s how it’s always been.
The peer review process that you now attack is your only process by which you can gain recognition . It’s worked fine this far , somehow because it rejects your evidence you suddenly want to change it.
You can prove a positive ,just find some better evidence.
Illogically extrapolating a problem regarding medical articles to all scientific peer review to then conclude that – ‘Anyone who thinks “peer review” is somehow part of the scientific method does not know what science is.’ is hardly scientific.
213
Good lord Frank, learn to read before you respond with some twisted mess of gibberish that has absolutely nothing to do with anything anyone is talking about.
92
R,
I’m talking about the peer review process, what OT gibberish are you on about ?
211
No, you are claiming that peer review changed a theory when it was the empirical evidence that changed it. This has been pointed out to you a few times yet you persist with your delusions of how science works.
The theory would have changed with or without peer review.
Peer review is not the only process by which to gain recognition.
Quite the contrary, there is nothing illogical about it. If the problem happens in medical journals it will happen in other journals. That “extrapolation” has nothing to do with the statement “Peer review is not a part of the scientific method.” Peer review has never has been a part of the scientific method. Peer review is part of the publishing process, which also has nothing to do with the scientific method. You would think by now you would have looked up the definition of the scientific method since you’ve been called out so many times for your lack of knowledge regarding it.
Don’t confuse the process of validation with peer review, they are two different things.
But we do understand Frank, you have to twist thing in order to support what you think is an argument because what you think is an argument won’t stand up any other way.
121
Robert has said it better than I can, but my point is that the peer review process does nothing to “change the science”. Peer review is all about protecting the reputation of the publication.
In Maori, the concept is called, “Mana”, which means more than reputation, it also means deference and respect. That is what the Journals strive for, and that is the whole reason for, and purpose of, the modern, orchestrated, peer review.
Prior to the emergence of the Scientific Publishing Industry, scientists just simply circulated their papers to their peers, for review and comment. Simple. Now, apart from orchestrating a process of “gate-keeping”, what do the “official Journals” add to that process.
80
One of Richard Feynman’s essays is about science being just as much subject to cultural influences as politics, etc. I’ve searched through my books but haven’t found it. Does anyone know the title of it or which collection it’s in.
60
Almost 12 hours later – five ticks (thanks) but no answer that helps me. I’m sure the essay isn’t only in my imagination.
00
I struggle with peer review. On the one hand as a senior editor of a medical journal I sift the nonsense (for example: plagiarised; repetitive; flawed – unable to separate chance, bias and confounding from the findings and subsequent claims) from the manuscripts that possess merit. The process is demanding and time consuming. On the other hand, I have had my own original, insightful research rejected, to be later published in a low impact journal, become the focus of ‘original’ investigation at a leading institution elsewhere.
I learned that the rejection of my own research was predicated on the ‘disposition’ of the editorial team of a journal at the time. As it departed from the clinical ‘consensus’ it was editorially dispatched with alacrity, although it passed the muster of peer-review.
It became evident to me that in my discipline (and others), there are empire builders, a phenomena inextricably linked to funding and current practice paradigms. As is the case with any other domain, challenge to the moment entertains little in the way of contestation. The moment is often supported unwittingly, consensually or as a matter of policy. The policy imperative that engenders this may be a simple matter of journal editorial policy, discipline policy, institutional politics or ‘political correctness’.
Ensuring the peer review process is subject to equal scrutiny and transparency as the submitted manuscript (all anonymised) may be one way to circumvent distortion of a process that has fundamental merit. This has already commenced in the open-access arena. It is, I have observed, sneered upon by those supportive of “climate science.” They are reflexively disposed to dismiss low impact journals and open access.
210
“I learned that the rejection of my own research was predicated on the ‘disposition’ of the editorial team of a journal at the time. As it departed from the clinical ‘consensus’ it was editorially dispatched with alacrity, although it passed the muster of peer-review.”
The usefulness of “peer review” is at an end. Its flaws outweigh any benefits.
If an editor wants and needs the submitted papers reviewed, then there should be a “review board” of people whose names are known and they should never review papers that directly impact their own field of research. Go for unbiased reviewers as much as possible.
Also troubling is rumors of a paper by some unknown fellow being rejected and then later some VIP type scientist or team then “makes a discovery”. I am concerned that the next Einstein will be kept from adding to human knowledge: and that is so troubling that it outweighs any benefits of the present system in my view.
Disclosure: I am a no-body, have always been a no-body, I am descended from no-bodies, and I will strive to remain a no-body till I leave this mortal plane.
160
So how DO you do it Jo? I’m on your side re AGW but sniping at peer review is unhelpful if you don’t have anything better to replace it. I am a research scientist and I also consider that I have suffered from the process you describe. But what is better? Nobody expects to find ground-breaking papers in Nature anyway, just high-quality ones within the present paradigm, and important new experimental results. And the innovative stuff *did* get published, but elsewhere.
70
Something better to replace it would be nice, but more to the point right now exposing the fact that just because something was peer reviewed that does not make it a) science, b) valid, c) good, d) all of the above.
Too many times we hear the “peer review” mantra as though that “settles” the issue. Generally from people who not only don’t understand peer review but who have no training beyond their high school science classes.
To me it isn’t so much “sniping” at it as exposing the flaws in it so a few eyes might be opened. Considering the target audience we are discussing I don’t hold my breath that it will result in any revelations for them but at least the effort is being made.
121
What are you saying?
17
I’m not going to spoon feed you an explanation. Do you not have any reading comprehension skills? Read it again, think about it, it should be obvious but given the issues with reading comprehension I’ve been seeing on campus over the last few years it wouldn’t surprise me if it confuses some.
111
Robert: Same question to you as to Jo: what would you replace it with?
30
It depends on what we are trying to accomplish. At this point I’m not even thinking of what to replace it with, I’m more interested in educating certain parties to the fact that peer review does not mean that because a paper went through it and was published all of the assumptions, formulas, math, references, etc. were checked. Too many people seem to have this idea that because it was peer reviewed it was meticulously scrutinized, results reproduced, and therefore whatever the paper states has been validated. Sometimes that may happen, more often than not it doesn’t.
People who want to wave the “peer review” banner around as though it means a particular paper is beyond question need to understand what peer review is complete with all of the failings it has. It is not the “be all, end all” passing of judgement many people seem to think it is.
Sometimes peer review just means that someone looked at the paper to see that it followed a particular format and was legible. But since so many people want to claim peer review validates a paper then we need to educate them on what it really means. You may see that as “sniping” at the process. I don’t.
140
Robert, I admit all of those failings of peer review. I’ve suffered some of them. No scientist I know thinks this system is perfect, and I welcome the education of the general public – who pay for a lot of the research that is done – to that fact. But, to anybody who wishes to end peer review (which might not include you, I accept), I do ask – courteously, I hope – what would you replace it with?
70
Robert and Anton ,
I don’t think we should be thinking of replacing Peer Review but modifying how it works and also explaining more widely what Robert is saying about what PR means.
I think that , as was alluded to above about anonymity , that once the paper is accepted for publication the reviewers names should be published as well and also the comments they made. The later is easy with online publication ( with a link) but maybe a bit tricky with hard copies ( to avoid it ruining the look of the layout of the paper). But even a link to the publisher’s website would do it, where the reader could find a file of the comments. This would make it transparent and stimulate better debate on the paper. I have seen this approach with an online medical journal and even as a layman it was helpful to me.
Some might argue publication of the reviewers name would make people reluctant to do the review. If that is the case, they should not be doing reviews.
50
Ross, that is the crux of the matter, we don’t need to replace it we need to fix it.
But people need to understand, some more so than others, that just because something has been through peer review that does not mean it was validated. It only means it went through peer review.
And peer review seems to vary in scope and purpose depending on who is doing it. Standardization of the process might be a place to start. Then it would not matter which publication, which reviewers, or what topic, it will be handled the same each time and every time. At present I don’t see that being the case.
50
Thanks for this comment Anton. It got published, cited and built upon. So what if nature missed out. Their bad and peer review did its job.
111
WOS
00
Anton, you are looking at it.
Peer review is already replaced by open publishing on the net.
Current peer review serves no one except the publisher, it never did say anything about the science.
The argument of authority conferred upon certain conjecture by it clearing “Peer Review” is absurd.
As is most argument from authority in science matters.
The reformation and the scientific method are/were tools to bypass the herd mentality.
Funnily enough just cause Big Chief says it is so, does not make it so, the overweening arrogance of tax funded academics is a reason Climatology as pseudoscience has continued for far too long.
The irony of bureaucrats as the final arbitrator of truth, may escape the committees comprehension.
231
+ 10 . Very well stated.
50
John,
The internet is a game-changer, for sure; the question is how to utilise it. The original purpose of scientific journals, which began in the later half of the 17th century, was twofold: dissemination, and quality control. The internet makes the former redundant. Peer review has been the way the latter was handled, But if you junk it then notice that there is a remarkable amount of rubbish around, and how to separate out genuine science from it? Scientists need some form of quality control. What we can now live without is publishers who make us do our own typesetting, charge us page charges, make us sign away copyright on our own work, and then charge the libraries of our own institutions vast sums to buy back our own written-up research. Thank you for the last 300 years and goodbye. But the scientific community still has to work out its own quality control system.
I believe that referees should remain anonymous or lawsuits will be flying around from some litigious scientists, and anybody who reads this blog can think of a few.
40
Just an opinion from a layman, I would ask, why don’t each of the scientific bodies set up at arms length a so called ‘clearing house’ for all papers and once reviewed and cleared have them published on line. This would hopefully eliminate the junk sites and adopt some of the suggestion made above on this thread re rules and procedures. I’m sure that some of the better minds here could put together a workable solution.
10
Anton surely replication is the ultimate review.
The original concept of pursuing an inspiration by running it by your friends and colleges, exploring its value and possibilities is science.
The question becomes who are ones peers?What makes them such?
The active researchers?
The administrator?
Or the fund approval committee?
As for quality assurance, that would probably require a network of like minded persons who share such ideas as quality eventually shines through, if it challenges one member they will pass it on.If not it dies..
10
My view is that peer review is like democracy – lousy until you start looking at the alternatives.
00
JR,
So is peer review broken in all disciplines ?
Should the flat-earthers demand more evidence ?
Should we replace it with trial by blog ?
28
Would depend what you are peering at.
But if it is taxpayer funded via funding committee the answer will be yes.
No one can think for you, either the conjecture holds up to testing or it does not.
There is no certainty but it remains the best method we have so far.
Basic scientific method is a tool, who uses it and how they employ it is always wide open.
Seems many people are very uncomfortable with uncertainty, possibly hardwired to believe.
Most of the excuses offered up to justify current publishing practises are intellectual laziness.
As in I do not want to waste my time reading unknown material, please predigest it for me, masters.
51
JR,
So gov funded science = bad, fossil fuel funded = good, not much thinking going on there.
113
Science ?
We are talking anti science here, peer review as a means of controlling scientists and protecting poor practises.
I have no evidence of a fossil fuel funded attempt to usurp power and steal wealth using claims of science as a cloak.
Policy based evidence manufacturing is however, now a well established bureaucratic practise.
110
The lack of thinking, and the reluctance to accommodate new ideas, sits entirely with you, Frank.
50
“demand more evidence”
ANY evidence Frank? !!!! you have none. !
92
G,
Stop shouting, it’s Christmas you know.
The evidence is in the real scientific world, please go look, you can whine away in the CT sandpit but the onus is on you to disprove it.
215
You are the one unable to prove even the most basic belief of the CAGW hoax.
I have posted links to papers that cast great doubt to that hypothesis.
Now its up to you…. You have produced nothing.
Treat it as a brain-washed religion, or provide the evidence to back it up.
Are you really that lazy, that you can’t provide your own proof ??
102
Then provide the evidence or shut the hell up Frank.
The onus of proof is not on us, we don’t need to disprove it when it hasn’t yet been proven.
You are getting to be as tiresome as BA4 and, at least in my opinion, deserve the same treatment.
Since you cannot or will not learn anything about how the scientific method works or how hypotheses are validated then you really shouldn’t try talking science to anyone.
121
Hi Frank, it’s me, your conscience,
Now you know that all the observable evidence shows:
1. Weather patterns within natural variations.
2. Hotter periods in recent history until records were homogenised to conform with prevailing opinion.
3. CO2 based models all failed to predict actual events, proving only that CO2 is not a factor.
4. Majority conforming to a false consensus based on opinions of 59 papers carefully selected from 11,944 papers.
5. Anyone who dares question the consensus let alone the science is harassed, threatened, marginalised and abused.
6. Dams are full, snow on the ground, polar ice increasing, crop yields increasing.
Stop me anytime, Frank – feel free to correct any of the above points with real observations.
172
And seriously.. expecting me to do your work for you, because you can’t.
Quite funny, in a “sick” kind of way 🙂
92
‘…the onus is on you to disprove it.’
The pause in temperatures means CO2 is not a pollutant and in the absence of warming the government funding for AGW theory should continue apace, until global cooling sets in.
61
Actually Frank, in science, someone proposes a theory and then sees if that theory is supported by empirical evidence.
The onus is on the proponent of a theory to demonstrate how his or her theory is supported by empirical evidence.
When there is no empirical support the theory gets trashed.
Of course, some people continue to support a theory even in the absence of empirical evidence.
They do so by adding ad hoc supplementary hypotheses to save the major theory (a Quinean web).
So, for instance, if someone, say, predicted catastrophic global warming and there was none, they might excuse the absence of evidence with exhortations of, say: ‘Low solar activity’ or ‘Oceans ate the global warming’ or ‘Chinese coal use’ or ‘Montreal Protocol’ or ‘What ‘pause’?’ or ‘Volcanic aerosols’ or ‘Stratospheric Water Vapor’ or ‘Faster Pacific trade winds’ or ‘Stadium Waves’ or ‘Coincidence!’ or ‘Pine aerosols’ or ‘It’s “not so unusual” and “no more than natural variability”‘ or ‘”Scientists looking at the wrong ‘lousy’ data”‘ or ‘Cold nights getting colder in Northern Hemisphere’ or ‘We forgot to cherry-pick models in tune with natural variability’ or ‘Negative phase of Interdecadal Pacific Oscillation’ or ‘When there is an absence of empirical evidence’ or … etc
Even if all these excuses were refuted more could be generated (ad hoc hypotheses are anti-falsification).
But they don’t need to be refuted because the onus is on the proponent to show their theory is correct.
Until a theory is empirically validated it remains… a theory. That is basic logic.
70
The only thing I would change in that would be to replace theory with hypothesis, since that is as far as the various claims being made ever got. The validation stage keeps failing for them so it can never advance beyond hypothesis. In any other field it would have been a discarded hypothesis by this point.
100
Eh? Did some catastrophic warming take place while I was decorating the Xmas tree?
If not, you cannot /possibly/ compare CAGW denial to flat earth affirmation.
CAGW has never been observed.
It is therefore a theory/conjecture/assumption that stands or falls on the /empirical/ evidence (not computer modeling).
The empirical evidence does not support it.
Game over.
60
Peer-review in today’s science means nothing. Anybody can get a paper ‘peer-reviewed’……. even Maggie Simpson, Edna Krabappel, and Kim Jong Fun from a non-existent affiliation (“Belford University”);
http://www.vox.com/2014/12/7/7339587/simpsons-science-paper
Even though I’m nothing but a retired motor-mechanic, I might even join the PNAS where I can write 4 papers/studies a year…. and get to choose my own reviewers;
http://www.vox.com/cards/savvy-science-reader/peer-review
180
Nature admits that they don’t always pick a winner… And? They are saying something long known that the “top” journals will reject papers that well exceed their own average (pick your criteria). since they are a self appointed elitist tome, what is the evidence that papers are rejected for bad science or significance? How many papers that outperform the nature norm were rejected for being bad and how many for not meeting their ideal?
112
The last 538 comments from GeeAye in 2014
“Have been a WOS too”
But Merry Xmas Gee Aye
10
As it turns out, peer review is not needed in science and may well be detrimental to the exercise of honest science. Consider this:
The above from: http://michaelnielsen.org/blog/three-myths-about-scientific-peer-review/
The entire article is worth your time to read. It is not very long.
From the climate-gate e-mails we saw that “the team” was using peer-review to stop truth from entering the record when that truth went against the agenda of “the team”. Science should be done in the open with crystal clear transparency and peer review is a great impediment to that happening; at least in climate “science”.
Peer review might have been useful until politics entered the picture, but since almost all science now has funding by the state in some regard or the other — peer review is just useful as censorship of real science.
230
“Anyone who thinks “peer review” is somehow part of the scientific method does not know what science is.”
Well said, Jo!
261
Off topic – but thanks for all the hard work Jo in bringing the many articles that show how ill-founded the AGW argument is and allowing someone like SF to continually show how ludicrous their head in the sand views are. To see these individuals constantly resort to calling us skeptice “denialists” shows the fact that they cannot argue facts or actual science. Merry Christmas and there is a glimmer of light at the end of this long tunnel.
101
WE avoid many problems by demanding that authors submit the raw data, lab results, printouts ect.
80
The field of climate change ‘science’ has blatant instances of agnotology (hide the decline, climategate) which is different than a reluctance to publish data that disproves a hypothesis/theory.
In the field of climate ‘science’ there is a cabal of ‘scientists’ who believe it is their moral duty (they work to push an agenda as opposed to solve scientific problems) to hide the observations and analysis that disproves the extreme AGW ‘theory’. The most obvious example is the effort to hide unequivocal paleo evidence of past cyclical warming and cooling, that matches the pattern of warming recently observed. The pattern of warming recently observed does not match the pattern of warming if CO2 was the forcing agent. A second example is the effort to hide the implications of the fact that there has been no increase in planetary temperature for 17 years. The IPCC is a who’s who of the climate change misleading cabal scientists which makes sense, as ‘IPCC’ directors and non-government agencies that are associated with the IPCC picked the cabal scientists to write the IPCC ‘reports’, the IPCC has formed to push the AGW movement, not to solve a scientific problem. Direct action to mislead the public and politicians concerning scientific issues is active propaganda, agnotomogy, not science.
http://link.springer.com/article/10.1007%2Fs11191-013-9647-9
Climate Consensus and ‘Misinformation’: A Rejoinder to Agnotology, Scientific Consensus, and the Teaching and Learning of Climate Change
http://wattsupwiththat.files.wordpress.com/2012/09/davis-and-taylor-wuwt-submission.pdf
192
Somebody posted a link yesterday in Weekend Unthreaded which discussed the hiding of 80 years worth of reliable ocean PH data in order to push the “ocean acidification” meme. I was very interesting.
142
That would be this link.
72
and picked up a here.
82
and at WUWT. 🙂
82
G,
How is cherry picked evidence from 3 other sceptic sites seen as proof of ‘hidden’ data ?.
Your living in a closed self referential loop.
221
Have you found that paper yet Frank ?
Nothing so far ?
Keep trying little child. !
122
It seems that poor Frank lives in a ZERO reference world. 🙂
132
Yet you will willingly take information from 3 sites that support your beliefs, regardless of whether or not the information they present is valid, as proof. amazing how you can justify doing the same thing you berate another for. Well not really as we know you’re stuck in a loop. You’ve been trying to demand reversal of proof from us for how long now? When, per the scientific method and how hypotheses have been validated for centuries, the burden of proof does not fall on us, it falls on those claiming that CO2 causes whatever they are claiming.
Grow up, get an education, learn how to think. If you can manage even one of those I will be surprised.
101
In my line of work, any evidence is better than no evidence at all. We can sort the truth out for ourselves, when we have more time.
So Frank, would you like to help our little project by providing some evidence of your own? At the moment, “The three sites”, are looking pretty good, but I would just love to balance that with your evidence.
I know you must have some, because you are so fervent in your opinions.
How about it, Frank?
30
The study was not about climate science or about articles submitted to Nature.
It studied 1008 articles submitted to three top medical journals in 2003/2004. The journals only published around 6% of the articles but many of the rejected ones were eventually published in less prestigious journals.
You can read the full study here (pdf)
30
David,
Welcome to cherry-picking world.
08
Yep, the AGW world is one big heap of cherry pips.
You know that
otherwise you might be able to post something of substance to back up its basic ideas.
But you can’t, can you.
NOTHING !!
71
Griss it seems apparent that neither one of them can read. Nowhere did it say the study was about articles published in Nature and nowhere did it say the study was about climate science. The topic is peer review, and if it happens in the medical journals only a fool would think it doesn’t happen in the others as well.
Now if David comprehended what he read here he would have noticed that we could already read the full study via the link to it provided when this topic was created.
And Frank responds with a cherry picking comment? That’s all either of them are engaged in.
You know, I really don’t care if someone sees the issue the same as I do as long as they can discuss it intelligently. I’m not a fan of bans for numerous reasons, but I’m really beginning to think Frank needs to get the BA4 treatment. He doesn’t learn, is incapable of discussion, and seeks only to disrupt things.
91
As they say,” ’tis the season to be jolly”
So Frank’s clown act must be part of the show. 🙂
81
R
‘if it happens in the medical journals only a fool would think it doesn’t happen in the others as well.’
What a piece of logic .
And you have the gall to claim you’re arguing rationally.
114
Explain the flaw then Frank. I doubt you can. I’m not concerned about what you think, the record shows I’ve outed you for your lack of ability to argue rationally on numerous counts. It’s no surprise that you can’t come up with anything better than that as a response.
81
I don’t mind him wasting his time here.
He does immeasurable damage to the AGW meme because he again shows that the general alarmista has basically nothing in the way of science to back their irrational brain-washed belief/religion.
Its all just non-thinking regurgitation of force-fed BS. !
71
Hey Griss, I don’t mind him wasting his time either, I don’t mind me wasting my time when I choose to do so, I do mind when someone like Frank wastes my or others time. You know, strange as it is to say, back in the early days at least the trolls had some science background. The ones we get these days seem to be stuck on stupid.
But it is holiday time, so I plan to waste some of my time over the next two days off by playing with the dogs, watching some movies, maybe some playing a little Wildstar or Skyrim.
Have a very Merry Christmas, a wonderful New Year, a Happy Hanukkah, or a pleasant whatever it is you choose to call your particular holidays.
50
Yeah, happy whatever to all, as well.
Even the trolls need some cheer in their lives. 🙂
30
Frank,
You hit the nail squarley on the head.
Robert’s post could have more correctly been written as:
‘It happens regularly in Pro-Global Warming journals with great reward so the unscrupulous use that practice as a precedence for the others as well – as evidenced in recent medical journals’.
yours, etc,etc,
Frank’s Conscience
61
Sorry, but I don’t believe Frank has a conscience.
….although he has got sucked in by the Con-Science.
50
Frank that’s all we hear from you lot over and over ‘cherry pick, peer review, the models say, 97% blah blah fu#$ing blah..’
I’ll take any pro CAGW scientist seriously when they 1/ Stop relying on models as evidence 2/ Stop throwing ad hominem’s at other scientists when they don’t agree on a professional level 3/ Stop making excuses on why their hypothesis has failed 4/ Never use the word ‘consensus’ when discussing science ever again.
Seems pretty simple eh? then again a fool can turn a simple task into a difficult one without much effort.
72
Yonnie,
I didn’t think there was an $ in fu#king…
20
You obviously don’t frequent the wrong clubs.
40
Say, Rereke,
How’s it going down at the Sheepman’s Association anyways?
11
I wouldn’t know. That is an Australian Association, and they like to keep themselves to themselves, if you get the point, and understand the ramifications.
10
It always seems to cost me some $ before everything is said and done… 😛
10
Well the $ do come after the # pound, your telling the story Farnk. 😉
20
This is why I am an independent engineer creating things that work and am able to demonstrate that they actually do work. Even then, there was one instance of an organizer of a major conference on my subject matter area who held that my demonstration had to be faked. This even though he submitted a sample of his choosing and my process, demonstrated in real time in front of him found what he had hidden in his sample.
It worked as we claimed but he declared that it could not possibly work. He had a vested interest in holding that it could not possibly work because his entire career depended upon countless papers showing that it could not. He was not up to the emotional and intellectual challenge of being proved wrong.
Science? No, what passes for science these days is a medieval closed shop guild. Outsiders or unexpected breakthroughs will not be allowed to pass. That is unless it comes from the approved big names and, even then, it must be true to the guild position.
However, since my process can be demonstrated to work better, faster, cheaper than current methods and be portable as well, the profit motive comes to the fore. My process is now in economic play and is proving its value.
One example of that proof is that an archeological sample was examined by my process and found in less than a day what took six months for traditional methods. That’s a 150 to 1 improvement in efficiency. THAT gets people’s attention such that it overcomes the not invented here bias. When there is real money to be made, such biases can be overcome.
The take away: Do the science in the background. Turn the science into a valuable product. Sell and profit. That totally bypasses the medieval closed shop science guild. Which would you rather have, a growing bank account or a growing pile of publications and citations? I made my choice a long time ago and told the science guild to go to stuff it.
231
Sounds like another “model”. And like most of the models in this genre, it filters out the extremes, which means the bad, as well as the very good.
There is nothing that replaces the human discretion. And when you refuse to admit your own bias, you have eliminated from your repertoire as well. Nature is merely guilty of confirmation bias, and their record, by their own admission, is no better than a monkey score.
20
This article today is a good indicator of the breadth of the problem:
Settled Science: Liberals Are Ruining the Social Sciences
So with such an overwhelming bias towards one tribal group you can see why peer review would likewise be completely biased. Climate science also leans extremely heavily towards progressive politics.
There has to be a shakeup in both academia and journals to free up science from this monoculture, before science itself is lost for generations.
120
US Liberals are the socialists. Australian Liberals are hopefully conservatives, (some of the time)
31
handjive peer-review:
SMH 24 December, 2014
Short lived carbon price a winner
“Environment Minister Greg Hunt has quietly published data, just two days before Christmas, showing the second year of operation of Australia’s carbon price was more successful at reducing emissions than the first.
New data from Australia’s National Greenhouse Gas Inventory show emissions declined across Australia by 1.4 per cent over the 12 months to June.”
http://www.smh.com.au/environment/climate-change/new-data-shows-record-fall-in-carbon-emissions-20141223-12d1z3.html
. . .
Does a drop in carbon(sic) emissions stop catastrophic global warming?
(That is the purpose of a carbon(sic) tax/direct action)
2013: the Angry Summer
Hottest Year Ever
2014: Hottest Year Ever
2015 to be hottest year ever
Conclusion:
Observations show that a carbon(sic) tax that reduces emissions does not – cannot stop doomsday climate change.
Reviews, peers et al, are welcome.
Please include link.
81
In the not too near future it will be several more cuts in the death of the resource industry.
The money grabbers & carpet baggers & smart lawyers still hold sway.
Soon to be Extremely Poor Fellow My Country:-
http://www.rba.gov.au/chart-pack/balance-payments.html
read it & weep.
30
Just imagine peer review applied to this blog. You would need four likes before being published, but one dislike would mean the submission was rejected. How to stop anything that doesn’t fit the consensus in one easy step.
Jo, as you said, “being in with the herd affects your financial rewards.” Perhaps we should start calling the warmistas the herd, or even the flockers? Cows and sheep endorse consensus:-)
50
Flockers ? I like that one !
And some of them are females with young …
30
I was talking to a shepherd once, when suddenly we were hit by a cloudburst. The rain was torrential, and the lightening quite frightening.
I had no idea what to do, until the shepherd took control and said, “Rereke, lets get the flock out of here!”
40
In the middle of this year,2014, the ham radio club that I’m a member of had club member (also a ham operator and employed by the CSIRO) gave a talk on climate change . His opening remarks were all about peer review as if that gave authority to the rest of his talk. The ‘alarmist’ protagonists ALWAYS use peer (pal) review as their proof.
O/T Are any of you guys chemists and members of the RACI?
R-COO- K+
30
Ooops a couple of typos there. Sorry.
10
Yes (retired) and no.
10
I was tempted to ask if the talk was delivered in Morse code, but restrained myself.
20
I always thought that peer-review was to check that the paper was fit to put forward for scientific discussion.
It is not under any circumstances a guide to correct or incorrect.
Just another piece of the puzzle.
If the methodology, data, code etc are not immediately available, the paper cannot be discussed, and should be withdrawn.
81
ps.. The blocking of “contrary” papers to the AGW meme is the blocking of scientific discussion.
That is why anti-AGW climate blogs have become so popular, they allow discussion.
It is also why trolls often frequent these blogs. They are trying to block discussion.
You can tell that by the lack of substance in their comments.
92
“peer review….effective at blocking the controversial ones”
I would suggest the reason for this is largely about protecting social power and control. People protect their social position, and a controversial paper might challenge this.
A free market is a little better at dealing with this problem, in that a business (in theory) cannot stop someone else from putting a new revolutionary product or idea on the market. The market itself then decides whether it is useful, or accurate, or better than before, or whatever.
But in science, what sometimes happens is that those who have cornered ‘the market’ so to speak(in this case what gets put in the public domain) make the decision on whether or not something is useful, or accurate, or better, or whatever, not the broader public market, as such.
In other words, science is not really a consumer product, it is a more what could be described as a ‘bureaucratic product’ (although there is at least some market dimension to it-take for example dinosaurs and their popularity, there wouldn’t be much ‘dinosaur research’ if they weren’t so popular).
Making science more of a market-oriented or democratic process (such as occurs through blogging etc) might have some advantages, but there would be some disadvantages as well, (such as people wanting their ‘science’ to tell them what they want to hear, such as their future love life-like astrology etc etc. People would inevitably trade accuracy for entertainment).
20
Exactly WHY are we getting so many iterative papers ? so much dross ?
it was not always so.
That is the root of the problem imho. A lack of quality, a blizzard of poor thinking and cagw is not the only area
40
The problem not addressed here is the absurd warmist concept that if the conclusions of a paper are acceptable to some peers, mere publication means the conclusions are right. It doesn’t and that is not what review means. Papers in journals serve to inform researchers of developments and ideas in a field, not to lay down laws. So the idea that the science is ‘in’, a phrase never heard before, is just offensive to logic and science and research. The fact that people might agree also does not make conclusions right.
No, peer review simply means publication of the results and conclusions and even opinions and hypotheses from research after a precautionary reading of the papers by peers. Publication itself does not mean reviewers endorse those conclusions as necessarily right and in fact reviewers should not even pass judgement on conclusions unless they are clearly unjustified from the data. The peer review process is to trap plagiarism, non novel papers, unreasonable conclusions but the general rule if a scientist is confident is to publish and be damned. Possibly. That was certainly the truth in Charles Darwin’s case and presumably since. Science is established only when models fit data very well and can generate accurate conclusions. In the case of Global Warming, not even the Warming is right, so the models to date are useless. It is even hard to know what people mean by Climate Change or what it has to do with CO2.
71
Hannam’s Christmas Eve “No Joy to the World” opus – how much is peer-reviewed? how much is accurate? how much is fortune-telling?
24 Dec: SMH: Peter Hannam: As shepherds watched, it got hotter and hotter
Human health – and that of other animals and even plants – is likely to become an ever more pressing public issue as temperatures rise with global warming, cities grow and populations age.
(HUH!) If Australia’s test cricketers suffer heat stress during the Boxing Day Test in Melbourne or January’s match at the Sydney Cricket Ground, it won’t be for want of trying…
On the other hand, there are reasons why the scenarios, grim as they are for heat, may be optimistic not least because they did not account for population growth…
Brent Jacobs, research director of the Institute of Sustainable Futures at the University of Technology, Sydney, is among those examining the heat mapping to help planners limit current and future effects of the warmth.
Dr Jacobs says developers of new housing estates usually leave little in the way of green spaces and water features that – while costly to maintain – would help counter the inevitable ***heat-island effects…
Cricket Australia dietitian Michelle Cort, meanwhile, says people who are overweight or particularly muscular are among those who should pay special heed to warnings about heatwaves…
http://www.smh.com.au/environment/climate-change/as-shepherds-watched-it-got-hotter-and-hotter-20141223-124o27.html
11
on the other hand, Greenpeace’s Sauven is full of joy!
23 Dec: Guardian: John Sauven: 2015: the beginning of the end for climate sceptics
(John Sauven is executive director of Greenpeace UK)
Now an agreement has been struck between China and the US on carbon emissions, we can be optimistic about the coming year
2014 was the year when climate diplomacy got back on track. It’s a steep track, and we’re moving way too slowly, but as it’s the season of goodwill let’s accentuate the positive
Aligned with this are progressive moves from some major figures in the worlds of business and local and regional government, who are making commitments to 100% renewable energy…
The climate denial lobby and its fossil fuel funders will be even more focused in the coming months. For us this might just be the end of the beginning. For them it’s the beginning of the end…
Once the world starts setting hard limits on emissions, all the business plans of all the oil majors in the world become obsolete, wishful thinking and the global economy starts adjusting to a low-carbon future in earnest. From that day forward fossil fuels go into permanent retreat..
With trillions of dollars at stake, don’t expect a clean fight. Happy new year.
http://www.theguardian.com/sustainable-business/2014/dec/23/2015-climate-sceptics-us-china-agreement-carbon-emissions
reality bites. the following is in the middle of a global economic downturn!
22 Dec: Reuters: Alister Doyle: Top firms’ greenhouse gas emissions rise, despite call for cuts
Greenhouse gas emissions by the world’s top 500 companies rose 3.1 percent from 2010 to 2013, far off the cuts urged by the United Nations to limit global warming, a study showed on Monday.
The top 500 firms by capitalisation accounted for 13.8 percent of world greenhouse gas emissions and 28 percent of gross domestic product in 2013, according to the report, drawn up by the information provider Thomson Reuters and BSD Consulting, a global sustainability consultancy…
http://www.reuters.com/article/2014/12/22/climatechange-companies-idUSL6N0U622A20141222
11
hilarious Parenti piece, in which he puts what Obama really does in brackets for some reason, but then gives his wish-list of what he might do. gullible what?
22 Dec: HuffPo: Force Big Government to Kill Big Carbon
by Christian Parenti, Contributing Editor, The Nation
(Christian Parenti teaches at New York University in the Global Liberal Studies program. He is the author of four books, the most recent being “Tropic of Chaos: Climate Change and the New Geography of Violence” )
Imagine putting a quarter, to one third, of all known U.S. fossil fuel reserves beyond the reach of Big Carbon…
(Alas, Obama usually does the opposite. In 2013, the Administration, via the Interior Department’s Bureau of Land Management, offered up 5.7 million acres for lease to industry. The Interior Department also sped up the permitting process for drilling and opened an additional 59 million acres for oil and gas drilling offshore in the Gulf of Mexico. And, the BLM approved more than 800,000 acres for extra-filthy tar sands and oil shale development in the Green River Formation, a vast stretch of terrain in Utah, Wyoming and Colorado that contains 2 to 7 times more energy and pollution than the Alberta Tar Sands. All very, very bad.)
A second, more difficult action would be to cancel existing leases whenever there can be found sufficient technical, financial, or environmental problems…
The third thing Obama could do is go after producing leases, which can be cancelled for violations of law, regulation, or lease terms, but only after a judicial proceeding…
If there is one thing we know about Obama it is that he is vain, wants to be loved by everyone, and absolutely hates criticism from the left. Perhaps that’s why he seems to respond to it…
???Clearly, the president did not like having Michael Brune, Bill McKibben, and scores of other high profile figures arrested at the end of his driveway anymore than he like tens of thousands of activists — many of them veterans of the 2008 Obama canvas — condemning him personally for selling out his daughters’ futures…
The climate science is very clear: We do not have many years left to avoid the worst of runaway climate change, the movement’s ultimate short to medium term goal must be closing the fossil fuel industry. What force, what mechanisms, which institutions could actually do this? Does anyone really imagine that the fossil fuel industry can be convinced to change by way of smart arguments, or shamed out of existence, or tricked into believing there is a carbon bubble by way of spin and headlines, or even starved of investment capital?
Let’s be as radical as reality itself. Ultimately, only Big Government, (if forced to by the people) will be strong enough to subdue and euthanize Big Carbon.
http://www.huffingtonpost.com/christian-parenti/force-big-government-to-k_b_6368962.html?utm_hp_ref=green&ir=Green
11
Earlier this year from Tony Thomas: Climate Papers Without Peer
30
via Tallbloke, the producer of Lew-papers (some already retracted) said in an interview:
Let’s hold up a mirror to Stephan Lewandowsky and wait for him to open his eyes.
70
I suppose ya gotta laugh or bloody cry. The EIA numbers show just 0.4% of global energy comes from useless solar and wind and they forecast that by 2040 just 2.2% of global energy will come from these two green????? technologies.
WHOOOPPPEEE and by that time co2 emissions will have soared by another 13.5 billion tonnes per year to about a total of 45 billion Ts P year.
Mostly from China India etc( non OECD) while the evil OECD countries emissions will nearly flat-line until 2040. This idiotic nonsense will cost us endless 100s of billions $ for SFA return on the investment and zip change for co2 emissions or temp and climate.
http://www.theglobeandmail.com/globe-debate/why-innovation-is-the-best-path-to-a-climate-solution/article22100934/
30
Philip Campbell,
Editor-in-chief of Nature said:
Nature editor actually believes what he writes
How can papers be accessed with this level of blindness & past IPCC history?
JUST rubber stamp to Climate Alarmists
“Children just won’t know what snow is?”
“The rains that fall won’t be enough to fill our rivers & dams?”
The whole CAGW scam is dissolving fast, and the Liars, Cheats & Parasites will all be exposed
71
I think the research into cites of papers comes up one step short. What if, for example, the preponderance of cites are from papers written by fringe elements such as Dragon Slayers? The presumption is the number of cites and not where or how they’re used is an indicator of quality in a controversial paper. It may only be the controversial paper is being cited as a bad example. More is needed.
01
December 09, 2014:
Rolling Blackouts Taking Toll on S. African Economy
http://www.voanews.com/content/rolling-blackouts-taking-toll-on-south-african-economy/2551891.html
00
not sure how 30 million becomes 53 million, but the researchers apparently think that, with a little tinkering, & some more research, it might work out ok.
19 Dec: University of Essex: Is the Government’s multi-billion-pound smart meter roll-out a waste of energy… and money?
Government initiatives to reduce our energy use by installing 30 million smart meters by 2020 may do little to substantially cut our fuel bills, according to new research from Essex.
Smart meters with in-home displays have been growing in popularity globally as a key weapon in the fight against our increasing energy use and carbon emissions.
However, after closely examining research, the team at Essex have found the meters may only have a limited effect on reducing energy consumption in homes.
“The success of these devices is entirely dependent on the consumers being interested in them and engaging with them,” explained Kathryn Buchanan, from the Department of Psychology, who carried out the research. “It may seem obvious, but it seems this ‘human factor’ has been largely overlooked by policymakers.”
The study, published in Energy Policy journal, questions the effectiveness of multi-billion-pound plans to replace 53 million gas and electricity meters in 30 million UK homes and small businesses by 2020…
Analysing research data from 524,479 people in 156 field trials, the researchers found that whilst in-home displays were a good way of raising awareness of energy use and linking consumption with cost, customers only saved on average about 2% – which on an average home energy bill of £1,284 would mean only £2-3 savings per month…
Worryingly, the third concern was that for some households the in-home displays could trigger unintended consequences leading to increased energy use after consumers realised the cost of certain behaviour – ie taking a longer shower – would not cost them as much as they thought…ETC
http://www.essex.ac.uk/news/event.aspx?e_id=7227
Feb 2015: Energy Policy: Science Direct: The question of energy reduction: The problem(s) with feedback
Open Access
Abstract
With smart metering initiatives gaining increasing global popularity, the present paper seeks to challenge the increasingly entrenched view that providing householders with feedback about their energy usage, via an in-home-display, will lead them to substantially reduce their energy consumption…
4.1. Dangerously cold homes?
Due to a scarcity of research, it is uncertain how households suffering from fuel poverty will react to feedback…
5. Conclusion and policy implications
In the UK, current government policy requires energy suppliers to install 53 million smart meters in domestic homes by 2020 …
As such, we propose that there is a clear need to reconsider whether investing billions of pounds in providing consumers with the proposed IHDs is a worthwhile endeavour, especially given that they (a) may not benefit consumers and (b) are likely to become outdated fast as, unlike other platforms (e.g., smart phone applications, websites), they cannot be readily updated…
Consequently, we suggest that the challenge is to develop and rigorously test innovative feedback mechanisms that actually engage consumers and take into account the potential shortcomings that we have outlined…
We propose that with these elements in place, there may be some hope that the domestic energy demand reduction aspirations of world-wide smart metering initiatives may be met.
http://www.sciencedirect.com/science/article/pii/S0301421514006739
21
World wide eh? So who was the bright one that figured out how to hook a smart meter up to a pile of burning dung in Africa? I want to see the spec sheet and wiring diagram for that install.
I’m not convinced most of Africa or most “developing” countries have any “domestic energy demand reduction aspirations.” So it’s not really world wide then is it? /semi-sarc
20
They lie about what smart meters are actually supposed to do. What they neglect to say is Smart Meters allow residential electricity to be turned off selectively so the system can be balanced.
Energy InSight FAQs
As the likelihood of an unstable grid increases due to solar/wind coupled with a decrease of reliable base load coal power, expect smart meters to become mandatory and rolling blackouts of residential power to be come common.
00
Smart meters also, and more importantly, allow utilities to bill on different scales according to the time of day electricity is used.
10
97% consensus settled science?
Like Linus waiting for the return of the Big Pumpkin … except more pathetic:
“We also know that the Pacific flips and flops between states with weak and strong winds,
so it’s just a matter of time before the winds weaken once again.
“Global warming will likely accelerate when Pacific trade winds weaken.
Scientists can’t say exactly when this switch will happen, Thompson adds:”
http://www.carbonbrief.org/blog/2014/12/pacific-winds-change-the-speed-of-global-warming-says-new-study/
10
Proposition: Before serious consideration is given to an explanation (in scientific terms for observations), ask the question,
” But what does it un-explain? ”
[ Cannot recall where this comes from, it may have been stage magician James Randi or Carl Sagan ]
In the (in)famous MM peer reviewed hockey stick paper, the well known and accepted medieval warm period, and the mini ice-age, had disappeared. This wiping out of accepted knowledge on the past one thousand years of history appeared to be of no importance to the reviewers. Instead of trying to find supporting evidence for the interpretation of rings from just one tree, an entire industry devoted itself to covering up the contradictory evidence.
One example of the peer review process failing, dramatically.
40
Peer review a shambles.
http://progcontra.blogspot.co.uk/2014/12/climate-alarmist-paper-retracted-due-to.html
10
The Wall Street Gerbil published an article this summer on peer-review and science fraud.
The Corruption of Peer Review Is Harming Scientific Credibility —Dubious studies on the danger of hurricane names may be laughable. But bad science can cause bad policy.
Unfortunately it can not be read without subscription. However there is a good review of the article.
Peer Review, Corruption, Junk Science, and Advocacy Science
There are three rather interesting points to be taken from those two articles.
First, The WSJ is a business newspaper yet it is carrying this story that would normally only be of interest to academics and scientists.
Second the story is taken up not by a science blog but a Criminal Justice Blog of all things.
Third is the revelations of exactly what The Proceedings of the National Academy of Sciences is up to when it comes to ‘peer-review’
What is the National Academy of Sciences and why is a Legal blog concerned?
In other words the idiots who actually published papers in the Proceedings of the National Academy of Sciences. such as “… “Female hurricanes are deadlier than male hurricanes,” –it concluded that hurricanes with female names cause more deaths than male-named hurricanes–ostensibly because implicit sexism makes people take the storms with a woman’s name less seriously….” are the same idiots that are ‘advising’ the US Government. (Slams head against brick wall.)
The bright spot is they are becoming so laughable even the lawyers and business types are taking notice.
The pedestal that Academia takes for granted is crumbling.
I am reminded of the origin of the name of Verity Jones blog, Digging in the Clay
Seems the digging with a lot of help from academics be-clowning themselves is starting to pay off.
10
I am not surprised by this at all. Science is hard work, (even more so on Thursday) and no one likes the thought that their ideas are wrong.
‘The Hitch-Hiker’s Guide to the Galaxy’ by Douglas Adams.
If human endeavour occupies a bell curve, with the majority of expertise in the centre then both extremes are naturally weeded out by the peer review process (the infamous consensus of settled science).
My view is that the work of mavericks should always be considered, not for the quality of the ideas, but for the range of the data and references that the maverick often collects to support their unusual contentions. Data is data and if it is correctly collected (a huge quality issue in itself) then the ideas can assessed in context, discounted when necessary and hopefully the new paradigm recognised and accepted.
40
http://www.ctvnews.ca/sci-tech/chinook-salmon-could-be-wiped-out-by-2100-study-1.2158718#commentsForm-592033
Just to add to the fun from the silly masses:
“Chinook salmon could be wiped out by 2100: study
The Canadian Press
Published Monday, December 22, 2014 1:30PM EST
VANCOUVER — New climate-change research involving a University of British Columbia scientist predicts that one of the West Coast’s most prized salmon stocks could be wiped out over the next 85 years.
A study has concluded that there is a five per cent chance of a catastrophic loss of the chinook salmon by 2075, and that upwards of 98 per cent will be gone by 2100, if climate change warms the water.
An international research team looked at the ability by the chinook to adapt to warming water temperatures caused by climate change.
UBC zoologist Anthony Farrell was part of the research group and says the juvenile salmon studied developed serious heart problems in water temperatures higher than 24.5 C.
Once past that temperature, the study found that the heart couldn’t go any faster and would either slow or go arrhythmic.
The study was recently published in the journal Nature Climate Change.”
Of course, this “study” ignores the fact that any fish tossed into the equivalent of the backyard hot tub will be in trouble. Also ignores the fact that turning the entire North Pacific into that hot tub will take the near equivalent of the sun going nova so don’t worry too much about it.
00
[…] http://joannenova.com.au/2014/12/nature-admits-peer-review-filters-out-controversial-champion-papers… […]
00