I really like Allie Brosh’s Alot. I think there’s a dash of bantha in him.
My response to “Something’s Rotten in Bethesda — The Troubling Tale of PubMed Central, PubMed, and eLife”
Text is from http://scholarlykitchen.sspnet.org/2012/10/22/somethings-rotten-in-bethesda-the-troubling-tale-of-pubmed-central-pubmed-and-elife/ by Kent Anderson, Oct 22, 2012. My comments are capitalized and in bold (I’m not yelling, just trying to distinguish between the original text and my response).
PubMed Central (PMC) says it’s not a publisher. PubMed claims to have a fair process that it applies uniformly to include journals in its index. [NO, THAT’S MEDLINE. MEDLINE IS A SUBSET OF THE PUBMED DATABASE] And as a government agency, PMC is generally prohibited from competing with one or more thriving private industries.
Yet, last week, with a set of actions around the fledgling funder-backed journal eLife, it became clear that PMC is a publisher, that PubMed’s process for indexing journals [AGAIN, YOU’RE CONFUSING PUBMED WITH MEDLINE] is not uniformly or fairly applied and can be exploited, and that PMC competes with both publishers and publishing technology companies, leveraging PubMed itself in these competitions.
Let’s break this down claim by claim.
PubMed Central As Publisher
PMC states definitively on its site that it is not a publisher:
PMC, itself, is not a publisher.
What is a publisher? An entity which engages in:
… the activity of making information available to general public. [SELECTIVELY QUOTING WIKIPEDIA AS YOUR SOURCE FOR A DEFINITION IS VERY SUSPECT. HERE’S THE FULL QUOTE: “Publishing is the process of production and dissemination of literature, music, or information”]
PubMed has long been a publisher of abstracts. [PUBMED DOESN’T CREATE THE ABSTRACTS] PMC is clearly a publisher of full-text content. [PMC DOESN’T CREATE THE CONTENT] Therefore, their contention that PMC is not a publisher is simply wrong.
However, there are degrees of publishing — secondary, embargoed publication is qualitatively different than primary, immediate publication. A downstream publisher is different from a primary publisher. PMC is widely understood to be a secondary, embargoed publisher. [PMC IS A DATABASE, JUST LIKE OVID HAS A DATABASE OF FULL-TEXT ARTICLES, AS DOES EBSCO, AS DO OTHERS. JOURNALS SIGN AGREEMENTS TO ALLOW THEIR CONTENT TO BE USED BY THESE DATABASES]
But what if it’s becoming a primary publisher?
PMC and PubMed [MEDLINE, NOT PUBMED] require publishers to establish independent editorial and publishing practices and have these reviewed and evaluated after they’ve been in effect for some time. This is how it’s described in PubMed and PMC documentation, and this what the community generally believes to be true and adheres to. For instance, getting content into PMC requires publishers to meet both technical and scientific standards, including:
The journal must have a reasonable number of published articles in order for NLM to make a decision about its scientific quality. [ACCORDING TO www.ncbi.nlm.nih.gov/pmc/pub/pubinfo/ “Smaller samples may be evaluated at PMC’s discretion.”]
A journal will be deemed to be eligible for inclusion in PMC if the NLM Selection and Acquisition Section determines that it conforms to the scientific quality criteria specified in the Collection Development Manual of the National Library of Medicine. [I REFER YOU TO http://www.nlm.nih.gov/tsd/acquisitions/cdm/formats29.html#1027134 “The Library’s first priority is collecting scientific or scholarly journals containing signed papers that report original research. The intent is to assemble a comprehensive collection of the world’s most significant research journals in all subjects collected at NLM, from all countries and in any language.” AND TO: http://www.nlm.nih.gov/tsd/acquisitions/cdm/policy.html “ Coverage of the scholarly biomedical literature shall be comprehensive; coverage of other biomedical literature may be more selective. The intent is to ensure that the collection represents the intellectual content and diversity of the world’s biomedical literature.”]
In standard communications with publishers about these protocols, the NLM and PubMed state:
A journal needs to be included in the NLM Collection to meet PMC’s scientific quality standard. This review can take place once [the title] has published 15 articles.
[DO YOU HAVE A SOURCE FOR THIS? I HAVEN’T SEEN THIS NUMBER]
So, a journal has to publish 15 articles before any review can take place.
But what would it mean if a journal’s first articles were published on PMC, and only on PMC? No review process could have occurred. There would have been no independent publication practices established, and therefore nothing to review. [REVIEW PROCESS FOR ELIFE http://www.elifesciences.org/the-journal/review-process/ BOARD OF REVIEWING EDITORS http://www.elifesciences.org/about/elife-community/reviewing-editors/ ] Allowing articles into PMC otherwise would be an abrogation of standards, or something equally questionable. It would be even more of a violation of stated policy if fewer than 15 articles were involved. [AGAIN, WHERE’S THIS POLICY?]
Last week, eLife published its first papers — four research articles, four commentaries, and an editorial. That’s a total of nine articles, short of the 15 required, and the minority are scientific reports. Yet, where did eLife publish them? On eLifesciences.org, their domain? On HighWire Press, where its publisher site is being built? No. It published them on PubMed Central. All URLs listed on the eLife site point to this US government Web site (http://www.ncbi.nlm.nih.gov/pmc). In a comment on this blog last week, Robert Kiley of Wellcome Trust pointed to the only published version of eLife’s introductory editorial — which resides on PMC. The US government is acting as the online publisher of eLife.
eLife branding is prominently displayed. [EVERY JOURNAL IN PMC HAS A BRANDED HEADER. SEE http://www.ncbi.nlm.nih.gov/pmc/issues/206112/ http://www.ncbi.nlm.nih.gov/pmc/journals/1420/ http://www.ncbi.nlm.nih.gov/pmc/journals/990/ http://www.ncbi.nlm.nih.gov/pmc/journals/258/ ] Within the eLife logo, the four links work separately and well. This was not a sloppy job, but a careful integration.[ AS WITH EVERY SINGLE OTHER JOURNAL IN PMC] This required coordination and planning between eLife and PMC. PMC was clearly acting as an extension of eLife. [YOU’RE SAYING THAT PMC IS “ACTING AS AN EXTENSION” OF ALL JOURNALS IN PMC?]
Screen shot of how eLife looks on PMC.
eLife put a statement on its site about how PMC was publishing its articles:
To avoid unnecessary delays in making the first accepted articles available, we are listing articles that are ready for publication here and publishing the articles themselves on PubMed Central (PMC), the public archive for life and biomedical science literature at the US National Library of Medicine. Additional papers will be published this fall and, as planned, we will launch eLife’s journal Web site this Winter.
"Unnecessary delays" would be those we all tolerate as independent publishers in our dealings with PubMed and PMC — waiting for our hosting platform to be ready, [NOT A REQUIREMENT] proving our independent ability to publish by abiding by the timeframe, [WHAT TIMEFRAME?] article count, [WHAT ARTICLE COUNT?] and technical requirements [IF ELIFE DIDN’T MEET THE TECHNICAL REQUIREMENTS, IT WOULD LITERALLY BE IMPOSSIBLE FOR THEIR ARTICLES TO APPEAR IN PMC] of PubMed or PMC, and so forth. eLife editors are being disingenuous by saying these things are “unnecessary.” They are required, [MEDLINE HAS STRICT REQUIREMENTS BUT ELIFE IS NOT INDEXED BY MEDLINE] and they are the norm. The are only “unnecessary” if someone helps you short-circuit the process.
The agency of the statement is also interesting — “we” (meaning eLife) are publishing the articles themselves on PMC. This assignment of agency to eLife makes it seem that PMC had no choice in the matter, [HOW ARE THEY SUPPOSED TO PHRASE IT?] and that the US government, through the NLM and NCBI, is merely a publicly available and free publishing platform, ala WordPress.
Where was the review? Where were the standards? Where was the process?
I spoke with David Lipman of NCBI about this. He told me that eLife submitted a PMC application following what he termed “the regular procedure.” During the application process, eLife’s representatives mentioned that their site was not up. A discussion ensued, the details of which he claimed to know nothing about, and the powers that be at PMC or NLM agreed to publish the eLife articles on PMC before the normal requirements had been met, and before eLife had demonstrated any independent ability to publish. I asked him if the NLM Acquisition and Selection Section had approved the application. He said they had. I tried to contact Joyce Backus, the head of the Section, over the weekend prior to the publication of this post, but we weren’t able to connect. If warranted, I’ll write more about the questions I asked her later.
Lipman claimed that eLife “seemed like a reasonable bet” owing to the quality of its backers and its editorial staff. Yet, when asked why a publisher like AAAS or the American Chemical Society or Lippincott or a hundred others don’t benefit from the same affordances [THEY ALL HAVE THE SAME OPPORTUNITY TO SUBMIT TO PMC. HERE’S AN ACS JOURNAL: http://www.ncbi.nlm.nih.gov/pmc/journals/1791/ AND LIPPINCOTT USED TO PARTICIPATE: http://www.ncbi.nlm.nih.gov/pmc/journals/230/ AND AAAS HAS JOURNALS INDEXED IN MEDLINE; IT COULD CHOOSE TO PARTICIPATE IN PMC TOO] and don’t by extension seem like “a reasonable bet,” he had no good answer.
Clearly, PMC is the primary publisher of eLife. [PMC DOES NOT CREATE THE ARTICLES] Articles in eLife appear nowhere else — not in print, not on a publishing platform through independent private means, nowhere. The editors of eLife are representing that it was their doing, [REASONABLE PEOPLE WOULD ASSUME THAT ELIFE CHOSE TO APPLY FOR PARTICIPATION IN PMC. OF COURSE YOU NEED PMC’S ASSISTANCE] but that’s clearly not true — they needed agreement and assistance from PMC, which is owned by an agency of the US government.
The implications of this are troubling. US taxpayers are now directly subsidizing [US TAXPAYERS ARE SUPPORTING NLM. SO WHAT? ARE YOU ARGUING THAT PMC IS DIRECTLY SUBSIDIZING EVERY PARTICIPATING JOURNAL?] a Delaware-based 501(c)3 that is also partially a UK-based charity. NLM, PMC, and NCBI have given one entity special accommodation [I REFER YOU TO THE POLICIES AT http://www.ncbi.nlm.nih.gov/pmc/pub/pubinfo/ ] and violated its own policies and practices to make this happen.
As if this weren’t enough, PMC’s role as a primary publisher [A PUBLISHER SELECTS, PREPARES, AND DISSEMINATES MATERIAL. PMC DOES NOT RUN AN ARTICLE SUBMISSION PROCESS, DOES NOT EDIT THE ARTICLES, AND DOES NOT PROVIDE THE XML DATA COMPRISING THE PUBLISHED ARTICLE] doesn’t stop with eLife. The online version of the Journal of the Medical Library Association (JMLA) is also published primarily on PMC. [JMLA CHOSE TO MAKE ITS CONTENT FREELY AVAILABLE VIA PMC (WITH PMC’S AGREEMENT AND ASSISTANCE, OF COURSE).] PMC also publishes the Journal of Biomolecular Techniques for the Association of Biomolecular Resource Facilities (ABRF). The PMC site for this journal is framed in the ABRF site.
In much the same way that any commercial or non-profit publisher is the publisher of a society’s journal, PMC is a primary publisher — for eLife, and for at least two other journals. [WHO IS THE PUBLISHER OF A JOURNAL? THE ENTITY SELECTING AND PREPARING THE MATERIAL? THE PRINTER OF THE HARD COPY? THE PROVIDER OF THE WEBSITE? THE PUBLISHER OF THE ANNALS OF FAMILY MEDICINE IS THE ANNALS OF FAMILY MEDICINE, INCORPORATED. IT IS NOT AAFP (WHICH RUNS THE PRODUCTION), NOR MODERN LITHO, NOR ALLEN PRESS. NOT EVEN HIGHWIRE PRESS IS THE PUBLISHER. HIGHWIRE PRODUCES ONLINE VERSIONS OF CONTENT AND “FACILITATE[S] THE DIGITAL DISSEMINATION” OF CONTENT. http://highwire.stanford.edu/about/]
PubMed’s Indexing Process Can Be Subverted [PUBMED DOES NOT EQUAL MEDLINE]
eLife published its first articles on October 15, 2012 — one week ago. Yet, search PubMed, and lo and behold, eLife articles are included in PubMed. The record in the NLM Catalog still states “PubMed Coverage to be announced” and “Not currently indexed for MEDLINE.” This distinction is functionally meaningless [WELL, THAT’S THEIR IGNORANCE. THERE ARE LOTS OF ITEMS LISTED IN PUBMED BUT NOT IN MEDLINE] to our readers, authors, and most of our editors. PubMed is what they use [WHO RELIES ONLY ON PUBMED? AUTHORS/RESEARCHERS HAD BETTER NOT BE USING PUBMED ALONE, OR ELSE THEY’RE DOING LOUSY SEARCHES. THERE’S A GOOD REASON WHY INSTITUTIONS PAY OVID, EBSCO, DIALOG, ETC, FOR ACCESS TO MEDLINE AND OTHER DATABASES], and PubMed is where they want to be indexed. [NO, THEY WANT TO BE INDEXED IN MEDLINE. INCLUSION IN PUBMED IS SECOND-BEST] So it’s no surprise that PubMed is what is being leveraged by PMC. [WHY WOULD NLM SEPARATE PMC AND PUBMED? IT MAKES NO SENSE FOR AN ENTITY TO HAVE ONE DATABASE FULL OF CONTENT AND NOT CONNECT IT TO ANOTHER ONE OF THEIR DATABASES. PUBMED ALSO LINKS USERS TO THE PUBLISHERS’ WEBSITES, TO LIBRARIES, AND TO OTHER DATABASES.]
The confusion between PubMed and MEDLINE, [WHICH YOU HAVE DEMONSTRATED QUITE WELL YOURSELF] along with how PMC can be used as a backdoor into PubMed, [PMC IS NOT A “BACKDOOR” INTO PUBMED] is what eLife is exploiting, [EXPLOITING? YOU MAKE IT SOUND LIKE A BAD THING TO PROVIDE PUBLIC ACCESS TO BIOMEDICAL RESEARCH] with the help of having PMC as its primary publisher. [AGAIN, I COMPLETELY DISAGREE WITH THE WAY YOU’RE THROWING AROUND THE WORD “PUBLISHER.”]
It takes more to get into MEDLINE. In the Collection Development Manual, the following statement appears:
Once an electronic journal has been accessible for at least six months, an editor or publisher may request that the journal be reviewed for possible indexing if at least 20 articles have been published and made available online.
This statement has a few aspects. The electronic journal must be accessible for at least six months — independently. Not published on PMC, but accessible through private means as evidence that it is sustainable and independent. For six months — not six hours or six minutes. By having PMC as its primary publisher, [NO AGAIN] eLife is subverting these requirements. [I REPEAT, ELIFE IS NOT INDEXED IN MEDLINE] They are not likely to launch as an independent journal — independent of US government publishing operations, [THE GPO ISN’T PUBLISHING ELIFE, IS IT?] that is — until early next year.
My organization began publishing two online-only journals more than a year ago. We followed the stated requirements, which are reiterated on the application itself. [YOU MEAN YOU APPLIED FOR INDEXING IN MEDLINE? YOU CONTINUE TO CONFLATE PUBMED, MEDLINE, AND PMC] We waited six months and 20 articles before submitting out applications, even a little longer to ensure that some technical issues could be worked out and some editorial roles assigned after the journals were launched. We are still waiting to be indexed in PubMed and MEDLINE. We followed the process. Dozens and dozens of other publishers do so, as well. [YES, AND THEY WAIT THEIR TURN FOR THE SELECTION COMMITTEE TO REVIEW THEIR JOURNALS FOR INDEXING IN MEDLINE. THE PROCESS IS COMPLETELY, UTTERLY DIFFERENT FOR PMC.]
Being indexed in PubMed [YOU DON’T GET “INDEXED” BY PUBMED. YOU MIGHT HAVE CITATIONS LISTED IN PUBMED, YOU MIGHT HAVE SELECTED ITEMS ADDED FOR INDEXING IN MEDLINE, YOU MIGHT HAVE YOUR FULL JOURNAL SELECTED FOR INDEXING IN MEDLINE] is a big deal for fledgling journals, because editors, authors, and readers often mistake PubMed for MEDLINE. Indexing in PubMed [IF SOMEONE CLAIMS TO BE INDEXED IN PUBMED WHEN THEY’RE NOT IN MEDLINE, THAT IS MISLEADING AND PEOPLE WILL CATCH ON. ANY DECENT PUBLISHER WOULD PROCLAIM INDEXING IN MEDLINE, NOT PUBMED] makes a difference in attracting editors, it makes a difference in attracting good papers, and it makes a difference in establishing legitimacy with an audience. Allowing certain journals to shortcut their way into PubMed [NO ONE SHORTCUTS THEIR WAY INTO PUBMED. THERE ARE PRACTICES FOR ADDING CITATIONS, WHICH ARE SELECTED BY NLM. EVEN JOURNALS INDEXED IN MEDLINE DON’T AUTOMATICALLY HAVE EVERY PUBLISHED ITEM ADDED TO MEDLINE OR EVEN PUBMED] in direct violation of its own stated processes and requirements is just wrong, but to leverage this to move the needle for PMC only compounds the problem. And to allow a new publisher to leverage PMC as eLife has goes one step further.
There was no one- to two-year wait for eLife to get into PubMed through MEDLINE because they got in immediately by being in PMC. [YEP, JUST LIKE ALL THOSE OTHER JOURNALS WHO WERE WILLING TO MAKE THEIR FULL-TEXT CONTENT AVAILABLE VIA PMC]. No process akin to what other publishers have to go through. [JUST ABOUT ANY BIOMEDICAL PUBLISHER CAN MAKE THEIR CONTENT FREELY AVAILABLE VIA PMC. THEY WANT YOUR FULL-TEXT CONTENT.] Immediate acceptance into PubMed. [NOT REALLY. IF YOU’RE NOT INDEXED BY MEDLINE, YOU CAN’T SEND YOUR CITATION DATA DIRECTLY TO THEM. PMC HAS TO TAKE ELIFE’S DATA AND SEND THE CITATIONS TO PUBMED. AGAIN, NOT MEDLINE.] Magically. Inexplicably. With the US government as its primary publisher. [THE PUBLISHER OF ELIFE IS ELIFE SCIENCES PUBLICATIONS, LTD]
Before I spoke with him, I emailed Lipman, head of NCBI, last week to get his perspective on this. Here’s what he said:
The NLM Selection and Acquisition Section considers a number of factors in deciding whether the scientific quality of a journal merits acceptance into PMC. The minimum number of articles required for this evaluation may vary based on the credentials of the publisher and/or sponsoring organization and of the editorial board, and on the quality of the journal’s editorial policies and practices. eLife citations appear in PubMed after the articles are released in PMC, which is no different from any other non-Medline journal in PMC.
In my conversation with him a couple of days later, Lipman referred again and again back to this email, as if it explained everything. [YES, IT DOES EXPLAIN EVERYTHING] Yet, when I quoted the NLM’s own stated standards [WHICH STANDARDS? YOU KEEP MIXING UP THE PMC STANDARDS WITH THOSE OF MEDLINE. HE PROBABLY COULDN’T FIGURE OUT HOW TO EXPLAIN IT MORE CLEARLY] back to him, the pregnant pause was notable. [MAYBE HE WAS TRYING TO BE POLITE AND NOT SNARKY] And, as mentioned above, he couldn’t explain why the “practices” and “policies” and “credentials” of eLife might be stronger than those of a hundred other, more established publishers. [SHOW ME THE ESTABLISHED BIOMEDICAL PUBLISHER WHO HAS BEEN DENIED ACCESS TO PMC.]
A matter of minutes after I received Lipman’s email quoted above, another publisher I was speaking with received a response from PMC to a general question about their policies and approach, to see if being open access (OA) made a difference to the process:
One of your new journals (it doesn’t matter if it is OA or not) will need to publish at least 15 articles before it can be evaluated by the Selections Group. Once it is approved by the Selections Group, then sample articles can be sent to PMC for data evaluation. When you’re ready to submit a new journal, please just send me the title, ISSN(s), and url.
So, again we have the 15-article requirement — for everyone, it seems, except eLife. [SEE http://www.ncbi.nlm.nih.gov/pmc/pub/pubinfo/ “The sample set must comprise approximately 50 articles and be representative of the variety of article types, styles and file formats found in the journal. Smaller samples may be evaluated at PMC’s discretion.” KEY WORD: DISCRETION]
Lipman’s response is coded and disingenuous because he says that things like “credentials,” “policies,” and “practices” inform their decisions about what gets this magical treatment, [I THINK THE WELLCOME TRUST, THE MAX PLANCK INSTITUTE, AND THE HOWARD HUGHES MEDICAL INSTITUTE QUALIFY AS HAVING GOOD “CREDENTIALS.”] but obviously the process as understood by people who work with the Selections Group is not the process Lipman or someone else he knows uses. [SNARK]
As I mentioned above, I asked him about this in our phone call. There was not a satisfactory answer. He said grand things like “we look at the big picture” and “we’ve been burned by publishers who have moved or stopped journals” [THERE ARE JOURNALS WHO AGREED TO MAKE THEIR FULL-TEXT CONTENT AVAILABLE ON PMC AND THEN BACKED OUT LATER. A DATABASE WITH RANDOM, SHORT SEQUENCES OF ARTICLES IS NOT VERY USEFUL. PMC HAS AN INTEREST IN ACCEPTING CONTENT FROM PUBLISHERS WHO ARE COMMITTED TO MAKING THEIR CONTENT AVAILABLE AND HAVE THE FUNDING TO STAY ALIVE] (how moving a journal affects PubMed is beyond me), and “it seemed like a reasonable bet” to take eLife early.
Let’s take his words at face value — would any traditional publisher lack in “credentials” or “policies” or “practices” around a new journal and quality? [I REFER YOU TO ELSEVIER’S Australasian Journal of Bone & Joint Medicine AS ONE EXAMPLE] Not very likely. In fact, if quality and reliability were the criteria guiding decision-making — and you might hope they would be — you’d expect a fast lane for established publishers, many of which have been around for more than a century, attract top-tier editors, and have advanced publishing infrastructure. [THEY’RE WELCOME TO APPLY TO PMC, BUT I GUESS THEY’RE AFRAID OF LOSING MONEY] How a journal like eLife — which hasn’t proven it can publish independently of US government subsidy, which hasn’t published the requisite number of articles, [AGAIN, I REFER YOU TO “Smaller samples may be evaluated at PMC’s discretion” http://www.ncbi.nlm.nih.gov/pmc/pub/pubinfo/ ] and which has yet to demonstrate solid editorial or business practices [THEY HAVE THEIR BOARD OF REVIEWING EDITORS http://www.elifesciences.org/about/elife-community/reviewing-editors/ , THEIR EDITORIAL LEADERSHIP http://www.elifesciences.org/about/elife-community/editorial-leadership/ , THE ARTICLES THEY SUBMITTED TO PMC, AND THEIR FUNDING FROM THREE OUTSTANDING NONPROFIT INSTITUTIONS] — gets a free pass suggests a strong ulterior motive is at work, something that may approach outright cronyism.
Lipman’s criteria are at worst euphemisms for a specific set of policies and practices, perhaps OA, as some people I spoke with suspect — yet, even other OA publishers don’t get this magic. [YES THEY DO. MAYBE NOT QUITE AS QUICKLY, I DON’T KNOW. BUT I SEE A LOT OF HIDAWI JOURNALS IN PMC. I HAVE RECEIVED HUNDREDS OF ANNOUNCEMENTS OF JOURNALS BEING ADDED TO PMC.] They have to wait, follow procedures, and prove they can publish independently. So this exception was even more specific — it was for eLife, and that’s very troubling.
In addition to being unfair and infuriating, this event also shows how PMC is part of the competitive landscape with a definite role in determining who wins and loses. It leverages PubMed in this regard. It takes an active and biased role in who benefits and who suffers. It is competitive. And this is our next topic.
[OKAY, I’M TIRED OF ARGUING AND PROBABLY WASTING MY TIME. MAYBE SOMEONE ELSE CAN CONTINUE MY ATTEMPT TO REFUTE THE INCORRECT AND MISLEADING STATEMENTS IN THE REMAINER OF KENT’S POST.]