Mr Anderson’s posting and his subsequent responses to comments misrepresent PLoS ONE in four main areas: the PLoS ONE editorial board, the editorial process; the content being published; and the overall goal of PLoS ONE.

1. The editorial board

During his post, Mr Anderson implied that the PLoS ONE Editorial Board (who are now 1,000 strong and as academic editors are responsible for the editorial decisions on PLoS ONE) are accepting sub-standard papers simply to make more money for PLoS. Several academic editors took justified exception to this in the comment thread and elsewhere. The tremendous success of PLoS ONE is in large part a result of the dedication and commitment of the PLoS ONE Editorial Board, and any implication that the PLoS ONE Board members are collectively doing a poor job of evaluating submissions is unjustified.

2. The editorial process

The PLoS ONE editorial and peer review process is very similar to the editorial process on any journal, and is clearly explained on the PLoS ONE web site. The critical difference, and the thing that makes PLoS ONE potentially revolutionary, is that the editors and peer reviewers make no judgment as to the potential impact of the work. Their goal is to focus on scientific rigour alone, although this is by no means an easy task.

Every submission goes through a very substantial quality control check before it reaches an academic editor. This check covers reporting standards, research ethics, competing interests, funding information and so on. We believe that our ‘QC check’ is one of the most extensive and rigorous processes in the industry, and it is overseen by professional editors who are frequently consulted in advance of further peer review. Submissions are then assigned to academic editors, who send the articles to external peer reviewers and take overall responsibility for the editorial decision on the submission assigned to them. Submissions are assessed specifically against the PLoS ONE editorial criteria. We also provide summary statistics on the peer review process and as repeatedly evidenced by published PLoS ONE authors, submissions typically receive excellent peer review comments and go through multiple revisions before ultimately being accepted. PLoS and the Academic Editors are committed to achieving very high standards in the editorial process.

3. The content

Mr Anderson’s post seems to be prompted by some concerns about a specific recent PLoS ONE article, which was critiqued by one of his co-editors on the Scholarly Kitchen blog. The irony here is that one of the goals of PLoS ONE is to open up this kind of dialogue post-publication, so that everyone can benefit from a more open discussion. What we are striving to do better is to capture more of this discussion and link it to the articles themselves.

Putting that one article aside, the implication of the post is that PLoS ONE content is substandard in some way, but evidence to support this claim is not provided. In fact, we do have evidence from usage, citation and media/blog coverage that the quality of content in PLoS ONE is extremely high, and would compare favourably with most journals. For example over 84% of the more than 2700 articles published in 2008 have been cited (from Scopus data), and 39% cited 4 times or more. These and other article-level metric data are publicly available (for all PLoS Journals) on our web site.

PLoS ONE has also been selected for coverage by all major indexing services, including Web of Science and Medline. A round-up of some of the most high-profile content published in PLoS ONE in 2009 was posted in January.

4. The goal of PLoS ONE

As mentioned above, the key innovation of PLoS ONE is that the peer review process involves no judgment about the potential impact of a submission. This is because our view is that the current system of sorting the almost 2 million articles that are published each year, into the existing set of 25,000 journals before publication, with all the delays and redundancy that this involves, is not the best way to organize all research findings in an online world.

Instead, PLoS ONE peer reviews submissions on the basis of scientific rigour, leaving the assessment of the value or significance of any particular article to the post-publication phase. We would fully agree that it is in the post-publication phase that PLoS has not yet achieved as much as we would like. Nevertheless, the addition of article-level metrics to all PLoS content last year is an important step in this direction and there will be further developments in this area.

PLoS ONE thus frees authors from a system of journals that is biased against publication, which means that authors can publish their work swiftly. We have been surprised and delighted that so many authors (now well over 60,000 of them) have supported PLoS ONE by sending us their submissions. As a result of this groundswell of support from the research community, the growth of PLoS ONE has been spectacular and, we believe, unparalleled in the history of scientific publishing (the journal is less than 4 years old and is on target to be the largest peer-reviewed journal in the world this year).

Given that the costs of publication are fully covered by publication fees, it is true that PLoS ONE has helped to move PLoS much closer towards independent economic sustainability. However, it is also the case that the PLoS Community Journals launched in 2005 (PLoS Computational Biology, PLoS Genetics, PLoS Pathogens) are fully economically sustainable through publication fees and are now making a positive financial contribution to our organisation. There is no question that the financial impact of PLoS ONE is important for PLoS and for open-access publishing more broadly, but to focus purely on this aspect of PLoS ONE is to miss the real significance of PLoS ONE, which was established as an innovative publication vehicle aiming to dramatically improve the pace and efficiency of scientific communication relative to the established order of academic journals.

This significance was formally recognized last year, when PLoS ONE received a major industry award from ALPSP for publishing innovation. The judges indicated that PLoS ONE “combines the traditional values of the journal with innovative online features to create an inclusive and efficient publication channel. It is bold and successful and shaping the future of publishing.”

Finally, we must acknowledge the tremendous support of the research community who as authors, reviewers and editors have already established PLoS ONE as an efficient and effective peer-reviewed publication, which is successfully challenging the traditional notion of a journal.

Pete Binfield, Publisher (PLoS ONE and PLoS Community Journals)
Mark Patterson, Director of Publishing

We have learned a lot from the surveys, primarily that levels of satisfaction amongst authors are generally very good. We also identified areas where services can be improved, and we’ve adjusted our services in a number ways during 2009.

We have now summarized the results of the surveys along with how we have responded to some of the suggestions during 2009 in a short presentation, which is publicly available. We feel it’s important to share these results in particular with the authors who were kind enough to complete the questionnaires, and we would like to express our thanks to the hundreds of authors who took part. We are planning to repeat the surveys next year, so that we can monitor how views have changed and can respond to suggestions for further improvements.

Meanwhile, we welcome feedback from anyone who is using our services, as an author, reviewer, editor or reader.

Their Request for Information (RFI) lasts for just 30 days and expires on 7 January 2010, so we'd like to encourage you to get involved sooner rather than later. This is an opportunity for us to shape a broader public access policy – how it should be implemented, what type of technology and features are needed, and how to manage it.

Adding your thoughts to the blog will help ensure that the administration form a balanced (the comment thread is moderated) view of stakeholders' interest. E-mail comments will also be accepted and will be posted to the blog by the moderators.

There are 3 main topics where the administration would appreciate your input (they also welcome general comments) and each one is open for a set period of time:

1. Implementation – expires 20 December 2009. Which Federal agencies are good candidates to adopt Public Access policies? What variables (field of science, proportion of research funded by public or private entities, etc.) should affect how public access is implemented at various agencies, including the maximum length of time between publication and public release?

2. Features and Technology – 21-31 December 2009. In what format should the data be submitted in order to make it easy to search and retrieve information, and to make it easy for others to link to it? Are there existing digital standards for archiving and interoperability to maximize public benefit? How are these anticipated to change?

3. Management – 1-7 January 2010. What are the best mechanisms to ensure compliance? What would be the best metrics of success? What are the best examples of usability in the private sector (both domestic and international)? Should those who access papers be given the opportunity to comment or provide feedback?

Thanks for supporting this initiative.

The open-access and library community have been asking some pertinent questions about this new launch and our involvement with it which we'd like to address in this blog post.

Q: Is PLoS charging a fee for access to articles that appear in DeepDyve?

A: There is no financial gain to PLoS – all our content is freely available online to everyone, including commercial organizations, under the terms of the Creative Commons Attribution License that we use.

Q: Why has PLoS agreed to provide its content to DeepDyve?

A. The Creative Commons License means that no permission is required to reuse PLoS content – in fact, creative reuse for commercial as well as non-commercial purposes is encouraged. Readers might like to know that almost every organization that wants to use PLoS content in bulk checks in with us first out of courtesy and this was the case with Deep Dyve.

Q: Is DeepDyve an interim solution that restricts the progress of open-access?

A: When comprehensive open-access is a reality, demand for DeepDyve's service may reduce but since most of the world's research is still locked behind subscription or pay-per-view barriers, DeepDyve at least offers an alternative way to get hold of restricted access articles for an affordable price.

Q. Is PLoS doing this to gain eyeballs on its content?

A. PLoS content is freely available to everyone who wants to reuse it. We want as many people as possible to take advantage of this content because research information is most powerful when more people can discover and use it and naturally, we're in favor of maximum exposure for the work of PLoS authors.

Q. Is this the start of a proliferation of new business models that will make use of open-access content?

A. As the influence of open-access grows, and more content becomes openly available, organizations are going to find new and creative ways to reuse it. Some of the added value that these organizations bring may well come at a cost to users who will exercise their spending power and decide which services will be ultimately successful.

Finally, when we raised some of the concerns of the community, listed above, with DeepDyve they were responsive and immediately made the status of open-access content clearer on their website.

This week – 19th-23rd October 2009 – is the Open Access week around the world – fitting nicely with the 5th birthday of PLoS Medicine. And when I say ‘around the world’ I really mean it. Just check out all the global events happening this week.

The OA Week is co-organized by Open Access Directory, PLoS, SPARC, Students for Free Culture, eIFL (Electronic Information for Libraries) and OASIS.

Many countries are participating this year, including some with numerous events all around the country. See, for example, all the events in Germany (there are 67 events in that country alone!), Netherlands, China and Japan.

You can get all the information and follow the events on the Open Access Week blog. There is also a nice round-up on the SPARC site.

As the week unfolds, we will blog more about it here. In the meantime, you can follow the news of the OA week on Facebook or by following PLoS on Twitter.You may also want to sign up to participate in the OASPA webinar (locationless – sign up to participate online).

A few months ago, PLoS initiated a program to provide a series of metrics on the individual articles published in all the PLoS Journals. You can see some examples here, here, here and here. There are two complementary benefits to the new approach.

First, we are focusing on articles rather than journals. The dominant paradigm for judging the worth of an article is to rely on the name and the impact factor of the journal in which the work is published. But it’s well known that there is a strong skew in the distribution of citations within a journal – typically, around 80% of the citations accrue to 20% of the articles. So the impact factor is a very poor predictor of how many citations any individual article will obtain, and in any case, journal editors and peer reviewers don’t always make the right decision. Indicators at the article level circumvent these limitations, allowing articles to be judged on their own scientific merits.

Second, we are not confining article-level metrics to a single indicator. As summarized by Michael Jensen, and discussed by many others including recently over at the Scholarly Kitchen, there’s a lot more to scientific impact than citations in the selection of journals covered by the Web of Science – the proprietary source of data that provides the impact factor calculation. Citations can be counted more broadly, along with web usage, blog and media coverage, social bookmarks, expert/community comments and ratings, and so on. Our own efforts are so far confined to citations (as measured by Scopus and PubMed Central), social bookmarks (as made by users of Connotea and CiteULike), and blog coverage (as recorded by Bloglines, Postgenomic and Nature Blogs), and these metrics will be improved and expanded over the coming months. The good news is that many of these indicators can be collated automatically, using openly available web tools that constantly update information on the article itself.

The presentation of a comprehensive array of this data is an enticing prospect. When an article has been published, we have tended to regard that as the end of the story (barring corrections or the occasional retraction). But if, as frequently happens, a very good article has been published in a specialist journal after being rejected from a highly selective one, it would be great to indicate to a user that this article is actually looking pretty significant, and show how its influence develops over the months and years.

Rather than basing judgments on the importance of research on the opinions of two or three reviewers and editors, article-level metrics will attempt to capture the actions and opinions of entire communities of readers to give a rich and sophisticated picture of research impact that will be helpful to authors and readers alike. Readers may then frame that picture in the context of their particular field and their own work.

To realize the vision for article-level metrics there are still some significant hurdles to clear: it won’t be enough simply to provide indicators without some context or guidance on how to interpret them; some indicators (particularly citations) take months to build up limiting their value as early indicators of impact; and standards will need to be developed so that the indicators are reliable and as free as possible from gaming and manipulation.

A clear editorial selection process will always have a place before publication in a scholarly journal. But a reduction in the reliance on the impact factor for so many aspects of research assessment could be massively liberating. PLoS Medicine, to cite an example close to home, has recently restated its mission – focusing on the diseases and risk factors that have the most profound impacts on global health. By carefully selecting articles that are likely to have the biggest influence on global health and using innovative and diverse approaches to assess and indicate that influence, PLoS Medicine will be a greater force, regardless of how many citations an average article accrues

Looking towards other modes of publishing, PLoS ONE is predicated on the notion that judgements about impact and relevance can be left almost entirely to the period after publication. By peer-reviewing submissions purely for scientific rigour, ethical conduct and proper reporting before publication, articles can be assessed and published rapidly. Once articles have joined the published literature, the impact and relevance of the article can then be determined on the basis of the activity of the research community as a whole. Article-level metrics and indicators, along with other post-publication features are part and parcel of the PLoS ONE approach, and could help readers to filter and sort literature after it is published. Ultimately, the aim of adding value to articles after publication is to improve the whole process of scientific communication and accelerate research progress itself. You can read more about article-level metrics in the context of PLoS ONE, and a talk is also available online from Pete Binfield (Managing Editor of PLoS ONE).

Article-level metrics and indicators will become powerful additions to the tools for the assessment and filtering of research outputs, and we look forward to working with the research community, publishers, funders and institutions to develop and hone these ideas. As for the impact factor, the 2008 numbers were released last month. But rather than updating the PLoS Journal sites with the new numbers, we’ve decided to stop promoting journal impact factors on our sites all together. It’s time to move on, and focus efforts on more sophisticated, flexible and meaningful measures.