Christine Cormack Wood – 2018 January 31
Second in our Meet the members blog series is Lenny Teytelman, co-founder and CEO of protocols.io, who gives us a bit of insight into his background and why he started protocols.io, what the future plans for protocols.io are, and how they use and benefit from being a Crossref member.
I am a computational and experimental biologist, and it was my struggle with correcting a published research method as a postdoctoral researcher at MIT that led me to co-found protocols.io. I spent a year and a half correcting a single step of a research recipe. Instead of 1ul of a chemical, it needed 5, instead of a 15-minute incubation, it needed an hour. But this was a correction of something previously published, not a new method, so absurdly, it was not a result that I could publish. That means I got no credit for this year and a half, and more importantly, every other scientist using this recipe is either getting misleading results or has to waste 1-2 years rediscovering what I know—rediscovering something that I’d love to share, but have no easy way of doing so.
So, I became obsessed with creating a central place where scientists can easily share and discover detailed research recipes. We’re open access, free-to-read and free-to-publish, with web & mobile apps that make these protocols dynamic and interactive.
Currently, methods sections of research papers are full of things like “we used a slightly modified version of the method reported in paperX”. Here are two examples:
Looking for protocol in 1997 paper: "as described in (x) et al '96". Finds '96 paper: "as described in (x) '87." Finds '87 paper: Paywall. pic.twitter.com/OEJMpjIEBW— Morgan Halane (@themorgantrail) November 2, 2017
2017: “Devices were fabricated as previously described [ref 8]”— Daniel Gonzales (@dgonzales1990) January 17, 2018
[ref 8] 2015: “Devices were fabricated as previously described [ref 4]”
[ref 4] 2013: “Devices were fabricated as previously described [ref 2]”
[ref 2] 2009: “Devices were fabricated with conventional methods”
We are working to increase reproducibility, by encouraging precise detailing of methods and then making it easy to keep these methods up-to-date, long after the paper is published. More broadly, our mission is to accelerate science by getting the detailed knowledge out of paper notebooks, and getting it out in months, instead of years.
Both the content, and the audience for it, has been expanding recently. When we launched in 2014, the protocols were almost exclusively wetlab biology recipes. In 2015, we added support for computational workflows and began to see bioinformatics methods. More recently, thanks to the referrals from PLOS ONE, we’ve started to see protocols for human trials, medical devices, psychology, and more. About half a year ago, we changed our landing page form “Open Access Repository of Life Science Methods” to the more general “Open Access Repository of Research Methods”.
The readership is also broadening, it’s no longer just professional researchers—we now have protocols and guidelines for undergraduate and high school students, instructions for citizen science projects, and even standard operating procedures for lab management. We’ve also been seeing more off-the-shelf use, with people sharing actual cooking recipes, and we recently began asking authors to classify whether they are sharing “research” or “non-research” instructions.
Without a doubt, we would be nowhere close to the adoption and sharing that we have now if we were not members of Crossref, registering DOIs for all public protocols. This is an absolute prerequisite for being included in author guidelines of journals, and we wouldn’t have grown in 2017 from two to over 200 journals that encourage authors to detail their recipes on protocols.io.
In addition to the benefit to protocols.io, there is a benefit to the scientists in terms of the quality control that Crossref ensures among the members. Much of this is behind the scenes and invisible to the researchers visiting protocols.io.
For example, in the beginning, we used to simply delete spam protocols. However, once we started issuing DOIs, we realized that we would be violating the Crossref requirements for minted DOIs if we simply trashed these. As a result, we had to build “retraction” functionality that allows us to take down content, put up a notice explaining the reason for removal, and keep the record so that the respective DOI continues to resolve. This is the correct way to handle removals of scientific content and it is Crossref that made us mature and improve the platform. (We’ve since had to use the retraction functionality at the request of scientists, and we’re glad we implemented it to comply with the Crossref requirements.)
Another example is the resolution report that we routinely get from Crossref, showing us which DOIs are broken. It highlights errors for us and helps us to investigate, identify, and prevent problems with the journal partners.
As I argued in my talk at the annual Crossref conference, we are finally in a position to connect scientists with the knowledge they need, automatically. Almost every scientist uses a reference manager such as Mendeley, Zotero, Paperpile, etc. to manage their literature bibliography. In turn, that means that in theory, when something happens to a paper or research objects connected to the paper (retraction, correction, update to the dataset accompanying the manuscript), the reference management platforms could notify every scientist who has that paper in their library.
The problem is that it isn’t feasible for every service like Mendeley to connect to every repository and publisher to track events connected to every paper. This is where Crossref is positioned so powerfully. By collecting the metadata linking papers to the research objects, Crossref can be the single source that the platforms need to query to see if there is news for their users related to any specific published paper. (More of this from my talk was captured really nicely in this blog post by the SMU librarian Aaron Tay.
Expanding protocols.io content to include chemistry workflows is an important goal for 2018-19.
We are also eager to start on connecting the protocols directly to the devices that the scientists use. Imagine you need to spin your cells for 30 seconds, but the centrifuge is accidentally set for 3 minutes. Our app should be able to connect to the equipment and alert the researcher to the wrong setting, asking if they are sure they want to proceed.
2018 May 30
2018 April 26