Publishing, Tools & Resources

Research Resource Identification (RRID)

In 2016 the National Institutes of Health (NIH) began to require grant applications to be reviewed with respect to reproducibility criteria (see Notice NOT-OD-15-103, Enhancing Reproducibility through Rigor and Transparency). This policy was a response to years of increasing concern about the reproducibility of scientific results:

  • A team from Bayer found when investigating studies of drug targets related to oncology, women’s health, and cardiovascular disease that only 20–25% of the findings could be consistently replicated (Prinz et al. 2011).
  • A group from Amgen found that of 53 “landmark” oncology studies, for only 6, or 11%, could the findings be confirmed (Begley and Ellis 2012).
  • The ongoing Reproducibility Project in Cancer Biology is attempting to replicate high-impact oncology studies: to date, of the 17 replication studies performed, just 5 reproduce the major findings of the original papers without partial, inconsistent, or uninterpretable results.

One barrier to reproducibility is a lack of sufficient detail about the materials and tools used in research. A 2013 study of over 200 journal articles found that in more than half, key experimental resources such as model organisms, antibodies, reagents, and cell lines could not be precisely identified (Vasilevsky et al. 2013). Even when suppliers and catalog numbers are included they can be insufficient: catalog or stock numbers can be changed, discontinued, reused, or duplicated at a different supplier.

The Research Resource Identification (RRID) Initiative was created to address the issue of experimental resource identification, primarily in the biomedical sciences. A broad group representing researchers, funders, government agencies, publishers, resource suppliers and other stakeholders collaborated on the development of a system of unique and persistent identifiers to enable experimental resources to be specified. RRIDs are machine readable, free to generate and access, and are consistent and searchable across publishers and journals. They have been incorporated into the Materials Design Analysis Reporting (MDAR) Framework for transparency in reporting life sciences studies, and the NISO Journal Article Tag Suite (JATS) XML markup standard.

Figure 1. Screenshot of the RRID Portal search page, https://scicrunch.org/resources

The core of the RRID system is the RRID Portal, a persistent identifier registry hosted by the FAIR Data Informatics Lab at UC San Diego. The portal can be searched for resources such as antibodies, plasmids, organisms, cell lines, and software and analytical tools. The portal draws metadata and identifiers from established community databases such as the AntibodyRegistry, Addgene, Mutant Mouse Research and Resource Centers (MMRC), Cellosaurus, and others. Once researchers find a specific resource in the portal, it can be cited in their work using the RRID (which most publishers link back to the record for the resource in the SciCrunch Resolver; see below). And if researchers wish to submit a new resource, the RRID portal links to the community databases where resources can be registered and identifiers assigned.

Figure 2. Screenshot of an example antibody record retrieved through the RRID Portal

Example: RRID:AB_572263

Google Scholar, with its combination of metadata and full-text search, provides the most comprehensive RRID search; SciCrunch Resolver records link to Google Scholar search results using a combination of RRIDs, supplier names and catalog numbers. The RRID Portal record also lists the PubMed Identifiers (PMIDs) of articles referencing the resource. PubMed and other databases do not have an indexed RRID field, but a search on an RRID will find any article where it has been included in the abstract or author-supplied keywords field. 

Dozens of journals have endorsed the RRID Initiative (see a list of endorsing journals). An example of a journal requesting authors to use RRIDs and incorporating them into the manuscript preparation and submission workflow is the Journal of of Comparative Neurology, whose Author Guidelines state:

“Authors are asked to use RRIDs to cite the resources used in their research where applicable in the text, similar to a regular citation or GenBank Accession number. For antibodies, authors are asked to include in the citation the vendor, catalogue number, and RRID both in the text upon first mention in the Methods section. For software tools and databases, authors should provide the name of the resource followed by the resource website if available, and the RRID. For model organisms, the RRID alone is sufficient.

“Authors are also asked to include the RRIDs in the list of keywords associated with the manuscript.”

The RRID system is a community-developed initiative for improving resource identification and achieving greater transparency and reproducibility in the life sciences literature. As with similar identifier systems, the more widely adopted RRIDs become the more useful they will be.

References

Begley C, Ellis L. 2012. Raise standards for preclinical cancer research. Nature 483: 531–533. https://doi.org/10.1038/483531a

Prinz F, Schlange T, Asadullah K. 2011. Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery 10:712. https://doi.org/10.1038/nrd3439-c1

Vasilevsky NA, Brush MH, Paddock H, Ponting L, Tripathy SJ, LaRocca GM, Haendel MA. 2013. On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ 1:e148 https://doi.org/10.7717/peerj.148

Elliott Smith, Emerging Technologies & Bioinformatics Librarian, University of California Berkeley

We welcome your comments and suggestions. If you have a resource that you would like to see highlighted, please leave us a comment.

Leave a comment