Systematic Reviews: Grey Literature

Grey Literature refers to non-traditional sources of information and may include: reports, leaflets, bulletins, dissertations etc. These sources of information may not be published through the traditional route and may not be available within typical bibliographic databases. Often, these materials are not peer-reviewed unlike journal articles.

Fig 1 shows the process of searching for grey literature sources.

Fig 1: Process of grey literature search. Adapted from: Duffield, A. et al. (2004). "Process to Identify the Grey Literature", Review of the published literature for the impact and cost-effectiveness of six nutrition related emergency interventions

Types

• Reports-including preprints; preliminary progress and advanced reports [including ongoing trials reports], institutional, internal, technical, and statistical reports
• Theses
• Conference proceedings
• Technical specifications and standards
• Translations (not distributed commercially)
• Bibliographies
• Technical and commercial documentation
• Official documents (issued in limited numbers)

Alberani, V et al. (1990). The use of grey literature in health sciences: a preliminary survey. Bulletin of the Medical Library Association. 78(4): 358-63.

 

Recommended Sources

There are a number of sources where grey literature can be found. These include:

Repositories

• View the Digital Repository (DR-NTU) open access contributed by researchers from LKCMedicine and other communities. These are final peer-reviewed manuscript of journal articles and conference papers.
• View the Digital Repository (DR-NTU) restricted access contributed by students from LKCMedicine and other communities. These are full-text versions of theses or reports.

Catalogue

• WorldCat which indexes millions of holdings from numerous libraries world-wide

Websites

The Internet is now a major source to retrieve grey literature and often a good starting point to research a particular topic area. Please check the links below:

Relevant to all

• Royal College of Physicians and Surgeons of Glasgow
• Royal College of Physicians of London
• Royal College of Physicians of Edinburgh
• Royal College of Surgeons of England
• Royal College of Surgeons of Edinburgh

You can also refer to our A to Z list of identified grey literature websites on the right column of this page. As your work progresses, you’ll become aware of the main organisations active in the medical/healthcare specialty. E.g. government bodies and research institutes. Do a thorough search of their webpages and articles/published works.

Databases

Here are some databases from our NTU library website that index conference papers and conference proceedings:

1) Web of Science

• Web of Science Conference Proceedings Citation Indexes
  • 2 conference proceedings citation indexes – a Science index, and a Social Sciences & Humanities Index.
  • Limit your search, select from a list of core collection databases in ” More Settings” section at bottom of the Web of Science search page.

2) CINAHL

3) ProQuest Dissertation & Theses Global (PQDT Global) & ProQuest Dissertations &Theses – UK and Ireland (Index to Theses)*

• This database has ordering information to obtain abstracts and copies of papers
• *Included in PQDT Global

4) EMBASE

5) PsycInfo

6) TRIP Database (clinical medicine) – (Click here)

The AACODS technique to enable evaluation and critical appraisal:

• Authority: Is the author credible?
• Accuracy: Is it supported by documented and authoritative references? Is there a clearly stated methodology? Is it ‘in line’ with other work on the same topic
• Coverage: Have limitations been imposed and are these stated clearly?
• Objectivity: Can bias be detected?
• Date: Can’t find the date? Rule of the thumb is to avoid such material
• Significance: Is it relevant? Would it enrich or have an impact on your research?
• Refer to the AACODS Checklist, put together by Jess Tyndal at Flinders University, is a good resource for more information on evaluating grey literature.

---

• Evidence-Based Practice materials in reference to the PICO question
  • Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model
  • Centre for Evidence-Based Medicine
  • Levels of Evidence FAME
  • NHMRC Levels of Evidence
• Quality of articles and reliability of online sources using the radar-test checklist
• Critical appraisal checklists on:
  • Case control
  • Cohort
  • Diagnostic studies
  • Randomized control trial
  • Survey

A

• ADOLEC (Adolescent Health) (Click here) – Adolescent health, with particular emphasis on information covering America.
• Agency for Healthcare Research and Quality (AHRQ) (Click here) – The Agency for Healthcare Research and Quality’s (AHRQ) mission is to produce evidence to make health care safer, higher quality, more accessible, equitable, and affordable, and to work within the U.S. Department of Health and Human Services and with other partners to make sure that the evidence is understood and used. AHRQ priorities are described.
• Amercian Psychiatry Association – (Click here)
• Amedeo Medical Literature Guide – You can source for medical literature grouped by journals according to specializations
• ACP Journal Club – Evidence-Based Medicine resource. ACP Journal Club Archives is a collection of articles and editorials from ACP Journal Club published between January 1991 and April 2008 when it was a bimonthly stand-alone journal. Since May 2008, ACP Journal Clubhas been published as a monthly feature of Annals of Internal Medicine that summarizes the best new evidence for internal medicine from over 130 clinical journals. Research staff and clinical editors rigorously assess the scientific merit of the medical literature as it is published, and a worldwide panel of over 5000 physicians assesses the clinical relevance and newsworthiness of rigorous studies.

B

• BMA
• BASE (Click here) – the Bielefeld Academic Search Engine
• Biomed Central (Springer) – One of the scholarly publishing industries with a large proportion of open access medical journals
• BMC Proceedings (Click here)
• Best Evidence Medical and Health Professional Education (BEME) – Click here
• BMJ – Click here
  • BMJ Specialties – news and articles from the BMJ relating to specific topics
  • Evidence Alerts
• British Library – wide range of material held, use the filter options to focus your search.

C

• Canadian Agency for Drugs and Technologies in Health (CADTH) – Click here
• CASH (Current Awareness Service for Health) – browsable topics of the latest news around a range of topics.
• CADTH (Canadian Agency for Drugs and Technologies in Health) – independent, not for profit Canadian-based organisation, providing evidence to support healthcare decisions.
• CogPrints (Click here) – Electronic archive for self-archive papers in the areas of psychology, neuroscience, and linguistics.
• Cochrane library grey literature Databases (Click here) – More grey literature databases are available through the Cochrane Handbook http://handbook.cochrane.org/.
  • Go to Part 2 > General methods for Cochrane reviews > Searching for studies > Sources to search>Bibliographic databases > Grey literature databases
• Community of Science (COS), Inc. contains information about scientific research- includes COS Funding and COS Expertise data for UK researchers
• Centers for Disease Control and Prevention (Click here) – You can view Morbidity and Mortality Weekly Reports (MMWR), Emerging Infectious Diseases (EID), Preventing Chronic Disease (PCD), Data & Statistics, Science Clips, Public Health Image Library and Publications.
• Childlink – Focus on legislation, policies and practices regarding children, young people and families living in the UK and Ireland. Includes government papers, statistics, research and legislation. (Create an account)
• ClinicalTrials.gov – This is a registry and results database of publicly and privately supported clinical studies of human participants conducted around the world. Part of the U.S. National Institutes of Health

D

• DESASTRES (disasters and emergencies) (Click here)
• Department of Health (DoH) – Publications, reports, policies and other information from the UK Department of Health

E

• Embase Conference Papers – Select database from NTU library website. Select Publication Type: Conference Abstract, Conference Paper or Conference review from Filter menu or in the advanced search menu. Filter by Conference also available
• eMedicine Special Topics – US-based evidence-based resources related to medical conditions

F

• Factiva – Select database from NTU library website
• Faculty of 1000 Medicine – The F1000 team  is composed of 8,000 senior scientists and leading experts in all areas of biology and medicine. The Faculty recommends the most important articles, rating them and providing short explanations for their selections.

G

• Google – If you know the names of conferences / societies in the area of your systematic review topic, searching by name will lead you to the proceedings, which are usually published on their website (E.g: SingHealth Duke-NUS Scientific Congress)
• Google Scholar – Google Scholar provides a simple way to broadly search for scholarly literature
• Grey Literature report (Click here) – New York Academy of Medicine
• Grey Source (Click here) – Examples of grey literature supported with organizations responsible for its production and/or processing. (See classification)

H

• Health Systems Evidence – from McMaster University, database of research regarding governance, finance, delivery and implementation strategies in healthcare systems
• HSA (Click here) – The Health Sciences Authority (HSA) was formed on 1 April 2001 as a statutory board of the Singapore Ministry of Health with the integration of five specialised agencies:
  • Centre for Drug Evaluation
  • Institute of Science and Forensic Medicine
  • National Pharmaceutical Administration
  • Product Regulation Department
  • Singapore Blood Transfusion Service
• HISA (public health) (Click here)
• HealthScotland

I

• Informaworld (Taylor & Francis): One of the scholarly publishing industries with a large proportion of medical journals

J

• NEJM Journal Watch–  NEJM Journal Watch equips today’s caregivers with the information and perspective needed to be clinically prepared and practice with confidence. NEJM Journal Watch is used by practicing clinicians – physicians, nurses, physician assistants, nurse practitioners, residents, students, pharmacists, and other healthcare professionals to stay informed and current in their fields.

K

• King’s Fund – reports, bulletins, blogs and news items, based on healthcare topics.

L

• LWWonline (Wolters Kluwer) – One of the scholarly publishing industries with a large proportion of medical journals

M

• Ministry of Health (Click here) – Singapore Government health website
• MedNar (Click here) – New York Academy of Medicine. Use Advanced Search to specify where to search (societies, government websites, etc)
• Medline Plus (Click here) – MedlinePlus is the National Institutes of Health’s Web site for patients and their families and friends. Produced by the National Library of Medicine, the world’s largest medical library, it brings you information about diseases, conditions, and wellness issues in language you can understand. MedlinePlus offers reliable, up-to-date health information, anytime, anywhere, for free.
• MedPageToday – Newsletters and latest meetings covered in Medicine to address various types of health issues
• MDLinx – MDLinx is an award-winning website that makes it possible for busy physicians and healthcare professionals to stay current on today’s latest medical news and information. MDLinx sorts, ranks, and summarizes medical news articles and journals for easy and accessible viewing. Personalized newsletters conveniently deliver the latest medical news to you. It’s never been easier to stay current. Test drive our exclusive medical content.
• M.J. Powers – MJ Powers & Co publishes monthly newsletters about the efficacy and safety of drugs and therapeutics. The Alerts contain summaries, from a wide variety of journals, that can help you solve a problem, warn you about safety, or reassure you about efficacy. Not only will you receive the latest research findings, but also useful tools to help you judge research quality, evaluate study independence, and guide you with statistics.
• McMaster Online Rating of Evidence – The Health Information Research Unit (HiRU) at McMaster University has developed the McMaster Online Rating of Evidence (MORE™) system to help define the best research to support evidence-based clinical practice, tailored to the individual interests of clinicians. New health care knowledge advances quickly and the marriage of the “Internet Age” and evidence-based health care makes it potentially easier for health care practitioners to keep up to date by matching high-quality evidence with individual practice needs.

N

• NICE (Click here) – Clinical standards, UK
• National Institutes of Health (NIH) (Click here) – The National Institutes of Health (NIH), a part of the U.S. Department of Health and Human Services, is the nation’s medical research agency — making important discoveries that improve health and save lives.
• National Institute on Aging (Click here) – NIA, one of the 27 Institutes and Centers of NIH, leads a broad scientific effort to understand the nature of aging and to extend the healthy, active years of life. NIA is the primary Federal agency supporting and conducting Alzheimer’s disease research. Now contains content formerly included in NIH Senior Health.
• NIH Health Information (Click here) – You can search for articles/newsletters on general health and disease management, clinical research trials, community resources and publications.
• NHS Evidence – Evidence base for UK Health & Social Care professions including primary research, statistics, guidelines and health technology assessments
• NHS Clinical Knowledge Summaries – now only available in the UK
• NHS Education for Scotland
• NHS Scotland: Scotland’s Health on the Web

O

• OpenDOAR (Click here) – Directory of academic open access repositories. Search for the full-text of material held in open access repositories listed in the Directory using ‘Search Repository Contents’, or use OpenDOAR to find repositories or groups of repositories that fit particular needs using our ‘Find’ facility.
• OpenGrey (Click here) – Multidisciplinary European database providing references to technical or research reports, doctoral dissertations, conference papers and other grey literature.

P

• Popline (reproductive health) – (Click here)

R

• REPIDISCA (sanitation and environmental sciences) – (Click here)
• Rehabdata – National Rehabilitation Information Centre

S

• Science.gov (Click here) – Gateway to over 50 million pages of authoritative selected science information provided by U.S. government agencies, including research and development results.
• Scopus – Select database from NTU library website. To locate conference proceedings in Scopus, search for article. Use the refine functions on the left hand side of the results page, limit your results to Conference Papers under the ‘Document Type’ section.
• Social Care Online – Information and research on all aspects of social care. Includes research briefings, reports, government documents and case studies from the UK
• Scottish Executive Health Department
• Scottish Government
• Scottish Intercollegiate Guidelines Network (SIGN)

T

• Tomorrow’s Medicine (Click here) – Singapore, SingHealth Duke-NUS Academic Medical Centre (Newsletter)
• The Medical Letter – The Medical Letter, Inc. is a nonprofit organization that publishes critical appraisals of new prescription drugs and comparative reviews of drugs for common diseases in its newsletter

U

• Union catalogues such as COPAC or WorldCat include reports and theses
• U.S Food and Drug Administration (FDA) (Click here) – The Food and Drug Administration is responsible for protecting the public health by ensuring the safety, efficacy, and security of human and veterinary drugs, biological products, and medical devices; and by ensuring the safety of our nation’s food supply, cosmetics, and products that emit radiation. Read more about FDA –Singapore, MOU Regarding New Medical Products
• U.S Department of Health & Human Services / HHS.gov (Click here) –  It is the mission of the U.S. Department of Health & Human Services (HHS) to enhance and protect the health and well-being of all Americans. We fulfill that mission by providing for effective health and human services and fostering advances in medicine, public health, and social services.
• U.S. Public Health Service (USPHS) (Click here)
• UK Pubmed Central – abstracts and some free full-text research articles for biomedical and health researchers

W

• World Bank Publications (Click here) – World Bank Publications or reports from around the globe
• World Health Organisation (Click here) – WHO Publications from the different regions. Here are a few organisations partnered with WHO.
  • CEHA Database (environmental health & sanitation) (Click here)
  • Pan American Health Organisation (PAHO) library (Click here) – Headquarters Library and Information Services supports the work of the Organization and provides a wide variety of professional and technical reference and information services covering a broad range of subject areas within the biomedical and public health field, including, historical information. It also collects, organizes, preserves and disseminates regional information produced by PAHO, both bibliographic and in full text, besides offering a selected collection on related health subjects affecting the people of Latin America and the Caribbean. The Library’s collections are referenced in its electronic PAHO/HQ Online Library Catalog, accessible through the Web.
  • Research 4 life (Click here) -The collective name for the four programmes – HINARI, AGORA, OARE and ARDI – Research4Life provides developing countries with free or low cost access to academic and professional peer-reviewed content online. Research4Life is a public-private partnership of the WHO, FAO, UNEP, WIPO, Cornell and Yale Universities and the International Association of Scientific, Technical & Medical Publishers. Working together with technology partner Microsoft, the partnership’s goal is to help attain six of the UN’s eight Millennium Development Goals by 2015, reducing the scientific knowledge gap between industrialized countries and the developing world.
  • Global Health Library (Click here) – Global and regional indexes to the scientific and technical literature. Many of the articles found in searches are free online such as those in the Bulletin of the World Health Organization.
  • Sexual and reproductive health – (Click here)
  • ICTRP Search Portal – Aims to provide a single point of access to information about ongoing and completed clinical trials. Provided by the World Health Organisation

• WorldWideScience.gov (Click here) – Global science gateway-accelerating scientific discovery and progress through a multilateral partnership to enable federated searching of national and international scientific databases.