Purdue Online Writing Lab College of Liberal Arts
Types of Sources
Welcome to the Purdue OWL
This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.
Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.
We live in an age overflowing with sources of information. With so many information sources at our fingertips, knowing where to start, sorting through it all and finding what we want can be overwhelming! This handout provides answers to the following research-related questions: Where do I begin? Where should I look for information? What types of sources are available?
This section lists the types of sources most frequently used in academic research and describes the sort of information that each commonly offers.
Print Sources
Books and Textbooks: Odds are that at least one book has been written about virtually any research topic you can imagine (and if not, your research could represent the first steps toward a best-selling publication that addresses the gap!). Because of the time it takes to publish a book, books usually contain more dated information than will be found in journals and newspapers. However, because they are usually much longer, they can often cover topics in greater depth than more up-to-date sources.
Newspapers: Newspapers contain very up-to-date information by covering the latest events and trends. Newspapers publish both factual information and opinion-based articles. However, due to journalistic standards of objectivity, news reporting will not always take a “big picture” approach or contain information about larger trends, instead opting to focus mainly on the facts relevant to the specifics of the story. This is exacerbated by the rapid publication cycles most newspapers undergo: new editions must come out frequently, so long, in-depth investigations tend to be rarer than simple fact-reporting pieces.
Academic and Trade Journals: Academic and trade journals contain the most up-to-date information and research in industry, business, and academia. Journal articles come in several forms, including literature reviews that overview current and past research, articles on theories and history, and articles on specific processes or research. While a well-regarded journal represents the cutting-edge knowledge of experts in a particular field, journal articles can often be difficult for non-experts to read, as they tend to incorporate lots of technical jargon and are not written to be engaging or entertaining.
Government Reports and Legal Documents: The government regularly releases information intended for internal and/or public use. These types of documents can be excellent sources of information due to their regularity, dependability, and thoroughness. An example of a government report would be any of the reports the U.S. Census Bureau publishes from census data. Note that most government reports and legal documents can now be accessed online.
Press Releases and Advertising: Companies and special interest groups produce texts to help persuade readers to act in some way or inform the public about some new development. While the information they provide can be accurate, approach them with caution, as these texts' publishers may have vested interests in highlighting particular facts or viewpoints.
Flyers, Pamphlets, Leaflets: While some flyers or pamphlets are created by reputable sources, because of the ease with which they can be created, many less-than-reputable sources also produce these. Pamphlets and leaflets can be useful for quick reference or very general information, but beware of pamphlets that spread propaganda or misleading information.
Digital and Electronic Sources
Multimedia: Printed material is certainly not the only option for finding research. You might also consider using sources such as radio and television broadcasts, interactive talks, and recorded public meetings. Though we often go online to find this sort of information today, libraries and archives offer a wealth of nondigitized media or media that is not available online.
Websites: Most of the information on the Internet is distributed via websites. Websites vary widely in terms of the quality of information they offer. For more information, visit the OWL's page on evaluating digital sources.
Blogs and personal websites: Blogs and personal sites vary widely in their validity as sources for serious research. For example, many prestigious journalists and public figures may have blogs, which may be more credible than most amateur or personal blogs. Note, however, that there are very few standards for impartiality or accuracy when it comes to what can be published on personal sites.
Social media pages and message boards: These types of sources exist for all kinds of disciplines, both in and outside of the university. Some may be useful, depending on the topic you are studying, but, just like personal websites, the information found on social media or message boards is not always credible.
Literature Reviews
What this handout is about.
This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.
Introduction
OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?
Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.
What is a literature review, then?
A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.
A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.
But how is a literature review different from an academic research paper?
The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.
Why do we write literature reviews?
Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.
Who writes these things, anyway?
Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.
Let’s get to it! What should I do before writing the literature review?
If your assignment is not very specific, seek clarification from your instructor:
- Roughly how many sources should you include?
- What types of sources (books, journal articles, websites)?
- Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
- Should you evaluate your sources?
- Should you provide subheadings and other background information, such as definitions and/or a history?
Find models
Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.
Narrow your topic
There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.
Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .
And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.
Consider whether your sources are current
Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.
Strategies for writing the literature review
Find a focus.
A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.
Convey it to your reader
A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:
The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.
Consider organization
You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:
First, cover the basic categories
Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:
- Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
- Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
- Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?
Organizing the body
Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.
To help you come up with an overall organizational framework for your review, consider the following scenario:
You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.
Now consider some typical ways of organizing the sources into a review:
- Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
- By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
- By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
- Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
- Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.
Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:
- Current Situation: Information necessary to understand the topic or focus of the literature review.
- History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
- Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.
Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?
Begin composing
Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:
However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).
Use evidence
In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.
Be selective
Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.
Use quotes sparingly
Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.
Summarize and synthesize
Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.
Keep your own voice
While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.
Use caution when paraphrasing
When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .
Revise, revise, revise
Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .
Works consulted
We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.
Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.
Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.
Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.
Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.
Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.
Make a Gift
Literature Review: Lit Review Sources
- Lit Review Types
- GRADE System
- Do a Lit Review
- Lit Review Sources
Where do I find information for a literature review?
Research is done by...
...by way of...
...communicated through...
...and organized in...
Types of sources for a review...
- Primary source: Usually a report by the original researchers of a study (unfiltered sources)
- Secondary source: Description or summary by somebody other than the original researcher, e.g. a review article (filtered sources)
- Conceptual/theoretical: Papers concerned with description or analysis of theories or concepts associated with the topic
- Anecdotal/opinion/clinical: Views or opinions about the subject that are not research, review or theoretical (case studies or reports from clinical settings)
A Heirarchy of research information:
Source: SUNY Downstate Medical Center. Medical Research Library of Brooklyn. Evidence Based Medicine Course. A Guide to Research Methods: The Evidence Pyramid: http://library.downstate.edu/EBM2/2100.htm
Life Cycle of Publication
Click image to enlarge
Scientific information has a ‘life cycle’ of its own… it is born as an idea, and then matures and becomes more available to the public. First it appears within the so-called ‘invisible college’ of experts in the field, discussed at conferences and symposia or posted as pre-prints for comments and corrections. Then it appears in the published literature (the primary literature), often as a journal article in a peer-reviewed journal.
Researchers can use the indexing and alerting services of the secondary literature to find out what has been published in a field. Depending on how much information is added by the indexer or abstracter, this may take a few months (though electronic publication has sped up this process). Finally, the information may appear in more popular or reference sources, sometimes called the tertiary literature.
The person beginning a literature search may take this process in reverse: using tertiary sources for general background, then going to the secondary literature to survey what has been published, following up by finding the original (primary) sources, and generating their own research Idea.
(Original content by Wade Lee-Smith)
- << Previous: Do a Lit Review
- Next: Readings >>
- Last Updated: Feb 20, 2023 9:44 AM
- URL: https://libguides.utoledo.edu/litreview
Warning: The NCBI web site requires JavaScript to function. more...
An official website of the United States government
The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
- Publications
- Account settings
- Browse Titles
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].
Chapter 9 methods for literature reviews.
Guy Paré and Spyros Kitsiou .
9.1. Introduction
Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).
Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).
The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).
When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.
The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.
9.2. Overview of the Literature Review Process and Steps
As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:
- formulating the research question(s) and objective(s),
- searching the extant literature,
- screening for inclusion,
- assessing the quality of primary studies,
- extracting data, and
- analyzing data.
Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).
Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).
9.3. Types of Review Articles and Brief Illustrations
EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.
9.3.1. Narrative Reviews
The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).
Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).
Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.
Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.
9.3.2. Descriptive or Mapping Reviews
The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).
In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.
An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).
9.3.3. Scoping Reviews
Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.
Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).
One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).
9.3.4. Forms of Aggregative Reviews
Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.
Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:
- Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
- Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
- Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
- Analyzing data using quantitative or qualitative methods.
- Presenting results in summary of findings tables.
- Interpreting results and drawing conclusions.
Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.
The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed independently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.
Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.
A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guidelines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.
In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).
9.3.5. Realist Reviews
Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).
To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).
The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.
9.3.6. Critical Reviews
Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).
Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.
9.4. Summary
Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.
Typology of Literature Reviews (adapted from Paré et al., 2015).
As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.
9.5. Concluding Remarks
In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.
We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.
To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.
- Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
- Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
- Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
- Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
- A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
- Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
- Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
- Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
- Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
- Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
- Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
- Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
- Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
- Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
- Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
- Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
- Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
- Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
- Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
- Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
- Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
- Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
- Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
- Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
- Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
- Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
- King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
- Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
- Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
- Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
- Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
- Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
- Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
- Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
- Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
- Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
- Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
- Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
- Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
- Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel .aisnet.org/cgi/viewcontent .cgi?article =1221&context =ecis2011 .
- Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
- Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
- Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
- Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
- Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
- Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
- Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
- Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
- Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
- Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
- Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
- Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
- Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
- Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
- Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
- Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
- Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
- Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
- Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
- Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
- Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]
This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/
- Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
- PDF version of this title (4.5M)
- Disable Glossary Links
In this Page
- Introduction
- Overview of the Literature Review Process and Steps
- Types of Review Articles and Brief Illustrations
- Concluding Remarks
Recent Activity
- Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach
Your browsing activity is empty.
Activity recording is turned off.
Turn recording back on
Connect with NLM
National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894
Web Policies FOIA HHS Vulnerability Disclosure
Help Accessibility Careers
Have a language expert improve your writing
Run a free plagiarism check in 10 minutes, generate accurate citations for free.
- Knowledge Base
- Working with sources
- Primary vs. Secondary Sources | Difference & Examples
Primary vs. Secondary Sources | Difference & Examples
Published on June 20, 2018 by Raimo Streefkerk . Revised on January 23, 2023.
When you do research, you have to gather information and evidence from a variety of sources.
Primary sources provide raw information and first-hand evidence. Examples include interview transcripts, statistical data, and works of art. Primary research gives you direct access to the subject of your research.
Secondary sources provide second-hand information and commentary from other researchers. Examples include journal articles, reviews, and academic books . Thus, secondary research describes, interprets, or synthesizes primary sources.
Primary sources are more credible as evidence, but good research uses both primary and secondary sources.
Table of contents
What is a primary source, what is a secondary source, primary and secondary source examples, how to tell if a source is primary or secondary, primary vs secondary sources: which is better, frequently asked questions about primary and secondary sources.
A primary source is anything that gives you direct evidence about the people, events, or phenomena that you are researching. Primary sources will usually be the main objects of your analysis.
If you are researching the past, you cannot directly access it yourself, so you need primary sources that were produced at the time by participants or witnesses (e.g. letters, photographs, newspapers ).
If you are researching something current, your primary sources can either be qualitative or quantitative data that you collect yourself (e.g. through interviews , surveys , experiments ) or sources produced by people directly involved in the topic (e.g. official documents or media texts).
A secondary source is anything that describes, interprets, evaluates, or analyzes information from primary sources. Common examples include:
- Books , articles and documentaries that synthesize information on a topic
- Synopses and descriptions of artistic works
- Encyclopedias and textbooks that summarize information and ideas
- Reviews and essays that evaluate or interpret something
When you cite a secondary source, it’s usually not to analyze it directly. Instead, you’ll probably test its arguments against new evidence or use its ideas to help formulate your own.
Prevent plagiarism. Run a free check.
Examples of sources that can be primary or secondary.
A secondary source can become a primary source depending on your research question . If the person, context, or technique that produced the source is the main focus of your research, it becomes a primary source.
Documentaries
If you are researching the causes of World War II, a recent documentary about the war is a secondary source . But if you are researching the filmmaking techniques used in historical documentaries, the documentary is a primary source .
Reviews and essays
If your paper is about the novels of Toni Morrison, a magazine review of one of her novels is a secondary source . But if your paper is about the critical reception of Toni Morrison’s work, the review is a primary source .
Newspaper articles
If your aim is to analyze the government’s economic policy, a newspaper article about a new policy is a secondary source . But if your aim is to analyze media coverage of economic issues, the newspaper article is a primary source .
To determine if something can be used as a primary or secondary source in your research, there are some simple questions you can ask yourself:
- Does this source come from someone directly involved in the events I’m studying (primary) or from another researcher (secondary)?
- Am I interested in evaluating the source itself (primary) or only using it for background information (secondary)?
- Does the source provide original information (primary) or does it comment upon information from other sources (secondary)?
Most research uses both primary and secondary sources. They complement each other to help you build a convincing argument. Primary sources are more credible as evidence, but secondary sources show how your work relates to existing research. Tertiary sources are often used in the first, exploratory stage of research.
What do you use primary sources for?
Primary sources are the foundation of original research. They allow you to:
- Make new discoveries
- Provide credible evidence for your arguments
- Give authoritative information about your topic
If you don’t use any primary sources, your research may be considered unoriginal or unreliable.
What do you use secondary sources for?
Secondary sources are good for gaining a full overview of your topic and understanding how other researchers have approached it. They often synthesize a large number of primary sources that would be difficult and time-consuming to gather by yourself. They allow you to:
- Gain background information on the topic
- Support or contrast your arguments with other researchers’ ideas
- Gather information from primary sources that you can’t access directly (e.g. private letters or physical documents located elsewhere)
When you conduct a literature review or meta analysis, you can consult secondary sources to gain a thorough overview of your topic. If you want to mention a paper or study that you find cited in a secondary source, seek out the original source and cite it directly.
Remember that all primary and secondary sources must be cited to avoid plagiarism . You can use Scribbr’s free citation generator to do so!
Common examples of primary sources include interview transcripts , photographs, novels, paintings, films, historical documents, and official statistics.
Anything you directly analyze or use as first-hand evidence can be a primary source, including qualitative or quantitative data that you collected yourself.
Common examples of secondary sources include academic books, journal articles , reviews, essays , and textbooks.
Anything that summarizes, evaluates or interprets primary sources can be a secondary source. If a source gives you an overview of background information or presents another researcher’s ideas on your topic, it is probably a secondary source.
To determine if a source is primary or secondary, ask yourself:
- Was the source created by someone directly involved in the events you’re studying (primary), or by another researcher (secondary)?
- Does the source provide original information (primary), or does it summarize information from other sources (secondary)?
- Are you directly analyzing the source itself (primary), or only using it for background information (secondary)?
Some types of source are nearly always primary: works of art and literature, raw statistical data, official documents and records, and personal communications (e.g. letters, interviews ). If you use one of these in your research, it is probably a primary source.
Primary sources are often considered the most credible in terms of providing evidence for your argument, as they give you direct evidence of what you are researching. However, it’s up to you to ensure the information they provide is reliable and accurate.
Always make sure to properly cite your sources to avoid plagiarism .
A fictional movie is usually a primary source. A documentary can be either primary or secondary depending on the context.
If you are directly analyzing some aspect of the movie itself – for example, the cinematography, narrative techniques, or social context – the movie is a primary source.
If you use the movie for background information or analysis about your topic – for example, to learn about a historical event or a scientific discovery – the movie is a secondary source.
Whether it’s primary or secondary, always properly cite the movie in the citation style you are using. Learn how to create an MLA movie citation or an APA movie citation .
Articles in newspapers and magazines can be primary or secondary depending on the focus of your research.
In historical studies, old articles are used as primary sources that give direct evidence about the time period. In social and communication studies, articles are used as primary sources to analyze language and social relations (for example, by conducting content analysis or discourse analysis ).
If you are not analyzing the article itself, but only using it for background information or facts about your topic, then the article is a secondary source.
Cite this Scribbr article
If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.
Streefkerk, R. (2023, January 23). Primary vs. Secondary Sources | Difference & Examples. Scribbr. Retrieved March 9, 2023, from https://www.scribbr.com/working-with-sources/primary-and-secondary-sources/
Is this article helpful?
Raimo Streefkerk
Other students also liked, how to avoid plagiarism | tips on citing sources, the basics of in-text citation | apa & mla examples, how to quote | citing quotes in apa, mla & chicago.
Types of Literature Review
There are many types of literature review. The choice of a specific type depends on your research approach and design. The following types of literature review are the most popular in business studies:
Narrative literature review , also referred to as traditional literature review, critiques literature and summarizes the body of a literature. Narrative review also draws conclusions about the topic and identifies gaps or inconsistencies in a body of knowledge. You need to have a sufficiently focused research question to conduct a narrative literature review
Systematic literature review requires more rigorous and well-defined approach compared to most other types of literature review. Systematic literature review is comprehensive and details the timeframe within which the literature was selected. Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis.
When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn. Meta-analysis is associated with deductive research approach.
Meta-synthesis, on the other hand, is based on non-statistical techniques. This technique integrates, evaluates and interprets findings of multiple qualitative research studies. Meta-synthesis literature review is conducted usually when following inductive research approach.
Scoping literature review , as implied by its name is used to identify the scope or coverage of a body of literature on a given topic. It has been noted that “scoping reviews are useful for examining emerging evidence when it is still unclear what other, more specific questions can be posed and valuably addressed by a more precise systematic review.” [1] The main difference between systematic and scoping types of literature review is that, systematic literature review is conducted to find answer to more specific research questions, whereas scoping literature review is conducted to explore more general research question.
Argumentative literature review , as the name implies, examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. It should be noted that a potential for bias is a major shortcoming associated with argumentative literature review.
Integrative literature review reviews , critiques, and synthesizes secondary data about research topic in an integrated way such that new frameworks and perspectives on the topic are generated. If your research does not involve primary data collection and data analysis, then using integrative literature review will be your only option.
Theoretical literature review focuses on a pool of theory that has accumulated in regard to an issue, concept, theory, phenomena. Theoretical literature reviews play an instrumental role in establishing what theories already exist, the relationships between them, to what degree existing theories have been investigated, and to develop new hypotheses to be tested.
At the earlier parts of the literature review chapter, you need to specify the type of your literature review your chose and justify your choice. Your choice of a specific type of literature review should be based upon your research area, research problem and research methods. Also, you can briefly discuss other most popular types of literature review mentioned above, to illustrate your awareness of them.
[1] Munn, A. et. al. (2018) “Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach” BMC Medical Research Methodology
John Dudovskiy
Join the Premium Student Club @Zero Cost!
Get Assignment Done by MAS Certified Experts
Flat 50% Off on Assignment Bookings
- Assignments
- Dissertations
- Research Proposal
- Report Writing
- Literature Review
- CDR Writing
- Resume Writing
- Engineering
- IT & Computer Science
- Value Added Services
- Plagiarism Check
- Proofreading & Editing
- Expert Consultation
- Live Session 1-On-1
- Quality Check
- By University
- By Subject Codes
- Reviews 4.5/5
- Help & Support
- Certified Experts
- Our Policies
- Ask your Question
- Request Callback
- Get $50 Off
- Get 500 Words Free
- Premium Membership
Quick Searches
- Samples/Resources - e.g. Management Assignment Resources
- Services - e.g. MATLAB Assignment Help
- Questions - e.g. Management Assignment Questions
- Support : FAQs , Policies, Ordering Support
Literature Sources: Basics of Literature Review
Author : Celina
Are you a student of English Literature and have got an assessment but not sure what to include or exclude? Come to our English Literature assignment help services. At My Assignment Services, we believe in providing top quality services at the best prices. The “Literature”, is a collection of articles of topics researched and are academic or scholarly in nature. Thus, a “literature source” can be referred to as a product of information produces or resulted by studying an object. This may include books, peer-reviewed articles or journals, dissertations, research papers or conference papers. While performing a literature review from the sources, it’s quite important to identify the major work as well as the supporting sources for the particular work review. We have a range of online assignment help experts who are available 24x7 to assist you whenever you come across any type of queries.
Types Of Sources
Primary sources:.
The termprimarysourceis used broadly to embody all sources that are original. Primarysourcesprovide first-hand information that is closest to the object of study. Primary sources vary by discipline.
- In the natural and social sciences, original reports of research found in academic journals detailing the methodology used in the research, in-depth descriptions, and discussions of the findings are considered primary sources of information.
- Other common examples ofprimarysourcesinclude speeches, letters, diaries, autobiographies, interviews, official reports, court records, artifacts, photographs, and drawings.
Secondary Sources:
Asecondary sourceis a source that provides non-original or secondhand data or information.
- Secondary sources are written about primary sources.
- Research summaries reported in textbooks, magazines, and newspapers are considered secondary sources. They typically provide global descriptions of results with few details on the methodology. Other examples of secondary sources include biographies and critical studies of an author's work.
Learn how to acknowledge any source of information using the APA 7th style .
Concepts included in Literature sources
- Literature sources increase the weight of the substance. It helps in providing academic assistance to students helps them in achieving good grades in the assignments and also prepares them in applying the leanings at their workplace.
- Academic help can make the nurses feel competent and provide help in cases where critical thinking skills are required. It helps students in realizing or seeing the concept from new eyes. It also makes the students gain different approaches and perspectives in approaching the assignments instead of only one.
- Finally, the provided work is in accordance with the requirements of both the student as well as the tutor. They can be cited in Harvard as well as the APA style.
Our online assignment help experts focuses on providing the solution within the word limit. The experts specialize in formulating the answers in a non-complex way. They explain the concepts and the learning outcomes which the educator is trying to achieve through the assessment.
Grab Your Literature Assignments Today!
My Assignment Services are the reliable platform where you can avail all sorts of assignment help services at the cheapest of the prices. We have a range of top quality assignment experts who can assist you to solve different assessment tasks; no matter how complex they are as well as the different assignment types. Opting our services is really easy and can be achieved within a single click. You can fill our online form, state your requirements and get the desired help possible.
Related Study Materials
- 700210 The Roles of Social Media News Consumption Assignment Sample
- 700210 Unveiling the Dark Side of Social Networking Sites Assignment Sample
- FND101 Procrastination Characteristics Assignment Sample
- 900108 The Impact of Online Social Networking Assignment Sample
- Exams Academic Essay Assignment Sample
- Successful Ageing Assignment Sample
- ENAB0040 Academic Writing Assignment Sample
- Organisation Culture in Two Contrasting Organisations Assignment Sample
- Engaging with Aboriginal and Torres Strait Islander Sample
- Enterprise Information Systems
Our Experts can answer your Assignment questions instantly.
Loved reading this Blog? Share your valuable thoughts in the comment section.
About the Author
Hello, everyone! I’m Celina, an educationist and academic consultant. I enjoy reading, learning, and implementing the same in the assignments. I also take a lot of pride in guiding students with their assignments. I have 6 years of experience in writing assignments for students and when not doing that I follow my passion for blogging. I spent my spare time often researching new trends of writing.
FDI Policies in India Assessment Answers
Brand and branding element assessment answers, related posts.
You Won’t Believe Changing a Course in Australia is This Easy
December 15, 2021.
How do Indian Students see International Education? - A Western Union Report
January 03, 2022.
Best Study Techniques For Students To Get High Grades
December 06, 2021.
Trending now
How to write a Reflection Paragraph in an Assignment?
Project management assignment sample online, impact of western systems and structures on aboriginal and torres strait islander cultures, how to solve cubic equation ax^3+ bx^2+ cx+d = 0, how to write an executive summary (see our sample), best digital marketing assignment sample online, marketing assignment sample online, how to write gibbs reflective cycle, project proposal assignment sample online, business management assignment sample online, why are ethics important in research, looking for nursing leadership essay samples.
The Student Corner
Subscribe to get updates, offers and assignment tips right in your inbox.
Popular Posts
- Assignment Help How to write a Reflection Paragraph in an Assignment? Mike Carey June 16, 2018
- Management Project Management Assignment Sample Online Charles Hill October 04, 2017
- General Impact of Western Systems and Structures on Aboriginal and Torres Strait Islander Cultures Celina January 15, 2020
- Engineering How To Solve Cubic Equation Ax^3+ Bx^2+ Cx+D = 0 Chloe Kirner March 22, 2019
- Marketing How to Write an Executive Summary (See Our Sample) Celina May 11, 2018
- Marketing Best Digital Marketing Assignment Sample Online Andy Johnson July 20, 2018
Download Guide Book
- Australia(+61)
- Germany(+49)
- Hong kong(+8)
- Ireland(+353)
- Jordan(+962)
- Kenya(+254)
- Malaysia(+60)
- New zealand(+64)
- Nigeria(+234)
- Pakistan(+92)
- Saudi arabia(+966)
- Singapore(+65)
- South africa(+27)
- Sweden(+46)
- United arab emirates(+971)
- United kingdom(+44)
- United states america(+1)
Searching across 1 Million Resources...
Doing your Assignment with our resources is simple, take Expert assistance to ensure HD Grades. Here you Go....
Select Coupon
Literature Reviews: Types of Literature
- Library Basics
- 1. Choose Your Topic
- How to Find Books
- Types of Clinical Study Designs
Types of Literature
- 3. Search the Literature
- 4. Read & Analyze the Literature
- 5. Write the Review
- Keeping Track of Information
- Style Guides
- Books, Tutorials & Examples
Different types of publications have different characteristics.
Primary Literature Primary sources means original studies, based on direct observation, use of statistical records, interviews, or experimental methods, of actual practices or the actual impact of practices or policies. They are authored by researchers, contains original research data, and are usually published in a peer-reviewed journal. Primary literature may also include conference papers, pre-prints, or preliminary reports. Also called empirical research .
Secondary Literature Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature. Examples include review articles (such as meta-analysis and systematic reviews) and reference works. Professionals within each discipline take the primary literature and synthesize, generalize, and integrate new research.
Tertiary Literature Tertiary literature consists of a distillation and collection of primary and secondary sources such as textbooks, encyclopedia articles, and guidebooks or handbooks. The purpose of tertiary literature is to provide an overview of key research findings and an introduction to principles and practices within the discipline.
Adapted from the Information Services Department of the Library of the Health Sciences-Chicago , University of Illinois at Chicago.
Types of Scientific Publications
These examples and descriptions of publication types will give you an idea of how to use various works and why you would want to write a particular kind of paper.
- Scholarly article aka empirical article
- Review article
- Conference paper
Scholarly (aka empirical) article -- example
Empirical studies use data derived from observation or experiment. Original research papers (also called primary research articles) that describe empirical studies and their results are published in academic journals. Articles that report empirical research contain different sections which relate to the steps of the scientific method.
Abstract - The abstract provides a very brief summary of the research.
Introduction - The introduction sets the research in a context, which provides a review of related research and develops the hypotheses for the research.
Method - The method section describes how the research was conducted.
Results - The results section describes the outcomes of the study.
Discussion - The discussion section contains the interpretations and implications of the study.
References - A references section lists the articles, books, and other material cited in the report.
Review article -- example
A review article summarizes a particular field of study and places the recent research in context. It provides an overview and is an excellent introduction to a subject area. The references used in a review article are helpful as they lead to more in-depth research.
Many databases have limits or filters to search for review articles. You can also search by keywords like review article, survey, overview, summary, etc.
Conference proceedings, abstracts and reports -- example
Conference proceedings, abstracts and reports are not usually peer-reviewed. A conference article is similar to a scholarly article insofar as it is academic. Conference articles are published much more quickly than scholarly articles. You can find conference papers in many of the same places as scholarly articles.
How Do You Identify Empirical Articles?
To identify an article based on empirical research, look for the following characteristics:
The article is published in a peer-reviewed journal .
The article includes charts, graphs, or statistical analysis .
The article is substantial in size , likely to be more than 5 pages long.
The article contains the following parts (the exact terms may vary): abstract, introduction, method, results, discussion, references .
- << Previous: Types of Clinical Study Designs
- Next: 3. Search the Literature >>
- Last Updated: Mar 11, 2022 2:42 PM
- URL: https://research.library.gsu.edu/litrev
Literature Review Research
Literature review, what is not a literature review, purpose of the literature review, types of literature review.
- Systematic Review
- Literature vs Systematic Review
- Systematic vs Meta-Analysis
- Planning your Literature Review
- Finding information
- Helpful Tips
- Research Tips
- Other Resources
- Citing your Sources
Chat with us!
Education Librarian
- What is a Literature Review?
- Summarizes and analyzes previous research relevant to a topic
- Includes scholarly books and articles published in academic journals
- Can be an specific scholarly paper or a section in a research paper
- Help gather ideas or information
- Keep up to date in current trends and findings
- Help develop new questions
A literature review is important because it:
- Explains the background of research on a topic.
- Demonstrates why a topic is significant to a subject area.
- Helps focus your own research questions or problems
- Discovers relationships between research studies/ideas.
- Suggests unexplored ideas or populations
- Identifies major themes, concepts, and researchers on a topic.
- Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
- Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
- Indicates potential directions for future research.
Keep in mind the following, a literature review is NOT:
Not an essay
Not an annotated bibliography in which you summarize each article you reviewed. A literature review goes beyond basic summarizing to critically analyze the reviewed works and their relationship to your research question.
Not a research paper where you select resources to support one side of an issue versus another. A lit review should explain and consider all sides of an argument to avoid bias, and areas of agreement and disagreement should be highlighted.
A literature review serves several purposes. For example, it
- provides thorough knowledge of previous studies; introduces seminal works.
- helps focus one’s own research topic.
- identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
- suggests previously unused or underused methodologies, designs, and quantitative and qualitative strategies.
- identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
- helps the researcher avoid the repetition of earlier research.
- suggests unexplored populations.
- determines whether past studies agree or disagree; identifies controversy in the literature.
- tests assumptions may help counter preconceived ideas and remove unconscious bias.
As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.
Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:
Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.
Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.
Historical Review Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.
Methodological Review A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.
Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"
Theoretical Review The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.
* Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147.
All content in this section is from The Literature Review created by Dr. Robert Larabee USC
- Next: Systematic Review >>
- Last Updated: Dec 22, 2022 1:47 PM
- URL: https://guides.lib.odu.edu/literaturereview
Harvey Cushing/John Hay Whitney Medical Library
Ysn doctoral programs: steps in conducting a literature review.
- Biomedical Databases
- Global (Public Health) Databases
- Soc. Sci., History, and Law Databases
- Grey Literature
- Trials Registers
- Data and Statistics
- Public Policy
- Google Tips
- Recommended Books
- Steps in Conducting a Literature Review
What is a literature review?
A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question. That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.
A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment. Rely heavily on the guidelines your instructor has given you.
Why is it important?
A literature review is important because it:
- Explains the background of research on a topic.
- Demonstrates why a topic is significant to a subject area.
- Discovers relationships between research studies/ideas.
- Identifies major themes, concepts, and researchers on a topic.
- Identifies critical gaps and points of disagreement.
- Discusses further research questions that logically come out of the previous studies.
APA7 Style resources
APA Style Blog - for those harder to find answers
1. Choose a topic. Define your research question.
Your literature review should be guided by your central research question. the literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.
- Make sure your research question is not too broad or too narrow. Is it manageable?
- Begin writing down terms that are related to your question. These will be useful for searches later.
- If you have the opportunity, discuss your topic with your professor and your class mates.
2. Decide on the scope of your review
How many studies do you need to look at? How comprehensive should it be? How many years should it cover?
- This may depend on your assignment. How many sources does the assignment require?
3. Select the databases you will use to conduct your searches.
Make a list of the databases you will search.
Where to find databases:
- use the tabs on this guide
- Find other databases in the Nursing Information Resources web page
- More on the Medical Library web page
- ... and more on the Yale University Library web page
4. Conduct your searches to find the evidence. Keep track of your searches.
- Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
- Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
- Review the abstracts of research studies carefully. This will save you time.
- Use the bibliographies and references of research studies you find to locate others.
- Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
- Ask your librarian for help at any time.
- Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.
Review the literature
Some questions to help you analyze the research:
- What was the research question of the study you are reviewing? What were the authors trying to discover?
- Was the research funded by a source that could influence the findings?
- What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
- Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
- If there are conflicting studies, why do you think that is?
- How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?
Tips:
- Review the abstracts carefully.
- Keep careful notes so that you may track your thought processes during the research process.
- Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
- << Previous: Recommended Books
- Last Updated: Feb 23, 2023 10:49 AM
- URL: https://guides.library.yale.edu/YSNDoctoral
- USC Libraries
- Research Guides
Organizing Your Social Sciences Research Paper
- 5. The Literature Review
- Purpose of Guide
- Design Flaws to Avoid
- Independent and Dependent Variables
- Glossary of Research Terms
- Reading Research Effectively
- Narrowing a Topic Idea
- Broadening a Topic Idea
- Extending the Timeliness of a Topic Idea
- Academic Writing Style
- Choosing a Title
- Making an Outline
- Paragraph Development
- Research Process Video Series
- Executive Summary
- The C.A.R.S. Model
- Background Information
- The Research Problem/Question
- Theoretical Framework
- Citation Tracking
- Content Alert Services
- Evaluating Sources
- Primary Sources
- Secondary Sources
- Tiertiary Sources
- Scholarly vs. Popular Publications
- Qualitative Methods
- Quantitative Methods
- Insiderness
- Using Non-Textual Elements
- Limitations of the Study
- Common Grammar Mistakes
- Writing Concisely
- Avoiding Plagiarism
- Footnotes or Endnotes?
- Further Readings
- Bibliography
A literature review surveys books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have explored while researching a particular topic and to demonstrate to your readers how your research fits within a larger field of study.
Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.
Importance of a Good Literature Review
A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:
- Give a new interpretation of old material or combine new with old interpretations,
- Trace the intellectual progression of the field, including major debates,
- Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
- Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.
Given this, the purpose of a literature review is to:
- Place each work in the context of its contribution to understanding the research problem being studied.
- Describe the relationship of each work to the others under consideration.
- Identify new ways to interpret prior research.
- Reveal any gaps that exist in the literature.
- Resolve conflicts amongst seemingly contradictory previous studies.
- Identify areas of prior scholarship to prevent duplication of effort.
- Point the way in fulfilling a need for additional research.
- Locate your own research within the context of existing literature [very important].
Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.
Types of Literature Reviews
It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.
In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.
Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].
Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.
Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.
Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.
Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.
Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.
NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.
Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews." Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.
Structure and Writing Style
I. Thinking About Your Literature Review
The structure of a literature review should include the following in support of understanding the research problem :
- An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
- Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
- An explanation of how each work is similar to and how it varies from the others,
- Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.
The critical evaluation of each work should consider :
- Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
- Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
- Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
- Persuasiveness -- which of the author's theses are most convincing or least convincing?
- Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?
II. Development of the Literature Review
Four Basic Stages of Writing 1. Problem formulation -- which topic or field is being examined and what are its component issues? 2. Literature search -- finding materials relevant to the subject being explored. 3. Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4. Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.
Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1. Roughly how many sources would be appropriate to include? 2. What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3. Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4. Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5. Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.
III. Ways to Organize Your Literature Review
Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it will still be organized chronologically reflecting technological developments in media. The only difference here between a "chronological" and a "thematic" approach is what is emphasized the most: the role of the Internet in presidential politics. Note however that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point made. Note that this is the most common approach in the social and behavioral sciences. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.
Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.
Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:
- Current Situation : Information necessary to understand the current topic or focus of the literature review.
- Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
- History : The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
- Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed articles and journals.
- Standards : Description of the way in which you present your information.
- Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?
IV. Writing Your Literature Review
Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.
Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information but that are not key to understanding the research problem can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.
V. Common Mistakes to Avoid
These are the most common mistakes made in reviewing social science research literature.
- Sources in your literature review do not clearly relate to the research problem;
- You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
- Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
- Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
- Does not describe the search procedures that were used in identifying the literature to review;
- Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
- Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.
Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.
Writing Tip
Break Out of Your Disciplinary Box!
Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.
Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.
Another Writing Tip
Don't Just Review for Content!
While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:
- How are they organizing their ideas?
- What methods have they used to study the problem?
- What theories have been used to explain, predict, or understand their research problem?
- What sources have they cited to support their conclusions?
- How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?
When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.
Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.
Yet Another Writing Tip
When Do I Know I Can Stop Looking and Move On?
Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:
- Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research? Does it forge a new path? Or, does is merely add more of the same thing being said?
- Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
- Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.
Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.
- << Previous: Theoretical Framework
- Next: Citation Tracking >>
- Last Updated: Feb 16, 2023 1:36 PM
- URL: https://libguides.usc.edu/writingguide
Want to create or adapt OER like this? Learn how BCcampus supports open education and how you can access Pressbooks . Learn more about how Pressbooks supports open publishing practices. -->
Chapter 5: The Literature Review
5.3 Acceptable sources for literature reviews
Following are a few acceptable sources for literature reviews, listed in order from what will be considered most acceptable to less acceptable sources for your literature review assignments:
- Peer reviewed journal articles.
- Edited academic books.
- Articles in professional journals.
- Statistical data from government websites.
- Website material from professional associations (use sparingly and carefully). The following sections will explain and provide examples of these various sources.
Peer reviewed journal articles (papers)
A peer reviewed journal article is a paper that has been submitted to a scholarly journal, accepted, and published. Peer review journal papers go through a rigorous, blind review process of peer review. What this means is that two to three experts in the area of research featured in the paper have reviewed and accepted the paper for publication. The names of the author(s) who are seeking to publish the research have been removed (blind review), so as to minimize any bias towards the authors of the research (albeit, sometimes a savvy reviewer can discern who has done the research based upon previous publications, etc.). This blind review process can be long (often 12 to 18 months) and may involve many back and forth edits on the behalf of the researchers, as they work to address the edits and concerns of the peers who reviewed their paper. Often, reviewers will reject the paper for a variety of reasons, such as unclear or questionable methods, lack of contribution to the field, etc. Because peer reviewed journal articles have gone through a rigorous process of review, they are considered to be the premier source for research. Peer reviewed journal articles should serve as the foundation for your literature review.
The following link will provide more information on peer reviewed journal articles. Make sure you watch the little video on the upper left-hand side of your screen, in addition to reading the material at the following website: http://guides.lib.jjay.cuny.edu/c.php?g=288333&p=1922599
Edited academic books
An edited academic book is a collection of scholarly scientific papers written by different authors. The works are original papers, not published elsewhere (“Edited volume,” 2018). The papers within the text also go through a process of review; however, the review is often not a blind review because the authors have been invited to contribute to the book. Consequently, edited academic books are fine to use for your literature review, but you also want to ensure that your literature review contains mostly peer reviewed journal papers.
Articles in professional journals
Articles from professional journals should be used with caution for your literature review. This is because articles in trade journals are not usually peer reviewed, even though they may appear to be. A good way to find out is to read the “About Us” section of the professional journal, which should state whether or not the papers are peer reviewed. You can also find out by Googling the name of the journal and adding “peer reviewed” to the search.
Statistical data from governmental websites
Governmental websites can be excellent sources for statistical data, e.g, Statistics Canada collects and publishes data related to the economy, society, and the environment (see https://www.statcan.gc.ca/eng/start ).
Website material from professional associations
Material from other websites can also serve as a source for statistics that you may need for your literature review. Since you want to justify the value of the research that interests you, you might make use of a professional association’s website to learn how many members they have, for example. You might want to demonstrate, as part of the introduction to your literature review, why more research on the topic of PTSD in police officers is important. You could use peer reviewed journal articles to determine the prevalence of PTSD in police officers in Canada in the last ten years, and then use the Ontario Police Officers´ Association website to determine the approximate number of police officers employed in the Province of Ontario over the last ten years. This might help you estimate how many police officers could be suffering with PTSD in Ontario. That number could potentially help to justify a research grant down the road. But again, this type of website- based material should be used with caution and sparingly.
Research Methods for the Social Sciences: An Introduction by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.
Share This Book
Types of Literature Reviews
Strategies for getting started, composition guidelines, how to locate reviews by discipline.
We also provide the following:
Types of Literature Reviews [Refresher]
The previous page provided an introduction to literature reviews and guidelines for determining the scope and purpose of your review. Next, we take a look at the different types of literature reviews and why a researcher might select one type over another.
A literature review helps your reader understand the relationship of your research to the scholarly work of others. Not all humanities research projects contain literature reviews, but many do. Keep in mind that the type of literature review you choose (see list below) pertains to the secondary research – other scholarly sources – and not to the primary literary work. For instance, an argumentative literature review presents and takes sides in scholarly arguments about the literary work. It makes arguments about other scholars’ work. It does not necessarily involve a claim that the literary work is itself making an argument. Likewise, a chronological literature review presents the secondary literature in chronological order.
If your research involves two or more subject areas, such as psychology and genre studies, you may need to create multiple literature reviews, one for each subject area. If the subject areas overlap with each other significantly (i.e., Marxism and Cultural Studies), you may combine them. Your literature review need not include everything about the subject area – you would need to write a book to cover a single theory – but only those concepts and methods that are most relevant to your research question.
Factors to Consider When Developing Your Literature Review
- Establish Criteria : We discussed the importance of defining the purpose and scope of your review on the previous page, but it’s worth reviewing here as well. This step will help you establish important criteria and focus your searching. For example, how many sources will you need? What types of sources (primary, secondary, statistics, media)? Is currency important? Do you know who the prominent authors or theorists are in your subject area? Take some time to map out these or other important factors before you begin searching journals and databases.
- Consider Your Audience : Unlike a work cited page or an annotated bibliography, both of which are lists of sources, a literature review is essayistic and can be considered a precursor to your final paper. therefore, it should be written in your own voice, and it should be geared toward a specific audience. Considering audience during this early stage will help focus your final paper as well.
- Find Models : We’ll discuss the different types of literature reviews and how to locate examples in the section below. However, even if you’re undecided about what type of review will work best for you, you may want to review some example literature reviews to get a sense of what they look like before you begin your own.
Maria J. Grant and Andrew Booth’s “A Typology of Reviews” identifies 14 distinct types of literature reviews. Further, the UCLA library created a chart to complement the article and for easy comparison of those 14 types of reviews. This section provides a brief summary of the most common literature reviews. For a more complete analysis, please see the full article and the chart .
To choose the most appropriate structure, put yourself in your reader’s shoes and think through their need for information. The literature review is about providing context for your contribution. How much context do people need? Keep it to the minimum necessary. These structures are not meant to be straightjackets but tools to help you organize your research. If you find that the tool is working, then keep using it. If not, switch tools or modify the one you are using. Keep in mind that the types of literature reviews are just different ways of organizing information. So, you can discuss literary trends without organizing your review of secondary literature by trend. It could be organized by theory or theme, for examples. In our literature reviews, we are not recounting other scholars’ arguments at length but merely providing key concepts so we can summarize the discussion so far and position our own claims.
Types of Reviews
- Traditional or narrative reviews : This approach will generate a comprehensive, critical analysis of the published research on your topic. However, rather than merely compiling as many sources as possible, use this approach to establish a theoretical framework for your paper, establish trends, and identify gaps in the research. This process should bring your research question into clearer focus and help define a thesis that you will argue for in your paper. This is perhaps the most common and general type of literature review. The examples listed below are all designed to serve a more specific purpose.
- Argumentative : The purpose of an argumentative literature review is to select sources for the purpose of supporting or refuting a specific claim. While this type of review can help the author make a strong case for or against an issue, they can also be prone to claims of bias.
- Chronological : A chronological review is used when the author wants to demonstrate the progression of how a theory, methodology, or issue has progressed over time. This method is most effective when there is a clear chronological path to the research about a specific historical event or trend as opposed to a more recursive theoretical concept.
- By trend : This is similar to the chronological approach except it focuses on clearly-defined trends rather than date ranges. This would be most appropriate if you want to illustrate changing perspectives or attitudes about a given issue when specific date ranges are less important than the ebb and flow of the trend.
- Thematic : In this type of literature review, the author will select specific themes that he or she feels are important to understanding a larger topic or concept. Then, the author will organize the sources around those themes, which are often based on relevance or importance. The value of this method is that the process of organizing the review by theme is similar to constructing an argument. This can help the author see how resources connect to each other and determine how as well as why specific sources support their thesis.
- Theoretical : The goal of this type of review is to examine how theory has shaped the research on a given topic. It establishes existing theoretical models, their connections, and how extensively they have been developed in the published research. For example, Jada applied critical race theory to her analysis of Sonny’s Blues , but she might also consider conducting a more comprehensive review of other theoretical frameworks such as feminism, Marxism, or postmodernism. Doing so could provide insight into alternate readings, and help her identify theoretical gaps such as unexplored or under-developed approaches to Baldwin’s work.
- Methodological : The approach focuses on the various methods used by researchers in a specific area rather than an analysis of their findings. In this case, you would create a framework of approaches to data collection related to your topic or research question. This is perhaps more common in education or the social and hard sciences where published research often includes a methods section, but it is sometimes appropriate for the digital humanities as well.
- Scoping : The aim of a scoping review is to provide a comprehensive overview or map of the published research or evidence related to a research question. This might be considered a prelude to a systematic review that would take the scoping review one step further toward answering a clearly defined research question. See below for more details.
- Systematic : The systematic review is most appropriate when you have a clearly-defined research question and have established criteria for the types of sources you need. In this way, the systematic review is less exploratory than other types of reviews. Rather, it is comprehensive, strategic, and focused on answering a specific research question. For this reason, the systematic review is more common in the health and social sciences, where comprehensiveness is more important than interpretation, than the humanities.
- Meta-analysis : Does your research deal with statistics or large amounts of data? If so, then a meta-analysis might be best for you. rather than providing a critical review, the meta-analysis will summarize and synthesize the results of numerous studies that involve statistics or data to provide a more comprehensive picture than would be possible from just one study.
You don’t need to keep strictly to one type. Scholars often combine features from various types of literature reviews.
When writing your literature review, please follow these pointers:
- Conduct systematic searches
- Use Evidence
- Be Selective
- Use Quotes Sparingly
- Summarize & Synthesize
- Use Caution when Paraphrasing
- Use Your Own Voice
Literature reviews can be published as part of a scholarly article, often after the introduction and sometimes with a header, but they can also be published as a standalone essay. To find examples of what reviews look like in your discipline, choose an appropriate subject database (such as MLA for literary criticism) and conduct a keyword search with the term “Literature Review” added in quotes:
Not only do these examples demonstrate how to structure different types of literature reviews, but some offer insights into trends and directions for future research. In the next section, we’ll take a closer look at some reading strategies to help guide you through this process.
- What types of literature review will you be using for your paper? Why did you make this selection over others? If you haven’t made a selection yet, which types are you considering?
- What specific challenges do you face in following a literature review structure?
- If there are any elements of your assignment that need clarification, please list them.
- What was the most important lesson you learned from this page? What point was confusing or difficult to understand?
Strategies for Conducting Literary Research, 2e by Barry Mauer & John Venecek is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.
Share This Book
- University of Michigan Library
- Research Guides
The Literature Review: For Dissertations
Literature to review.
- Introduction
- Getting Started
- Copyright Issues
- Guide Books
- Books can provide reference information, broad introductions to topics, in-depth treatments of specific topics, and can even serve as primary source material, depending on their content.
- Books can be scholarly or popular, so consider what type of information you need when selecting books for review.
- Books can be found in catalogs such as the U-M Library catalog and Worldcat , a union catalog containing millions of records cataloged by OCLC member libraries.
- Catalogs can be searched by author name, titles words or phrases, Library of Congress Subject Headings, Keywords, ISBN numbers, and other identifiers.
- Books not available locally can be requested via Interlibrary Loan .
Reference Materials
- Reference materials contain reputable background and quick information on specific topics and are rarely read front to back.
- Examples of reference works include dictionaries, encyclopedias, atlases, standards, cookbooks, handbooks, directories, etc.
- References can orient users to basic information, to experts in a particular discipline, and to specialized vocabulary used in particular disciplines. Reference works are rarely cited in academic literature.
Scholarly Journal Articles
- Scholarly journal articles are written by scholars to communicate information with other experts in the same field or discipline, and often use specialized language used by scholars in the same field or discpline.
- Scholarly articles are produced in a standard, formal format and usually list the scholarly affiliations of the authors
- They almost always include references to literature consulted by the authors
- They are often peer-reviewed and edited by editors with scholarly credentials
- They can be found by searching appropriate library article indexes (such as Web of Science ) or discovery tools (such as U-M Library Articles Search ), by browsing the contents of scholarly journals that serve the needs of a particular topic or discipline, or by reading through the cited references at the end of other articles.
- Patents are documents that describe a person's or an organization's legal right to exclusively use or create an invention.
- The Patents and Trademarks Research Guide explains how to search for patents.
- Data sets are collections of data that are often collected during the course of scholarly research. Data can be spatial and/or numeric, raw or processed, and qualitative or quantitative.
- Tools for finding data are included in several Library Research Guides , open access data repositories like Open Access Directory , and the Data Citation Index .
Archival Collections
- Archival materials may include personal papers, manuscripts, diaries, posters, fliers, maps and other primary sources. Locating archives and finding materials available takes time, patiences and many times help from a librarian. The University of Michigan Library subscribes to many online archive finding tools and resources . For help locating archives consult the Library Research Guide Locating Archives and Manuscripts .
- The University of Michigan Library provides access to over a thousand newspapers in a variety of formats. Some recent news is available temporarily in print but most newspapers are either available in microfilm or online. These include international newspapers and historical newspapers. See the library page on newspapers for more information and consult the Library Research Guide News Sources for extensive information on finding news.
- Magazines can be a rich source of cultural and anthropological research material. The library offers many magazines in print, microfilm and online. See the Library Research Guide The History of Magazines in America for further help.
Government documents
- The United States Government Printing Office has and continues to produce a great deal of information useful to researchers each day. Congress, the Supreme Court, the Office of the President and federal agencies can be rich sources of policy information, legislation and historical records. University of Michigan's Clark Library is a congressionally designated federal depository library. Librarians there can help you find documents and records created by the U.S. federal government.
- Clark Library is also an excellent place to get help finding state and local laws and legislation.
- International government information can be found in United Nations documents which are available in print and online from the library since 1946.
- Statistics reported by government or private sources can be useful when your dissertation relates to social issues. Sometimes, you need these statistics to illustrate the importance of your work.
- Visit Finding Statistics Guide for sources and tips.
Theses & Dissertations
- Dissertations on similar topics to yours may contain information and technical details not published in other forms. You may also get inspired by how others approach similar topics during their graduate career.
- Search for theses and dissertations via ProQuest Dissertations & Theses (US, UK, Canada, Ireland etc. ) or WorldCat Dissertations and Theses (North America, Australia, and Western European countries) and other dissertation databases we subscribe.
- In recent years, many North American universities start to post dissertations in their institutional repository. For example, you may find recent Univ of Michigan dissertations in our institutional Repository- Deep Blue Dissertations and Theses . Sometimes, you may find access to them through a Google search.
Conference proceedings
- For many fields, researchers present their most up-to-date research results at professional conferences. These results will later be published in conference proceedings, abstracts, or preprints. Sometimes, the conference proceedings can be full research articles as in many engineering and computer science areas. Conference Proceedings should often be consulted for latest development on your topic.
- Search for Conference proceedings via Web of Science , ProQuest Databases or Scopus by limiting the type of documents as Conference Proceedings etc.
Grey literature
- Grey literature is a blanket term for hard to find documents not formally published for public consumption (examples include technical reports, conference literature, white papers, memos and meeting minutes), not indexed in conventional indexing tools.
- Grey literature can be invaluable for the research you are doing.
- Visit the Grey Literature Research Guide to see how you can use them in your research.
Citation Analysis
Experts attempt to weigh the value of scholarly literature relative to other scholarly works. Some of this work is done through citation analysis, where formulas are designed to rank impact. The following guide has been created to explain current citation analysis practices.
An official website of the United States government
Here's how you know
The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.
The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.
Literature Search: Databases and Gray Literature
The literature search.
- A systematic review search includes a search of databases, gray literature, personal communications, and a handsearch of high impact journals in the related field. See our list of recommended databases and gray literature sources on this page.
- a comprehensive literature search can not be dependent on a single database, nor on bibliographic databases only.
- inclusion of multiple databases helps avoid publication bias (georaphic bias or bias against publication of negative results).
- The Cochrane Collaboration recommends PubMed, Embase and the Cochrane Central Register of Controlled Trials (CENTRAL) at a minimum.
- NOTE: The Cochrane Collaboration and the IOM recommend that the literature search be conducted by librarians or persons with extensive literature search experience. Please contact the NIH Librarians for assistance with the literature search component of your systematic review.
A collection of six databases that contain different types of high-quality, independent evidence to inform healthcare decision-making. Search the Cochrane Central Register of Controlled Trials here.
European database of biomedical and pharmacologic literature.
PubMed comprises more than 21 million citations for biomedical literature from MEDLINE, life science journals, and online books.
Largest abstract and citation database of peer-reviewed literature and quality web sources. Contains conference papers.
World's leading citation databases. Covers over 12,000 of the highest impact journals worldwide, including Open Access journals and over 150,000 conference proceedings. Coverage in the sciences, social sciences, arts, and humanities, with coverage to 1900.
Subject Specific Databases
Over 4.5 million abstracts of peer-reviewed literature in the behavioral and social sciences. Includes conference papers, book chapters, psychological tests, scales and measurement tools.
Comprehensive journal index to nursing and allied health literature, includes books, nursing dissertations, conference proceedings, practice standards and book chapters.
Latin American and Caribbean health sciences literature database
Gray Literature
- Gray Literature is the term for information that falls outside the mainstream of published journal and mongraph literature, not controlled by commercial publishers
- hard to find studies, reports, or dissertations
- conference abstracts or papers
- governmental or private sector research
- clinical trials - ongoing or unpublished
- experts and researchers in the field
- Library catalogs
- Professional association websites
- WorldCat - 1.5 billion items in this collection of library catalogs
- Google Scholar - Search scholarly literature across many disciplines and sources, including theses, books, abstracts and articles.
- Dissertation Abstracts - dissertation and theses database - NIH Library biomedical librarians can access and search for you.
- NTIS - central resource for government-funded scientific, technical, engineering, and business related information.
- AHRQ - agency for healthcare research and quality
- Open Grey - system for information on grey literature in Europe. Open access to 700,000 references to the grey literature.
- World Health Organization - providing leadership on global health matters, shaping the health research agenda, setting norms and standards, articulating evidence-based policy options, providing technical support to countries and monitoring and assessing health trends.
- New York Academy of Medicine Grey Literature Report - a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new gray literature publications in health services research and selected public health topics. NOTE: Discontinued as of Jan 2017, but resources are still accessible.
- Gray Source Index
- OpenDOAR - directory of academic repositories
Clinical Trial Registries
- International Clinical Trials Registery Platform - from the World Health Organization
- Australian New Zealand Clinical Trials Registry
- Brazilian Clinical Trials Registry
- Chinese Clinical Trial Registry -
- ClinicalTrials.gov - U.S. and international federally and privately supported clinical trials registry and results database
- Clinical Trials Registry - India
- EU clinical Trials Register
- Japan Primary Registries Network
- Pan African Clinical Trials Registry
Literature Review
- What is a literature review?
- What is its purpose?
- 1. Selecting your topic
- 2. Setting the topic in context
- 3. Looking at information sources
- 4. Using information sources
- 5. Getting the information
- 6. Organizing information (information management)
- 7. Positioning the literature review
- 8. Writing the literature review
About this guide
This research guide was developed for students at Bloomsburg University of Pennsylvania.
If you are a student from another school, you are welcome to peruse the guide, using the links above, but please know that our librarians can only provide general help to non-BU students. Contact the librarians at your own institution for help in using the resources available to you.
-Andruss Library
A literature review is a comprehensive summary of previous research on a topic. The literature review surveys scholarly articles, books, and other sources relevant to a particular area of research. The review should enumerate, describe, summarize, objectively evaluate and clarify this previous research. It should give a theoretical base for the research and help you (the author) determine the nature of your research. The literature review acknowledges the work of previous researchers, and in so doing, assures the reader that your work has been well conceived. It is assumed that by mentioning a previous work in the field of study, that the author has read, evaluated, and assimiliated that work into the work at hand.
A literature review creates a "landscape" for the reader, giving her or him a full understanding of the developments in the field. This landscape informs the reader that the author has indeed assimilated all (or the vast majority of) previous, significant works in the field into her or his research.
"In writing the literature review, the purpose is to convey to the reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. The literature review must be defined by a guiding concept (eg. your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries.( http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review )
Recommended Reading
Acknowledgements
Many thanks to Kate Houston and Libbie Blanchard of CQ University Libraries, (Queensland, Australia) whose LibGuide on the Literature Review served as a framework for this guide.
Designed and updated by Michael Coffta
- Next: What is its purpose? >>
- Last Updated: Feb 24, 2023 1:50 PM
- URL: https://guides.library.bloomu.edu/litreview
Andruss Library | 570-389-4205 | [email protected] ©Copyright Bloomsburg University of Pennsylvania • 400 East Second Street, Bloomsburg PA 17815-1301 • 570.389.4000
- University of La Verne
- Subject Guides
Literature Review Basics
- Primary & Secondary Sources
- Literature Review Introduction
- Writing Literature Reviews
- Tutorials & Samples
The Literature
The Literature refers to the collection of scholarly writings on a topic. This includes peer-reviewed articles, books, dissertations and conference papers.
- When reviewing the literature, be sure to include major works as well as studies that respond to major works. You will want to focus on primary sources, though secondary sources can be valuable as well.
Primary Sources
The term primary source is used broadly to embody all sources that are original. P rimary sources provide first-hand information that is closest to the object of study. Primary sources vary by discipline.
- In the natural and social sciences, original reports of research found in academic journals detailing the methodology used in the research, in-depth descriptions, and discussions of the findings are considered primary sources of information.
- Other common examples of primary sources include speeches, letters, diaries, autobiographies, interviews, official reports, court records, artifacts, photographs, and drawings.
Galvan, J. L. (2013). Writing literature reviews: A guide for students of the social and behavioral sciences . Glendale, CA: Pyrczak.
Persaud, N. (2010). Primary data source. In N. Salkind (Ed.), Encyclopedia of research design. (pp. 1095-1098). Thousand Oaks, CA: SAGE Publications, Inc.
Secondary Sources
A secondary source is a source that provides non-original or secondhand data or information.
- Secondary sources are written about primary sources.
- Research summaries reported in textbooks, magazines, and newspapers are considered secondary sources. They typically provide global descriptions of results with few details on the methodology. Other examples of secondary sources include biographies and critical studies of an author's work.
Secondary Source. (2005). In W. Paul Vogt (Ed.), Dictionary of Statistics & Methodology. (3 rd ed., p. 291). Thousand Oaks, CA: SAGE Publications, Inc.
Weidenborner, S., & Caruso, D. (1997). Writing research papers: A guide to the process . New York: St. Martin's Press.
More Examples of Primary and Secondary Sources
- << Previous: Writing Literature Reviews
- Next: Tutorials & Samples >>
- Last Updated: Feb 8, 2023 11:29 AM
- URL: https://laverne.libguides.com/litreviews
Information
- Author Services
Initiatives
You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.
All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.
Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.
Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.
- Active Journals
- Find a Journal
- Proceedings Series
- For Authors
- For Reviewers
- For Editors
- For Librarians
- For Publishers
- For Societies
- For Conference Organizers
- Open Access Policy
- Institutional Open Access Program
- Special Issues Guidelines
- Editorial Process
- Research and Publication Ethics
- Article Processing Charges
- Testimonials
- Preprints.org
- SciProfiles
- Encyclopedia
Article Menu
- Recommended Articles
- Google Scholar
- on Google Scholar
- Table of Contents
Find support for a specific problem in the support section of our website.
Please let us know what you think of our products and services.
Visit our dedicated information section to learn more about MDPI.
JSmol Viewer
The interest shown by potential young entrepreneurs in romania regarding feasible funding sources, in the context of a sustainable entrepreneurial education.
1. Introduction
2. literature review, 2.1. entrepreneurial education and sustainable development, 2.2. sources of funding, 2.3. sources of funding, 3. research method, 3.1. questionnaire development.
- RQ1. What is the level of knowledge possessed by the respondent in the field of entrepreneurship (part of sustainable entrepreneurial education entrepreneurship education)?
- RQ2. What are the main sources of financing known to the respondents?
- RQ3. What are the respondents’ attitudes and opinions regarding the different types of financing sources?
- RQ4. What is the future entrepreneurial intention regarding the establishment of a new business?
3.2. Data Collection, and Location of the Survey
3.3. study sample, 3.4. data analysis techniques, 4. results and discussions.
- RQ1. What is the level of knowledge possessed by the respondent in the field of entrepreneurship (part of sustainable entrepreneurial education entrepreneurship education) ?
Principal Component Analysis (PCA)
5. conclusions, 6. limitations and further research, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.
- Ministry of Investments and European Projects Sinteza Programelor Operaționale 2021–2027. Available online: https://mfe.gov.ro/sinteza-programelor-operationale-2021-2027/ (accessed on 24 October 2022).
- European Commission Report on an EU SME Referral Scheme. Available online: https://finance.ec.europa.eu/system/files/2022-07/220628-report-sme-referral-scheme_en.pdf (accessed on 23 October 2022).
- Etzkowitz, H.; Webster, A.; Gebhardt, C.; Terra, B.R.C. The Future of the University and the University of the Future: Evolution of Ivory Tower to Entrepreneurial Paradigm. Res. Policy 2000 , 29 , 313–330. [ Google Scholar ] [ CrossRef ]
- United Nation THE 17 GOALS Sustainable Development. Available online: https://sdgs.un.org/goals (accessed on 31 January 2023).
- Cumming, D.; Groh, A.P. Entrepreneurial Finance: Unifying Themes and Future Directions. J. Corp. Financ. 2018 , 50 , 538–555. [ Google Scholar ] [ CrossRef ]
- Pticar, S. Financing as One of the Key Success Factors of Small and Medium-Sized Enterprises. Creat. Knowl. Soc. 2016 , 6 , 36–47. [ Google Scholar ] [ CrossRef ]
- Ministry of Justice. National Trade Register Office Oficiul National al Registrului Comertului. Available online: https://www.onrc.ro/index.php/ro/statistici (accessed on 24 October 2022).
- Kang, Q.; Li, H.; Cheng, Y.; Kraus, S. Entrepreneurial Ecosystems: Analysing the Status Quo. Knowl. Manag. Res. Pract. 2021 , 19 , 8–20. [ Google Scholar ] [ CrossRef ]
- Gupta, V.K.; Turban, D.B.; Bhawe, N.M. The Effect of Gender Stereotype Activation on Entrepreneurial Intentions. J. Appl. Psychol. 2008 , 93 , 1053–1061. [ Google Scholar ] [ CrossRef ]
- Gupta, V.K.; Goktan, A.B.; Gunay, G. Gender Differences in Evaluation of New Business Opportunity: A Stereotype Threat Perspective. J. Bus. Ventur. 2014 , 29 , 273–288. [ Google Scholar ] [ CrossRef ]
- Langowitz, N.; Minniti, M. The Entrepreneurial Propensity of Women. Entrep. Theory Pract. 2007 , 31 , 341–364. [ Google Scholar ] [ CrossRef ]
- Amway 2020 AGER Brochure. Available online: https://www.amwayglobal.com/wp-content/uploads/2020/12/2020_AGER__Brochure_ENG1.pdf (accessed on 30 January 2023).
- Amway 2018 AGER Brochure. Available online: https://www.amwayglobal.com/wp-content/uploads/2018/05/Ager_2018_Brochure_Color.pdf (accessed on 24 October 2022).
- Edokpolor, J.E. Entrepreneurship Education and Sustainable Development: Mediating Role of Entrepreneurial Skills. Asia Pac. J. Innov. Entrep. 2020 , 14 , 329–339. [ Google Scholar ] [ CrossRef ]
- Rădulescu, C.V.; Burlacu, S.; Bodislav, D.A.; Bran, F. Entrepreneurial Education in the Context of the Imperative Development of Sustainable Business. Eur. J. Sustain. Dev. 2020 , 9 , 93. [ Google Scholar ] [ CrossRef ]
- Del Vecchio, P.; Secundo, G.; Mele, G.; Passiante, G. Sustainable Entrepreneurship Education for Circular Economy: Emerging Perspectives in Europe. Int. J. Entrep. Behav. Res. 2021 , 27 , 2096–2124. [ Google Scholar ] [ CrossRef ]
- Cui, J. The Impact of Entrepreneurship Curriculum with Teaching Models on Sustainable Development of Entrepreneurial Mindset among Higher Education Students in China: The Moderating Role of the Entrepreneurial Climate at the Institution. Sustainability 2021 , 13 , 7950. [ Google Scholar ] [ CrossRef ]
- Rashid, L. Entrepreneurship Education and Sustainable Development Goals: A Literature Review and a Closer Look at Fragile States and Technology-Enabled Approaches. Sustainability 2019 , 11 , 5343. [ Google Scholar ] [ CrossRef ]
- Egorov, E.E.; Lebedeva, T.E.; Prokhorova, M.P.; Shobonova, L.Y.; Bulganina, S.V. Youth Entrepreneurship: Motivational Aspects and Economic Effects. IOP Conf. Ser. Earth Environ. Sci 2019 , 272 , 032129. [ Google Scholar ] [ CrossRef ]
- Blokker, P.; Dallago, B. Introduction Economic Transformation and the Challenge of Youth Entrepreneurship in Central and Eastern Europe. In Youth Entrepreneurship and Local Development in Central and Eastern Europe ; Routledge: London, UK, 2017; pp. 1–28. [ Google Scholar ]
- Karadzic, V.; Drobnjak, R.; Reyhani, M. Opportunities and Challenges in Promoting Youth Entrepreneurship in Montenegro. Int. J. Bus. Econ. Sci. Appl. Res. 2015 , 8 , 13–20. [ Google Scholar ]
- Frimanslund, T.; Kwiatkowski, G.; Oklevik, O. The Role of Finance in the Literature of Entrepreneurial Ecosystems. Eur. Plan. Stud. 2023 , 31 , 372–391. [ Google Scholar ] [ CrossRef ]
- Geldhof, G.J.; Malin, H.; Johnson, S.K.; Porter, T.; Bronk, K.C.; Weiner, M.B.; Agans, J.P.; Mueller, M.K.; Hunt, D.; Colby, A.; et al. Entrepreneurship in Young Adults: Initial Findings from the Young Entrepreneurs Study. J. Appl. Dev. Psychol. 2014 , 35 , 410–421. [ Google Scholar ] [ CrossRef ]
- Cull, J. Mentoring Young Entrepreneurs: What Leads to Success. Int. J. Evid. Based Coach. Mentor. 2006 , 4 , 8–18. [ Google Scholar ]
- Shaw, K.; Sørensen, A. Coming of Age: Watching Young Entrepreneurs Become Successful. Labour Econ. 2022 , 77 , 102033. [ Google Scholar ] [ CrossRef ]
- Bruhn, M.; Zia, B. Stimulating Managerial Capital in Emerging Markets: The Impact of Business and Financial Literacy for Young Entrepreneurs; The World Bank. 2011. Available online: https://elibrary.worldbank.org/doi/abs/10.1596/1813-9450-5642 (accessed on 31 January 2023).
- Yoganandan, G. Challenges of Young Entrepreneurs. Asia Pac. J. Res. 2017 , 1 , 112–115. [ Google Scholar ]
- Boissin, J.-P.; Branchet, B.; Emin, S.; Herbert, J.I. Students and Entrepreneurship: A Comparative Study of France and the United States. J. Small Bus. Entrep. 2009 , 22 , 101–122. [ Google Scholar ] [ CrossRef ]
- Ragazou, K.; Passas, I.; Garefalakis, A.; Kourgiantakis, M.; Xanthos, G. Youth’s Entrepreneurial Intention: A Multinomial Logistic Regression Analysis of the Factors Influencing Greek HEI Students in Time of Crisis. Sustainability 2022 , 14 , 13164. [ Google Scholar ] [ CrossRef ]
- Bergmann, H.; Hundt, C.; Sternberg, R. What Makes Student Entrepreneurs? On the Relevance (and Irrelevance) of the University and the Regional Context for Student Start-Ups. Small Bus. Econ. 2016 , 47 , 53–76. [ Google Scholar ] [ CrossRef ]
- VOCEDplus. Youth Entrepreneurship|VOCEDplus, the International Tertiary Education and Research Database. Available online: https://www.voced.edu.au/content/ngv%3A71599 (accessed on 31 January 2023).
- Jakubczak, J. Youth Entrepreneurship Barriers and Role of Education in Their Overcoming-Pilot Study. Manag. Intellect. Cap. Manag. Innov. Sustain. Knowl. Learn. Incl. Soc. 2015 , 1775–1782. Available online: https://www.semanticscholar.org/paper/Youth-Entrepreneurship-Barriers-and-Role-of-in-Jakubczak/63fce59956d043b791ad1129ec493cc15ec5c638 (accessed on 31 January 2023).
- Liu, X.; Lin, C.; Zhao, G.; Zhao, D. Research on the Effects of Entrepreneurial Education and Entrepreneurial Self-Efficacy on College Students’ Entrepreneurial Intention. Front. Psychol. 2019 , 10 , 869. [ Google Scholar ] [ CrossRef ]
- Hussain, A.; Hashim, N. Impact of Entrepreneurial Education on Entrepreneurial Intentions of Pakistani Students. J. Entrep. Bus. Innov. 2015 , 2 , 43. [ Google Scholar ] [ CrossRef ]
- Bauman, A.; Lucy, C. Enhancing Entrepreneurial Education: Developing Competencies for Success. Int. J. Manag. Educ. 2021 , 19 , 100293. [ Google Scholar ] [ CrossRef ]
- Temoor, A.; Sara, R.R.; Muhammad, F.; Shenzhen, M.B.; Valliappan, R.; Nida, N.; Imran, A.S. Entrepreneurial Intentions of Pakistani Students: The Role of Entrepreneurial Education, Creativity Disposition, Invention Passion & Passion for Founding. J. Manag. Res. 2018 , 10 , 76. [ Google Scholar ] [ CrossRef ]
- Karyaningsih, R.P.D.; Wibowo, A.; Saptono, A.; Narmaditya, B.S. Does Entrepreneurial Knowledge Influence Vocational Students’ Intention? Lessons from Indonesia. Entrep. Bus. Econ. Rev. 2020 , 8 , 138–155. [ Google Scholar ] [ CrossRef ]
- Owen, R.; Haddock-Millar, J.; Sepulveda, L.; Sanyal, C.; Syrett, S.; Kaye, N.; Deakins, D. Chapter 7 The Role of Mentoring in Youth Entrepreneurship Finance: A Global Perspective. In Creating Entrepreneurial Space: Talking through Multi-Voices, Reflections on Emerging Debates Contemporary Issues in Entrepreneurship Research ; Emerald: Bingley, UK, 2019; pp. 115–135. [ Google Scholar ]
- Shrotriya, D.V. Internal Sources of Finance for Business Organizations. Int. J. Res. Anal. Rev. 2019 , 6 , 933–940. [ Google Scholar ]
- Hossain, M.; Oparaocha, G.O. Crowdfunding: Motives, Definitions, Typology and Ethical Challenges. Entrep. Res. J. 2017 , 7 , 20150045. [ Google Scholar ] [ CrossRef ]
- Bellavitis, C.; Filatotchev, I.; Kamuriwo, D.S.; Vanacker, T. Entrepreneurial Finance: New Frontiers of Research and Practice. Ventur. Cap. 2017 , 19 , 1–16. [ Google Scholar ] [ CrossRef ]
- Bonini, S.; Capizzi, V.; Cumming, D. Emerging Trends in Entrepreneurial Finance. Ventur. Cap. 2019 , 21 , 133–136. [ Google Scholar ] [ CrossRef ]
- Aimurzina, B.; Kamenova, M.; Omarova, A.; Bodaubayeva, G.; Dzhunusova, A.; Kabdullina, G. Major Sources of Financing Investment Projects. Entrep. Sustain. Issues 2019 , 7 , 1555–1566. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Ashourizadeh, S.; Saeedikiya, M.; Aeeni, Z.; Temiz, S. Formal Sources of Finance Boost Innovation: Do Immigrants Benefit as Much as Natives? Entrep. Bus. Econ. Rev. 2022 , 10 , 41–59. [ Google Scholar ] [ CrossRef ]
- Beck, T.; Demirguckunt, A.; Maksimovic, V. Financing Patterns around the World: Are Small Firms Different? J. Financ. Econ. 2008 , 89 , 467–487. [ Google Scholar ] [ CrossRef ]
- Cumming, D.; Johan, S. The Problems with and Promise of Entrepreneurial Finance. Strateg. Entrep. J. 2017 , 11 , 357–370. [ Google Scholar ] [ CrossRef ]
- Ghenea, M. Ghenea-Marius-Antreprenoriat-Cut.Pdf—PDFCOFFEE.COM. Available online: https://pdfcoffee.com/ghenea-marius-antreprenoriat-cutpdf-pdf-free.html (accessed on 30 January 2023).
- Chen, J.K.C.; Sriphon, T. Authentic Leadership, Trust, and Social Exchange Relationships under the Influence of Leader Behavior. Sustainability 2022 , 14 , 5883. [ Google Scholar ] [ CrossRef ]
- Lisdiono, P.; Said, J.; Yusoff, H.; Hermawan, A.A. Examining Leadership Capabilities, Risk Management Practices, and Organizational Resilience: The Case of State-Owned Enterprises in Indonesia. Sustainability 2022 , 14 , 6268. [ Google Scholar ] [ CrossRef ]
- Hor, S.C.; Chang, A.; Torres de Oliveira, R.; Davidsson, P. From the Theories of Financial Resource Acquisition to a Theory for Acquiring Financial Resources—How Should Digital Ventures Raise Equity Capital beyond Seed Funding. J. Bus. Ventur. Insights 2021 , 16 , e00278. [ Google Scholar ] [ CrossRef ]
- Hui, X.; Li, B.; Li, M. Entrepreneurial Management Equity Allocation and Financing Structure Optimization of Technology-Based Entrepreneurial Firm. Nankai Bus. Rev. Int. 2018 , 9 , 395–412. [ Google Scholar ] [ CrossRef ]
- Phuong, L.N.; Tuan, K.C.; Duc, N.N.; Thi, U.N. The Impact of Absorption Capability, Innovation Capability, and Branding Capability on Firm Performance—An Empirical Study on Vietnamese Retail Firms. Sustainability 2022 , 14 , 6422. [ Google Scholar ] [ CrossRef ]
- Yasir, N.; Xie, R.; Zhang, J. The Impact of Personal Values and Attitude toward Sustainable Entrepreneurship on Entrepreneurial Intention to Enhance Sustainable Development: Empirical Evidence from Pakistan. Sustainability 2022 , 14 , 6792. [ Google Scholar ] [ CrossRef ]
- Guerzoni, M.; Taylor Aldridge, T.; Audretsch, D.B.; Desai, S. A New Industry Creation and Originality: Insight from the Funding Sources of University Patents. Res. Policy 2014 , 43 , 1697–1706. [ Google Scholar ] [ CrossRef ]
- Von Maltitz, A.; Van der Lingen, E. Business Model Framework for Education Technology Entrepreneurs in South Africa. S. Afr. J. Entrep. Small Bus. Manag. 2022 , 14 , a472. [ Google Scholar ] [ CrossRef ]
- De Faoite, D.; Henry, C.; Johnston, K.; Van der Sijde, P. Education and Training for Entrepreneurs: A Consideration of Initiatives in Ireland and The Netherlands. Educ. Train. 2003 , 45 , 430–438. [ Google Scholar ] [ CrossRef ]
- Burov, V.Y.; Khokhlova, G.I.; Kretova, N.V. Development of Small and Medium-Sized Businesses in the Construction Industry through the Use of Business Networks. IOP Conf. Ser. Earth Environ. Sci. 2021 , 751 , 012136. [ Google Scholar ] [ CrossRef ]
- Birtch, T.A.; Au, K.Y.; Chiang, F.F.T.; Hofman, P.S. How Perceived Risk and Return Interacts with Familism to Influence Individuals’ Investment Strategies: The Case of Capital Seeking and Capital Providing Behavior in New Venture Financing. Asia Pac. J. Manag. 2018 , 35 , 471–500. [ Google Scholar ] [ CrossRef ]
- Brettel, M. Business Angels in Germany: A Research Note. Ventur. Cap. 2003 , 5 , 251–268. [ Google Scholar ] [ CrossRef ]
- Dina, V.; Gomezel, A.S. What Do We Know about Business Angels’ Decision Making Research Development? A Document Co-Citation Analysis. Econ. Bus. Rev. 2022 , 24 , 2. [ Google Scholar ]
- Landström, H.; Mason, C. Handbook of Research on Business Angels ; Edward Elgar Publishing: Cheltenham, UK, 2016; ISBN 978-1-78347-172-0. [ Google Scholar ]
- Lefebvre, V.; Certhoux, G. Acting as a Business Angel to Become a Better Entrepreneur: A Learning Innovation. Entrep. Educ. Pedagog. 2022 , 251512742110702. [ Google Scholar ] [ CrossRef ]
- Mason, C.M. The Real Venture Capitalists: A Review of Research on Business Angels; Hunter Center for Entrepreneurship: 2008. Available online: https://www.academia.edu/16857926/The_real_venture_capitalists_a_review_of_research_on_business_angels (accessed on 31 January 2023).
- Pierrakis, Y.; Owen, R. Startup Ventures and Equity Finance: How Do Business Accelerators and Business Angels’ Assess the Human Capital of Socio-Environmental Mission Led Entrepreneurs? Innovation 2022 , 1–25. [ Google Scholar ] [ CrossRef ]
- Politis, D. Business Angels and Value Added: What Do We Know and Where Do We Go? Ventur. Cap. 2008 , 10 , 127–147. [ Google Scholar ] [ CrossRef ]
- Ramadani, V. Business Angels: Who They Really Are. Strateg. Change 2009 , 18 , 249–258. [ Google Scholar ] [ CrossRef ]
- White, B.A.; Dumay, J. Business Angels: A Research Review and New Agenda. Ventur. Cap. 2017 , 19 , 183–216. [ Google Scholar ] [ CrossRef ]
- European Commission Business Angels. Available online: https://single-market-economy.ec.europa.eu/access-finance/policy-areas/business-angels_en (accessed on 24 October 2022).
- Tenca, F.; Croce, A.; Ughetto, E. Business Angels Research In Entrepreneurial Finance: A Literature Review And A Research Agenda. J. Econ. Surv. 2018 , 32 , 1384–1413. [ Google Scholar ] [ CrossRef ]
- Kamna, D.F.; Ilkhanizadeh, S. Can High-Performance Work Practices Influence Employee Career Competencies? There Is a Need for Better Employee Outcomes in the Banking Industry. PLoS ONE 2022 , 17 , e0264764. [ Google Scholar ] [ CrossRef ]
- Mallinguh, E.; Zoltan, Z. Financial Institution Type and Firm-Related Attributes as Determinants of Loan Amounts. J. Risk Financ. Manag. 2022 , 15 , 119. [ Google Scholar ] [ CrossRef ]
- Cehaji, A.; Kosak, M. Macroprudential Measures and Developments in Bank Funding Costs. Int. Rev. Financ. Anal. 2021 , 78 , 101943. [ Google Scholar ] [ CrossRef ]
- Deng, M.; Fang, X.; Tian, Z.; Luo, W. The Impact of Environmental Uncertainty on Corporate Innovation: Evidence from Chinese Listed Companies. Sustainability 2022 , 14 , 4902. [ Google Scholar ] [ CrossRef ]
- Spilker, H.D. Hedge Fund Family Ties. J. Bank Financ. 2022 , 134 , 106326. [ Google Scholar ] [ CrossRef ]
- Kim, Y. What Makes the Divergence between Cross-Border VCs and Domestic VCs Persist? In the Context of the Chinese VC Industry. Seoul J. Bus. 2018 , 24 , 15–43. [ Google Scholar ] [ CrossRef ]
- Gantenbein, P.; Kind, A.; Volonté, C. Individualism and Venture Capital: A Cross-Country Study. Manag. Int. Rev. 2019 , 59 , 741–777. [ Google Scholar ] [ CrossRef ]
- Carayannis, E.; Jones, P.; Liargovas, P.; Apostolopoulos, N. Entrepreneurship and the European Union Policies after 60 Years of Common European Vision: Regional and Spatial Perspectives. J. Small Bus. Entrep. 2020 , 32 , 517–522. [ Google Scholar ] [ CrossRef ]
- Dvouletý, O.; Srhoj, S.; Pantea, S. Public SME Grants and Firm Performance in European Union: A Systematic Review of Empirical Evidence. Small Bus. Econ. 2021 , 57 , 243–263. [ Google Scholar ] [ CrossRef ]
- Staniewski, M.W.; Szopiński, T.; Awruk, K. Setting up a Business and Funding Sources. J. Bus. Res. 2016 , 69 , 2108–2112. [ Google Scholar ] [ CrossRef ]
- Arundale, K. Raising Venture Capital Finance in Europe: A Practical Guide for Business Owners, Entrepreneurs and Investors ; Kogan Page Publishers: London, UK, 2007. [ Google Scholar ]
- Ceptureanu, S.I.; Ceptureanu, E.G. Challenges and Barriers of European Young Entrepreneurs. Manag. Res. Pract. 2015 , 7 , 34. [ Google Scholar ]
- Murray, G.C. A Policy Response to Regional Disparities in the Supply of Risk Capital to New Technology-Based Firms in the European Union: The European Seed Capital Fund Scheme. Reg. Stud. 1998 , 32 , 405–419. [ Google Scholar ] [ CrossRef ]
- Nicolau, C.; Foris, T. Training Future Entrepreneurs Using European Funds. A Descriptive Research on Start-Up Romania Programs. Ovidius Univ. Ann. Econ. Sci. Ser. 2016 , 16 , 372–377. [ Google Scholar ]
- Rusu, V.D.; Roman, A. Assessing the Role of Access to Finance for Young Potential Entrepreneurs: The Case of Romania. KnE Soc. Sci. 2020 , 301–324. [ Google Scholar ] [ CrossRef ]
- Romanian Government FEMEIA ANTREPRENOR Manual de Utilizare Pentru Depunerea Cererii de Finantare. 2022. Available online: https://turism.gov.ro/web/wp-content/uploads/2022/08/Manual-Femeia-Antreprenor.pdf (accessed on 1 February 2023).
- Nguyen, M. The Impact on Corporate Financial Leverage of the Relationship Between Tax Avoidance and Institutional Ownership: A Study of Listed Firms in Vietnam. Montenegrin J. Econ. 2021 , 17 , 65–73. [ Google Scholar ] [ CrossRef ]
- Qi, P.; Shang, Y.; Han, F. The Effects of Environmental Regulation on Investment Efficiency—An Empirical Analysis of Manufacturing Firms in the Beijing–Tianjin–Hebei Region, China. Sustainability 2022 , 14 , 6371. [ Google Scholar ] [ CrossRef ]
- Lin, L. Venture Capital Exits and the Structure of Stock Markets in China. Asian J. Comp. Law 2017 , 12 , 1–40. [ Google Scholar ] [ CrossRef ]
- Zhang, H.; Jiang, W. An Empirical Study on the Relationship between Economy and Finance in Underdeveloped Areas Based on the VAR Model. J. Sens. 2022 , 2022 , 2224239. [ Google Scholar ] [ CrossRef ]
- Cao, X.; Li, P.; Huang, X.; Fan, L. The Dual Mechanism of Social Networks on the Relationship between Internationalization and Firm Performance: Empirical Evidence from China. PLoS ONE 2022 , 17 , e0263674. [ Google Scholar ] [ CrossRef ] [ PubMed ]
- Tudor, G.-A.; Turlea, I.C.; Mitoi, E. Considerations on the Advantages of Using Leasing in Small and Medium-Sized Enterprises. Audit Financ. 2021 , 19 , 587–598. [ Google Scholar ] [ CrossRef ]
- Nechaev, A. Methods of Lease Payments Calculating in Terms of Innovations Financing. Montenegrin J. Econ. 2021 , 17 , 133–149. [ Google Scholar ] [ CrossRef ]
- Gauger, F.; Strych, J.-O.; Pfnür, A. Linking Real Estate Data with Entrepreneurial Ecosystems: Coworking Spaces, Funding and Founding Activity of Start-Ups. Data Brief 2021 , 37 , 107185. [ Google Scholar ] [ CrossRef ]
- BBC Internal and External Sources of Finance-Sources of Finance-Eduqas-GCSE Business Revision-Eduqas-BBC Bitesize. Available online: https://www.bbc.co.uk/bitesize/guides/zj7yy9q/revision/1 (accessed on 24 October 2022).
- Startup Café Creditul Furnizor-Principalul Mod de Finanțare Pentru Firme. În Ce Domenii Se Fac Afaceri Pe Banii Clienţilor. Available online: https://www.startupcafe.ro/finantari/finantari-credit-comercial-firme.htm (accessed on 24 October 2022).
- Meena, P. The Growth of Suppliers’credits to India, 1960–1969: Analysis of Major Issues. Indian Econ. J. 1982 , 29 , 53. [ Google Scholar ]
- Aaronson, D.; Bostic, R.W.; Huck, P.; Townsend, R. Supplier Relationships and Small Business Use of Trade Credit. J. Urban Econ. 2004 , 55 , 46–67. [ Google Scholar ] [ CrossRef ]
- Cuñat, V.; García-Appendini, E. Trade Credit and Its Role in Entrepreneurial Finance ; Oxford University Press: Oxford, UK, 2012. [ Google Scholar ]
- Rupeika-Apoga, R.; Danovi, A. Availability of Alternative Financial Resources for SMEs as a Critical Part of the Entrepreneurial Eco-System: Latvia and Italy. Procedia Econ. Financ. 2015 , 33 , 200–210. [ Google Scholar ] [ CrossRef ]
- Macht, S.A. Putting “Entrepreneurial Finance Education” on the Map. Educ. Train. 2016 , 58 , 984–1002. [ Google Scholar ] [ CrossRef ]
- Emenike, B. Impact of Entrepreneurial Finance on Business Start-Ups in Rivers State, Nigeria. Int. J. Adv. Acad. Res. 2021 , 7 . [ Google Scholar ] [ CrossRef ]
- Biggs, T.; Raturi, M.; Srivastava, P. Ethnic Networks and Access to Credit: Evidence from the Manufacturing Sector in Kenya. J. Econ. Behav. Organ. 2002 , 49 , 473–486. [ Google Scholar ] [ CrossRef ]
- Beck, T.; Pamuk, H.; Ramrattan, R.; Uras, B.R. Payment Instruments, Finance and Development. J. Dev. Econ. 2018 , 133 , 162–186. [ Google Scholar ] [ CrossRef ]
- Huyghebaert, N.; Van de Gucht, L.; Van Hulle, C. The Choice between Bank Debt and Trace Credit in Business Start-Ups. Small Bus. Econ. 2007 , 29 , 435–452. [ Google Scholar ] [ CrossRef ]
- Li, L.; Wu, D. Entrepreneurial Education and Students’ Entrepreneurial Intention: Does Team Cooperation Matter? J. Glob. Entrep. Res. 2019 , 9 , 35. [ Google Scholar ] [ CrossRef ]
- Sousa, M.J.; Carmo, M.; Gonçalves, A.C.; Cruz, R.; Martins, J.M. Creating Knowledge and Entrepreneurial Capacity for HE Students with Digital Education Methodologies: Differences in the Perceptions of Students and Entrepreneurs. J. Bus. Res. 2019 , 94 , 227–240. [ Google Scholar ] [ CrossRef ]
- Sansone, G.; Ughetto, E.; Landoni, P. Entrepreneurial Intention: An Analysis of the Role of Student-Led Entrepreneurial Organizations. J. Int. Entrep. 2021 , 19 , 399–433. [ Google Scholar ] [ CrossRef ]
- Rus-Casas, C.; Eliche-Quesada, D.; Aguilar-Peña, J.D.; Jiménez-Castillo, G.; La Rubia, M.D. The Impact of the Entrepreneurship Promotion Programs and the Social Networks on the Sustainability Entrepreneurial Motivation of Engineering Students. Sustainability 2020 , 12 , 4935. [ Google Scholar ] [ CrossRef ]
- Pauceanu, A.; Alpenidze, O.; Edu, T.; Zaharia, R. What Determinants Influence Students to Start Their Own Business? Empirical Evidence from United Arab Emirates Universities. Sustainability 2018 , 11 , 92. [ Google Scholar ] [ CrossRef ]
- Student Entrepreneurial Society Transilvania University of Brasov (SAS-UTBv) Societatea Antreprenoriala Studențească. Available online: https://www.unitbv.ro/cercetare/transfer-tehnologic-si-antreprenoriat/sas.html (accessed on 24 October 2022).
- Barbu, A.; Isaic-Maniu, A. Data Collection in Romanian Market Research: A Comparison between Prices of PAPI, CATI and CAWI. Manag. Mark. 2011 , 6 , 349–364. [ Google Scholar ]
- Howitt, D.; Cramer, D.; Popescu, A.; Popa, C. Introducere in SPSS Pentru Psihologie ; Polirom: Iași, Romania, 2006; p. 191. [ Google Scholar ]
- Malhotra, N. Marketing Research, an Applied Orientation , 4th ed.; Pearson Education LTD: Harlow, UK, 2004; pp. 558–582. [ Google Scholar ]
- Career Journal. Available online: https://revistacariere.ro/inspiratie/actual/a-fost-semnat-pactul-pentru-educatie-antreprenoriala-pentru-urmatorii-4-ani/ (accessed on 30 October 2022).
- Lecuna, A. Understanding Imagination in Entrepreneurship. Entrep. Res. J. 2021 , 000010151520210103. [ Google Scholar ] [ CrossRef ]
- Astuty, E.; Yustian, O.R.; Ratnapuri, C.I. Building Student Entrepreneurship Activities Through the Synergy of the University Entrepreneurship Ecosystem. Front. Educ. 2022 , 7 , 757012. [ Google Scholar ] [ CrossRef ]
- Dams, C.; Sarria Allende, V.; Cornejo, M.; Pasquini, R.A.; Robiolo, G. Impact of Accelerators, as Education & Training Programs, on Female Entrepreneurs. Entrep. Res. J. 2022 , 12 , 329–362. [ Google Scholar ] [ CrossRef ]
- Jennings, J.E.; Tonoyan, V. Research on Gender Stereotyping and Entrepreneurship: Suggestions for Some Paths Worth Pursuing. Entrep. Res. J. 2022 , 12 , 187–212. [ Google Scholar ] [ CrossRef ]
- Lazar, M. Profesionalism și etică în profesia contabilă și de audit. Rev. Audit. Financ. 2008 , 1 , 13. [ Google Scholar ]
- Strickland, B. FASB Updates Reporting Standard for Supplier Finance Programs—Journal of Accountancy. Available online: https://www.journalofaccountancy.com/news/2022/sep/fasb-updates-reporting-standard-supplier-finance-programs.html (accessed on 24 October 2022).
Share and Cite
Zamfirache, A.; Suciu, T.; Anton, C.E.; Albu, R.-G.; Ivasciuc, I.-S. The Interest Shown by Potential Young Entrepreneurs in Romania Regarding Feasible Funding Sources, in the Context of a Sustainable Entrepreneurial Education. Sustainability 2023 , 15 , 4823. https://doi.org/10.3390/su15064823
Zamfirache A, Suciu T, Anton CE, Albu R-G, Ivasciuc I-S. The Interest Shown by Potential Young Entrepreneurs in Romania Regarding Feasible Funding Sources, in the Context of a Sustainable Entrepreneurial Education. Sustainability . 2023; 15(6):4823. https://doi.org/10.3390/su15064823
Zamfirache, Alexandra, Titus Suciu, Carmen Elena Anton, Ruxandra-Gabriela Albu, and Ioana-Simona Ivasciuc. 2023. "The Interest Shown by Potential Young Entrepreneurs in Romania Regarding Feasible Funding Sources, in the Context of a Sustainable Entrepreneurial Education" Sustainability 15, no. 6: 4823. https://doi.org/10.3390/su15064823
Article Metrics
Article access statistics, further information, mdpi initiatives, follow mdpi.
Subscribe to receive issue release notifications and newsletters from MDPI journals
- Reference Manager
- Simple TEXT file
People also looked at
Review article, determining cancer stage at diagnosis in population-based cancer registries: a rapid scoping review.
- 1 School of Population Health, Curtin University, Perth, WA, Australia
- 2 Public Health, North Metropolitan Health Service, Perth, WA, Australia
- 3 School of Population and Global Health, The University of Western Australia, Perth, WA, Australia
- 4 Department of Health, Western Australia Cancer Registry, Clinical Excellence Division, Perth, WA, Australia
- 5 Cancer Network WA, North Metropolitan Health Service, Perth, WA, Australia
Introduction: Population-based cancer registries are the main source of data for population-level analysis of cancer stage at diagnosis. This data enables analysis of cancer burden by stage, evaluation of screening programs and provides insight into differences in cancer outcomes. The lack of standardised collection of cancer staging in Australia is well recognised and is not routinely collected within the Western Australia Cancer Registry. This review aimed to explore how cancer stage at diagnosis is determined in population-based cancer registries.
Methods: This review was guided by the Joanna-Briggs Institute methodology. A systematic search of peer-reviewed research studies and grey literature from 2000 to 2021 was conducted in December 2021. Literature was included if peer-reviewed articles or grey literature sources used population-based cancer stage at diagnosis, and were published in English between 2000 and 2021. Literature was excluded if they were reviews or only the abstract was available. Database results were screened by title and abstract using Research Screener. Full-texts were screened using Rayyan. Included literature were analysed using thematic analysis and managed through NVivo.
Results: The findings of the 23 included articles published between 2002 and 2021 consisted of two themes. (1) “Data sources and collection processes” outlines the data sources used, as well as the processes and timing of data collection utilised by population-based cancer registries. (2) “Staging classification systems” reveals the staging classification systems employed or developed for population-based cancer staging, including the American Joint Committee on Cancer's Tumour Node Metastasis and related systems; simplified systems classified into localised, regional, and distant categories; and miscellaneous systems.
Conclusions: Differences in approaches used to determine population-based cancer stage at diagnosis challenge attempts to make interjurisdictional and international comparisons. Barriers to collecting population-based stage at diagnosis include resource availability, infrastructure differences, methodological complexity, interest variations, and differences in population-based roles and emphases. Even within countries, disparate funding sources and funder interests can challenge the uniformity of population-based cancer registry staging practices. International guidelines to guide cancer registries in collecting population-based cancer stage is needed. A tiered framework of standardising collection is recommended. The results will inform integrating population-based cancer staging into the Western Australian Cancer Registry.
1. Introduction
Cancer staging at an individual level is integral to determining clinical management and estimating prognosis. Population-level cancer stage at diagnosis, defined as the extent to which a cancer has spread at initial diagnosis, is used to analyse the burden of cancer by stage and associated trends over time, which assists with planning for predicted demand on cancer health services and enables the evaluation of outcomes from cancer screening programs ( 1 – 3 ). Population-based cancer stage analysis also assists in understanding differences in cancer outcomes and survival ( 3 , 4 ). While international comparisons showed differences in overall cancer survival between countries, the differences are mainly explained by stage at diagnosis and stage-specific survival variations ( 4 ). Therefore, completeness, consistency and comparability of staging classification are required to enable international comparisons of cancer outcomes ( 5 – 7 ).
The American Joint Commission on Cancer's (AJCC) Tumour Node Metastases (TNM) is a comprehensive cancer staging classification system most widely used globally for staging solid tumours. However, it is not always available to population-based cancer registries (PBCRs) nor collected routinely by them ( 5 , 8 ). Limitations in collecting stage data by PBCRs generally relate to the availability and completeness of recorded cancer stage and other stage information in routinely received or accessible data sources ( 7 ). Furthermore, the collection of stage at diagnosis requires significant resources, which poses a substantial challenge for PBCRs that are resource constrained.
In the face of these barriers, PBCRs use different approaches to collect population-based cancer stage at diagnosis. This includes the simplified staging systems developed specifically for use by PBCRs for population-level analyses, including the Surveillance, Epidemiology and End Results' (SEER) Summary Stage and the European Network of Cancer Registries' (ENCR) Condensed TNM staging system ( 7 , 9 , 10 ). As a result, different PBCRs may use different staging classification systems to collect stage at diagnosis at a population-level. If there are no validated means of converting between different staging classification systems, then this can result in difficulties in making comparisons of stage at diagnosis data between PBCRs.
The lack of standardised collection of staging data in Australia has been identified as a major gap in the National Cancer Data Strategy by Cancer Australia ( 3 ). This led to the national Stage, Treatment and Recurrence project (STaR project, also known as National Collection of Registry-Derived Stage project) which enabled the consistent collection of Registry-Derived stage for the five highest incidence tumour groups (prostate, breast, lung, colorectal, and melanoma) across all Australian PBCRs for cancer cases diagnosed in 2011 ( 1 , 11 ). Registry-Derived (RD) stage was defined as the best estimate of cancer stage at diagnosis used for population-based analysis, as determined by PBCRs from available data sources ( 12 ). The Western Australian Cancer Registry (WACR) is one such PBCR which was involved in the STaR project.
The WACR was developed in 1982 and has since provided population-based cancer data for use in the planning of health care services and the support of cancer-related research at local, national, and international levels ( 13 ). Except for the duration of the STaR project, from 2018 onwards the WACR has collected pathological cancer stage at diagnosis incidentally when identified during routine cancer registration processes, however it is not a routinely collected data item. In alignment with Cancer Australia's position on the requirement for improved stage data collection, the Western Australia (WA) Cancer 2020–2025 Implementation Plan includes the key strategic action to “Develop a timely data collection for cancer stage at diagnosis” ( 14 ). Subsequently, the WA Cancer Staging Project has commenced which aims to develop, deliver and evaluate a state-wide population-based staging approach within the WACR.
A key consideration for the project has been how to ensure comparability and consistency with PBCRs from other Australian jurisdictions and from other countries, with the aim of enabling interjurisdictional and international benchmarking. At the time of this rapid scoping review, there was limited literature existing that examined how PBCRs collect population-based cancer stage at diagnosis. The aim of this rapid scoping review was to explore how population-based cancer stage at diagnosis is determined at a population-level. By providing an overview of the literature on population-based cancer staging approaches used by PBCRs, the findings of this review will provide critical information to inform the development and implementation of population-based cancer staging in the WACR.
A rapid scoping review was identified as the most appropriate approach as scoping reviews provide a broader overview of the literature compared to systematic reviews that address a targeted question ( 15 ) and uses a more systematic approach compared to literature reviews ( 16 ). The study was designed to be conducted in three months to present preliminary findings to key stakeholders of the WA Cancer Staging Project. Therefore, a rapid scoping review was conducted as defined by Tricco et al. ( 17 ) whereby aspects of the review process are simplified or omitted to enable the production of information in a timely manner. In this rapid review, the screening process was simplified where only a portion of articles for title and abstract screening were reviewed by a second reviewer and full-text screening was conducted by one reviewer. Yet, any queries regarding inclusion of the full-text were discussed with the second reviewer.
The review was undertaken following the Joanna-Briggs Institute methodology for scoping reviews ( 18 – 20 ). The Cochrane Database for Systematic Reviews ( 21 ) and the Joanna Briggs Institute (JBI) Evidence-based Practice Database ( 22 ) were searched and no existing reviews in this area were identified. The research team involved expertise in the WACR, PBCRs, population health, health psychology, cancer nursing, data science and qualitative, quantitative and mixed methods research. The composition of the team created a diverse range of perspectives on reviewing the data.
Registration of the protocol with Figshare ( 23 ) and Open Science Framework ( 24 ) were explored. Whilst protocol registration is not currently a requirement for scoping reviews as per the JBI framework ( 18 ) the timeframe to register was not a possibility due to the project timeframe and the need to conduct the review within three months to present the preliminary findings to the WA Cancer Staging Project key stakeholders.
As per the JBI framework, an initial protocol was compiled prior to undertaking the review ( 18 ). The protocol outlined the background and gaps in the literature, aims, methods and reporting of the review and was peer-reviewed by an experienced health researcher with expertise in scoping review methodology and qualitative methods. The initial protocol was positively received. The protocol was amended when the research team decided to conduct a rapid scoping review instead. The priori protocol was available to all authors and assisted the systematic search and screening process.
2.1. Eligibility criteria
Literature was included if the following inclusion criteria were met:
a) Peer-reviewed articles or grey literature sources using population-based cancer stage at diagnosis
b) Published in English
c) Published from year 2000 to 2021 inclusive
Literature was excluded based on the following exclusion criteria:
a) Only the abstract was available
The timeframe was chosen due to the emergence of registry staging during this period based on discussions between all authors. One of the authors was familiar with the literature in this area and guided the timeframe and grey literature sources to search.
2.2. Search strategy
The three-step search strategy as recommended by JBI ( 18 ) was adapted for the literature search. The first step involved the initial limited search of MEDLINE in consultation with an experienced research librarian using the research question and preliminary key terms. In consultation with the authors and the research librarian, the search terms were compiled, and specific research databases were chosen (MEDLINE, Embase, CINAHL, and Web of Science). The second step involved the full search of the final search terms in the selected research databases in December 2021. Table 1 lists the search terms used developed in consultation with LP, SS, RT, and RM and the research librarian. Google Scholar search terms used were simplified to fit within the character limit for searches. All searches were limited to English and between the years 2000–2021. The third step occurred after screening of the articles was completed and involved searching the reference lists of papers that met the inclusion criteria.
Table 1 . Search terms used for literature search.
Searches for grey literature defined as including reports, theses, conference proceedings, technical specifications and standards, translations, bibliographies, technical and commercial documentation, and official documents ( 25 ), were also undertaken. The grey literature search involved conducting searches in Google Scholar and Google advanced search, where the first 200 results, as recommended by Haddaway et al. ( 26 ), were reviewed by the first reviewer. Official websites of population-based cancer registries and organisations as identified from discussion with other authors were also searched for grey literature, including the websites for SEER ( 27 ), National Cancer Registration and Analysis Service (NCRAS) ( 28 ), Canadian Partnership Against Cancer ( 29 ), the International Cancer Benchmarking Partnership (ICBP) ( 30 ), ENCR ( 31 ), and International Agency for Research on Cancer (IARC) ( 32 ).
2.3. Selection of sources of evidence
The search results were imported into Research Screener which is a semi-automated machine learning tool developed at Curtin University ( 33 ). Research Screener facilitates title and abstract screening and automatically removes duplicate articles, enabling more rapid screening of literature for reviews ( 33 ) as was required for the short timeframe for the review. Seed articles are defined as articles that have been identified by the researchers as highly relevant based on their eligibility criteria and are used by the Research Screener algorithm to initially rank articles according to relevance ( 33 ). Six seed articles ( 5 – 7 , 11 , 34 , 35 ) were chosen for use with Research Screener in consultation with the research team. The algorithm repeatedly re-ranks the remaining unread articles based on what articles the reviewer has included and excluded.
The first reviewer screened all articles identified from the search of research databases by title and abstract using the eligibility criteria. A second reviewer independently screened the first 50 articles produced in Research Screener by title and abstract for eligibility and subsequently screened any additional articles identified by the first reviewer as meeting the eligibility criteria. Uncertainty or disagreements were resolved through discussion between the two reviewers. The eligible full-texts were imported into Rayyan's Systematic Review Screening Software ( 36 ) and the first reviewer performed the full-text review of eligible articles. Rayyan is a tool that assists with facilitating the systematic search and selection of studies for reviews, with features that demonstrate when multiple reviewers agree or disagree on the inclusion and exclusion of articles, and allowing the reason for exclusions to be listed per excluded article ( 36 ). Two texts resulted in uncertainty to include and were resolved by discussion between four of the authors (LP, SS, RT, RM). The reference lists of articles that met the eligibility criteria during full-text review were checked to identify additional relevant articles.
2.4. Data extraction
Data was extracted utilising a charting table developed by the two reviewers and adapted from the JBI template source of evidence details, characteristics and results extraction instrument ( 18 ). The table included the author(s), year of publication, country of origin, aims, type of article, and key findings that relate to the scoping review question.
2.5. Synthesis of results
Included full-texts were imported to qualitative data analysis software NVivo ( 37 ) which was used to manage the data. Thematic analysis was used to compare and contrast the findings across studies to enable identification and interpretation of patterns within the data ( 38 ). The data analysis was guided by Braun and Clarke's ( 38 ) six stages of thematic analysis (familiarisation with the data, generating initial codes, searching for themes, reviewing themes, defining and naming themes, producing the report) using an inductive and iterative process. The first reviewer read and reread all the included texts to familiarise with the data noting down preliminary ideas, generated a list of initial codes from the entire dataset in a data-driven approach, and grouped common codes to form themes. The themes at the level of the included codes and the overall dataset using a thematic map were reviewed by the first and second reviewers. The subsequent themes and subthemes were reviewed by all authors. Themes were further analysed and defined by considering how they conveyed a narrative related to the research question. Lastly, the results of the data analysis were written up into this article.
The search identified a total of 1,843 records ( n = 1,825 from database searches, n = 18 from other sources). After removal of duplicates and records missing abstracts, the remaining 1,323 records from database searches were screened by title and abstract. The resultant records as well as records from other sources were assessed for eligibility ( n = 43) ( Figure 1 ). The results of the search and the study inclusion process are presented in the adapted PRISMA-ScR flow diagram ( Figure 1 ) ( 19 ). The PRISMA-ScR flow diagram was adapted as the records from additional sources did not consistently have an abstract and as a result could not be screened by title and abstract using Research Screener. Instead, these records moved straight to the full-text review stage.
Figure 1 . PRISMA-ScR flow diagram for the rapid scoping review.
The final dataset consisted of 23 texts, including 15 peer-reviewed journal articles and eight electronic documents from official websites of cancer registries and cancer organisations. All texts were published between 2002 and 2021 ( Table 2 ).
Table 2 . Characteristics of included sources of evidence.
Two key themes were identified. (1) “Data sources collection processes,” reviews the data sources used and processes, including the timing of data collection. (2) “Staging classification systems,” outlines the various classification systems including tumour, node and metastasis; those that are categorised into local, regional, and distant; and miscellaneous staging classification systems. Table 3 summarises the key themes and subthemes.
Table 3 . Key themes and subthemes.
3.1. Theme 1: Data sources and collection processes
This theme covers the data sources employed and the data collection processes utilised by PBCRs, including whether data was processed by manual review or using computer algorithms and the timeframe to collect cancer information to determine population-level cancer stage at diagnosis.
3.1.1. Data sources used
PBCRs from different countries used different data sources for population-based cancer staging. The frequently used data sources were pathology reports ( 11 , 34 , 45 – 47 , 52 , 53 ), hospital notifications and discharge summaries ( 11 , 34 , 45 – 47 , 53 ), death certificates ( 34 , 43 , 45 , 46 , 52 , 53 ) and autopsy reports ( 43 , 45 – 47 , 53 ), and medical records ( 11 , 47 , 52 ). Under national and jurisdictional legislation, PBCRs in Australia are required to be notified of cancer diagnoses ( 11 , 34 , 35 ) and routinely received notification sources were pathology reports, hospital morbidity data, and death certificates ( 11 , 34 , 35 ). Other sources available varied depending on the jurisdiction. For example, the South Australian Cancer Registry received radiotherapy notifications ( 34 ). Similarly, the New Zealand Cancer Registry received pathology reports with new diagnoses of cancer under the Cancer Registry Act 1993, as well as hospital discharge reports, death certificates, and autopsy reports ( 53 ). The most routinely used data sources for the European PBCRs were pathology laboratories (used by 100% of PBCRs), hospital oncology records (93%), other hospital records (97%), radiotherapy departments (83%), haematology laboratories (80%), and death certificates (78%) ( 52 ).
Chemotherapy systems, radiotherapy systems, imaging systems, multidisciplinary team (MDT) meeting data, molecular testing results, and clinical audit data were less commonly available data sources for PBCRs. The NCRAS in England was the single PBCR that reported access to those aforementioned sources ( 6 , 47 , 50 ). The NCRAS also had additional access to multiple different linked datasets for its cancer registration processes ( 6 , 47 , 50 ), including national data on hospital activity, patient waiting times, diagnostic imaging, cancer screening, mortality, and national cancer audits ( 47 , 50 ). The National Cancer Patient Experience Survey and the Patient Reported Outcome Measures datasets were additional accessible data sources that were linked to cancer registration data ( 50 ).
Though pathology reports were a commonly utilised data source for PBCRs, several limitations were identified regarding their use as a source of stage data ( 43 , 52 ). First, pathological stage was not always recorded in pathology reports ( 34 , 45 ). Pathologists relied on the information provided by the surgeon when interpreting specimens which in turn affects the stage data provided in the pathology report ( 45 ). The development and adoption of synoptic structured reporting was recommended to ensure more consistent pathological information including cancer stage is provided to PBCRs ( 35 , 45 ). Second, pathology reports provided substantial pathological information however often lacked the clinical information necessary for aspects of staging not related to the primary tumour ( 35 , 43 ). Other data sources containing information on nodal involvement and distant metastasis were not as commonly available, including imaging reports and site-specific diagnostic and prognostic tests ( 35 , 43 ).
3.1.2. Processes and timing of data collection
This subtheme covers the PBCR data collection processes involved, as well as the timing of data collection used to determine population-based cancer stage at diagnosis.
Automated tools were used by some of the PBCRs to assist with cancer registration processes, such as the linkage of datasets, de-duplication of data, and consolidation of multiple notifications for a cancer case ( 11 , 47 ). However, population-based cancer staging was often described as a manual process for many PBCRs, including those in England, New Zealand, and Australia ( 11 , 34 , 45 , 47 , 50 ). Manual processes included the review of data sources for specific data items, often the direct assignment of cancer stage if not explicitly recorded in data sources, and entry of data into cancer databases by cancer registry coders and officers ( 11 , 34 , 45 , 47 , 49 ). For New Zealand, manual data extraction and entry was performed by cancer registry coders who specialised in specific tumour types ( 45 ). Except for the use of some tools for linkage and de-duplication of data by the NCRAS, the rest of the cancer registration process involved the manual extraction of cancer information by cancer registration officers in England ( 47 ). Registry-Derived stage was manually assigned by Australian PBCRs during the national STaR project and required significant training and manual effort by cancer registry coders ( 11 , 34 ). Degree of Spread (DoS), which reflected the extent of disease at diagnosis collected by the New South Wales (NSW) Cancer Registry on a population-based level, was provided through electronically coded hospital notifications however cancer registry coders also manually reviewed data sources (including pathology reports) to determine DoS ( 11 ). Training and detailed knowledge regarding cancer biology and staging were identified as key requirements for the efficient manual extraction of stage data and direct assigning of cancer stage ( 11 , 34 , 47 , 49 ).
Though less commonly used, the benefits of computer algorithms for population-based cancer staging were recognised, including reducing the potential human error and variation associated with manual assignment of stage ( 40 , 44 ). This was identified as the reason that Aitken et al. ( 40 ) programmed and utilised tumour-specific staging computer algorithms in their study on the feasibility of implementing the Toronto Childhood Cancer Stage Guidelines. Computer algorithm derived stage was checked against manually determined stage using the same staging criteria and reviewed when there was no agreement, with subsequent refinement of the algorithm and repeat testing until complete agreement was reached for multiple actual and hypothetical cases ( 40 ). Overall, there was good agreement between computer algorithm generated stage and manually assigned stage for most of the cancer types ( 40 ). However, the computer algorithms were only used in the step of calculating stage from data items manually extracted from medical records ( 40 ).
The Collaborative Stage (CS) System (also known as the Collaborative Stage Data Collection System) also utilised computer algorithms and was used by PBCRs in United States of America (USA) and Canada to collect stage data between 2004 and 2015 ( 42 , 44 , 49 ). The CS system was developed by the Collaborative Staging Task Force to enable collection of a specific set of data items that could be converted into multiple staging classification systems ( 44 , 49 ). Manual extraction of tumour-specific data items and entry into the CS System fields was also required to enable computer algorithms to automatically generate the tumour stage. The algorithm subsequently generated the AJCC TNM 6th ed. summary stage group and SEER Summary Stages 1977 and 2000 for the cancer case ( 44 ). The CS has since been discontinued and was no longer routinely used ( 49 ). It was recognised that when the CS system was ceased in 2016, cancer registries had to transition from using the computer algorithm-based CS system to the manual assignment of cancer stage ( 49 ).
The timing of data collection was largely based on definitions of stage at diagnosis. Stage at diagnosis was defined as the stage derived within 120 days of the date of diagnosis and, as a result, 120 days was often used as the timespan for data collection to determine stage at diagnosis ( 11 , 43 ). The NSW Cancer Registry used notification sources within 120 days of the date of diagnosis to determine tumour stage regardless of the staging classification system used ( 11 ). During the Australian STaR project, routine notification sources received within 120 days of the date of diagnosis were used to determine registry-derived stage ( 11 , 34 ). The International Cancer Benchmarking Partnership publication by Cabasag et al. ( 43 ) included the recommendation that the timing of data collection for stage at diagnosis was prior to the date that primary treatment is initiated or within four months after the date of diagnosis, depending on whichever is earliest, to ensure comparable definitions of stage at diagnosis. However, it was unclear if four months was equivalent to 120 days exactly.
The exception to this related to the use of treatment procedure records available to the NCRAS, where records with treatment procedures within 30 days before or after the date of diagnosis were used to determine pathological stage ( 6 ). If there were multiple records within this data collection period, then the record with the date closest to the date of diagnosis was given priority over other records ( 6 ).
3.2. Theme 2: Staging classification systems
Multiple classification systems were developed for population-based cancer staging by different groups and for various purposes and cancer types. The staging classification systems were broadly grouped into three groups: tumour, node and metastasis based; categorised into localised, regional, and distant groups; and miscellaneous systems.
3.2.1. Based on tumour, node, and metastasis
Staging classification systems with tumour, node, and metastasis stage variables were commonly used, including the AJCC TNM, condensed TNM, and essential TMN ( 7 , 43 , 48 , 52 ). Included in this group were also two registry approaches to population-based cancer staging using AJCC TNM.
The AJCC TNM system is the most widely used cancer staging system in clinical practice for solid tumours ( 7 , 11 , 43 , 52 ). It is maintained by the AJCC in collaboration with the Union for International Cancer Control (UICC), though the latter also has its own TNM system (known as UICC TNM) which is very similar to the AJCC version ( 7 , 49 ).
AJCC TNM was based on either clinical information (cTNM) gathered prior to starting treatment (including physical examination, imaging, or endoscopy) or on pathological information (pTNM) (using microscopic examination of specimens) ( 5 – 7 ). However, whether TNM was either clinical or pathological was not always recorded ( 5 ). Potentially this represented integrated TNM stage based on both pathological and clinical data, for example from multidisciplinary team meetings ( 5 ). However, it may have also been used when it was unknown whether TNM was based on clinical or pathological evidence ( 5 ). The AJCC TNM staging guidelines are regularly revised based on current clinical evidence, resulting in differences between editions, however the edition used was not always recorded in data sources with the TNM stage ( 5 , 6 , 43 ).
Alphanumeric codes were assigned using tumour-specific rules to three stage variables based on size or depth of invasion of the primary tumour (for the T value), involvement of regional lymph nodes ( N value), and presence of distant metastasis ( M value) ( 5 , 6 , 11 , 35 , 43 , 44 , 49 ). Combining these values gave the overall TNM summary stage group ranging from I to IV ( 5 , 43 , 44 ). Some tumour types also required additional non-anatomical information to determine stage, such as the use of serum tumour markers for testicular cancer ( 11 , 35 ). All three stage variables were required to determine TNM summary stage group, so when the TNM stage variables or other stage information was missing from data sources the TNM summary stage group could not be assigned ( 48 , 49 ). To deal with this, a “non-restrictive approach” was used that assumed that missing and unknown T, N and M values were equivalent to a zero value, which allowed TNM summary stage group to be derived ( 5 , 6 , 35 ). Another approach utilised stage data from other staging classification systems to replace missing TNM information, resulting in a “reconstructed stage” ( 48 ).
The TNM system was the most used staging classification system by PBCRs ( 7 , 43 , 48 , 52 ). It was used by 39% of European PBCRs in the 2013 study by Siesling et al. ( 52 ). Minicozzi et al. ( 48 ) reported that 25 of the 34 European PBCRs included in their 2017 study provided TNM stage data. TNM was additionally used by PBCRs in England, Ireland, Denmark, Norway, Sweden, Australia, and in USA and Canada when the CS system was utilised ( 5 , 11 , 34 , 42 – 44 , 47 , 49 ). UICC TNM was the most preferred system for European PBCRs.
Condensed TNM was a simplified staging system developed by the European Network of Cancer Registries to allow PBCRs to determine TNM stage when any or all three TNM data elements are not recorded in data sources ( 7 , 10 ). It was based on both clinical and pathological information, with preference given to pathology data for T and N ( 10 ). T was categorised based on tumour-specific definitions as either localised or advanced, and N and M as either absent or present ( 7 , 10 ). The overall stage categories were tumour localised, tumour with local spread, tumour with regional spread, advanced cancer (including both metastatic and non-resectable tumours), or unknown extent ( 10 ). Compared to the TNM system, Condensed TNM was less commonly used ( 7 , 48 ). For example, Minicozzi et al. ( 48 ) revealed that 56% of European PBCRs included in the study used Condensed TNM.
Essential TNM was a different simplified TNM staging system developed by the UICC and the IARC to allow stage to be derived when TNM stage data elements are not recorded ( 7 ). Site-specific staging guidelines using clinical and pathological information were presented in staging flowcharts and applied to breast, cervical, colorectal, and prostate cancers ( 7 ). T was classified as either localised or advanced, and both N and M were classified as either absent or present (with N further subdivided into regional limited and regional extensive if present) ( 7 ). The overall stage categories varied depending on the tumour type, ranging from localised limited, localised advanced, regional limited, regional extensive, and distant metastasis ( 7 ). These stage groups correlated with the TNM summary stage groups ( 7 ). It was unknown whether Essential TNM was used by any PBCRs however, the original article by Pineros et al. ( 7 ) reported pilot studies were occurring at the time of publication.
The two terms “Registry-Derived stage” and “registry defined stage” described two country-specific approaches to how the AJCC TNM system has been adapted for use by PBCRs for population-based cancer staging.
RD stage was the best estimate of TNM summary stage at time of diagnosis based on notification sources routinely available to PBCRs in Australia ( 11 , 34 ). It was derived from the STaR project whereby stage was collected for the five highest incidence cancers (prostate, breast, lung, bowel, and melanoma) by jurisdictional PBCRs in Australia using simplified business rules based on the AJCC TNM 7th edition ( 11 , 34 ). Cancer registry coders transcribed recorded T, N, and M values or manually assigned these values if missing, which required extensive training ( 11 , 34 ). Overall, Lawrance et al. ( 11 ) found that RD stage had higher agreement with AJCC Stage Group compared to NSW's Degree of Spread. However, since the conclusion of the STaR project, it was no longer routinely collected.
Registry defined stage described the single anatomical stage at diagnosis assigned by NCRAS cancer registration officers based on the TNM system using all relevant information available, including multidisciplinary team meetings, pathology reports, imaging results, and autopsy reports ( 47 ). Due to the multiple data sources available to the NCRAS, inconsistencies may occur between different datasets for the same cancer case ( 5 , 6 ). In response, a hierarchal approach that prioritised specific data sources for individual components of tumour stage was proposed ( 5 , 6 ).
3.2.2. Categorised into localised, regional, and distant
The second major category were population-based cancer staging systems that classified into localised, regional, and distant groups. This included Extent of Disease systems and the SEER Summary Stage system.
Extent of Disease (EoD) was a simplified staging system that classified into stage groups local, regional, and metastatic ( 5 , 48 ). EoD was generally a locally developed system developed for population-based cancer staging by a specific PBCR ( 5 ). For example, the EoD staging system utilised by the NSW Cancer Registry was DoS, which was routinely collected for non-hematological malignancies ( 11 ). EoD was reportedly used by PBCRs in Norway, New Zealand, Finland, Austria, Portugal, Czech Republic, and Estonia ( 5 , 43 , 48 , 53 ), with a number of other PBCRs in Europe that used it in combination with other staging classification systems ( 48 ).
The SEER Summary Stage 2018 was the most recent version of a SEER developed summary stage system used in USA ( 9 ). It applied to all cancer types and uses both clinical and pathological data ( 7 , 9 ). There were six major stage groups ( in situ , localised only, regional, distant site/node involved, benign/borderline, unknown if extension or metastasis) ( 9 , 43 ). SEER Summary Stage was infrequently updated and often used for long-term analyses of cancer stage ( 7 , 44 , 45 , 51 ). As it was a simplified staging system specifically developed for use by PBCRs, it was less complex to learn and use ( 7 , 44 ). However, it was not directly comparable with TNM stage groups and outside of cancer registries it was not well known ( 7 , 9 ). In addition to the PBCRs in USA, Canada and New Zealand PBCRs also reportedly used SEER Summary Stage ( 9 , 43 , 45 ).
3.2.3. Miscellaneous staging classification systems
Miscellaneous staging classification systems did not fit into the two other groups, including the SEER Extent of Disease Coding system, the Toronto Childhood Cancer Stage Guidelines, and site-specific classification systems including FIGO and Duke's classifications. Generally, there was less information available on these staging classification systems and whether PBCRs currently use them.
SEER Extent of Disease Coding was a coding system used for cancer stage developed by SEER ( 39 , 44 , 51 ). It applied to all tumour types and combines clinical and pathological information ( 51 ). Coding rules were tumour-specific based on both SEER Summary Stage 2000 and AJCC TNM 8th ed ( 51 ). Stage was recorded as a ten-digit code which combined five different fields: the size of the primary tumour, extension of the tumour, lymph node involvement, the number of pathologically positive regional lymph nodes, and the number of regional lymph nodes that were pathologically assessed ( 44 ).
The Toronto Childhood Cancer Stage Guidelines (henceforth known as the Toronto Guidelines) were developed by an international panel of experts in collaboration with the UICC with the aim of creating the first globally consistent approach to the population-based cancer staging of childhood cancers ( 40 , 41 , 46 ). The Toronto Guidelines applied to the 16 most common paediatric cancer types, with both the staging rules and the overall stage groups varying based on tumour type ( 40 , 41 ). Recognising the barriers regarding registry resources and data access, it used a two-tiered approach ( 40 , 41 , 46 ). Tier 1 criteria were less detailed and are intended for use by cancer registries with limited resources, such as those in low- and middle-income countries. Tier 2 criteria were more detailed and were developed for use by well-resourced cancer registries with access to data, like those in high-income countries. The Tier 2 categories could be collapsed down into the Tier 1 categories enabling comparisons ( 40 , 41 , 46 ). Aitken et al. ( 40 ) demonstrated the feasibility of implementing the Toronto Guidelines a within a PBCR in Australia, as an example of high-income country.
Though more often used in clinical practice, stage data from site-specific classification systems were collected by a few PBCRs ( 5 , 7 ). FIGO is the acronym for the Fédération Internationale de Gynécologie et d'Obstétrique which developed the staging classification system for gynaecological cancers ( 7 ). Stage groups ranged from I to IV that are comparable to TNM summary stage groups, however was not easily converted into other staging systems ( 5 ). PBCRs in Norway reportedly collected FIGO stage data for ovarian cancer ( 5 ). Duke's classification was a staging system utilised for colorectal cancer though it was no longer commonly used ( 7 ). Stage groups ranged from Duke's A to D that could be converted into comparable TNM summary stage groups however could not be easily converted into other staging systems ( 5 , 6 ). PBCRs in Norway and England reportedly collected Duke's stage for colorectal cancer ( 5 , 6 ).
4. Discussion
This rapid scoping review highlighted the data sources used by PBCRs to collect population-based cancer stage and the various staging classifications used. Differences in how PBCRs collect stage data affects the comparability of stage data when making national and international comparisons between jurisdictions and countries ( 5 – 7 ).
Though the timing of data collection for cancer stage is largely based on the definition of cancer stage at diagnosis within 120 days, variability in the timespan of data collection for primary cancers has been noted across the literature ( 55 ). Research studies from France, the USA, and European countries examining the risk of secondary primary cancer utilised a period of two months to define the synchronicity period (the period following the date of the first cancer diagnosis which is excluded when evaluating the risk of a second primary cancer) ( 55 ). However, variability in the time interval utilised from one month to six months was recognised across different studies on secondary primary cancers ( 55 ).
The data sources used for population-based cancer staging depend on what data is routinely available for a PBCR's registration processes, which tends to be cancer notifications containing new cancer diagnoses. As cancer diagnoses generally require a pathological confirmation of diagnosis, pathology reports were one of the commonest data sources received by PBCRs. However, the limitations associated with pathology reports are well described, particularly regarding the completeness of stage recorded and limited clinical information ( 35 , 43 , 45 ). In relation to the first limitation, there is evidence supporting the role of structured pathology reporting in improving the use of standardised terminology and the completeness of cancer pathology reports, including the completeness of stage data ( 56 – 58 ). Recognising this, the International Collaboration on Cancer Reporting was formed in 2011 as an international collaboration of pathology organisations that develops internationally standardised and evidence-based pathology datasets for cancer reporting ( 56 , 59 ). Improvements in the completeness of cancer stage reported on pathology reports would assist with the completeness of cancer stage collected by PBCRs from pathology reports and further research is required to explore this further. Regarding the second limitation, the data sources that may assist with determining these stage variables, such as imaging reports and MDT information, are not commonly available to PBCRs ( 35 ). Some PBCRs have utilised other approaches to deal with data gaps, such as the use of hospital notification data containing clinical codes for metastatic cancer ( 34 ). The major exception is the NCRAS which exemplifies a PBCR that does have access to the various data sources required to collect comprehensive stage data ( 47 , 50 ). However, when multiple sources are used as seen with the NCRAS processes there is the risk of conflicting information, which is why a hierarchal approach to prioritisation of data sources for specific stage data items is recommended ( 6 ).
Like other cancer registration processes, population-based cancer staging remains a largely manual process for PBCRs ( 7 , 45 , 50 ). However, manual extraction of stage data requires significant effort and time to undertake, particularly when cancer stage recorded in data sources is incomplete or absent and needs to be manually assigned ( 7 , 11 , 49 ). Subsequently it was recognised that for PBCRs that have limited resources, the collection of stage data may not be justifiable ( 11 ). Computer algorithms, such as those used with the Toronto Childhood Cancer Stage Guidelines and the Collaborative Stage System, are examples of tools used to minimise the variation in manual interpretation and application of staging rules. Another such example is the CanStaging + tool which was the first internationally validated, open-source cancer staging tool developed based on ICBP data ( 60 ). Though these algorithms still rely on the manual extraction and input of specific data items.
Acknowledging the challenges associated with large volumes of manual work, artificial intelligence tools have been utilised within PBCRs to streamline the processing of medical documents such as automated case-finding (identification of new and existing cancer cases) and auto-coding (abstraction and coding of information into data fields) ( 61 ). Additionally, increasing emphasis has been placed on the development of automated tools to assist PBCRs with cancer staging and research has been conducted into artificial intelligence methods to extract cancer stage from pathology reports and medical records ( 54 , 62 , 63 ). It is unclear whether these tools are currently being used by PBCRs though it is expected that this field of research will continue to grow and may be utilised by PBCRs in the future.
Nationally driven initiatives and programs allow for large scale reforms and improvements that are well-resourced, enabling consistency between PBCRs in different jurisdictions and resulting in a tendency towards country-specific approaches to population-based cancer staging. Following the ICBP's investigations on why cancer survival was lower in the United Kingdom compared to other high-income countries, significant reforms were undertaken in England to standardise cancer registration practices (including the transition from eight regional cancer registries into a singular centralised registry and utilisation of more data sources for registry processes) establishing a comprehensive and consistent approach to registry staging practices in England ( 5 , 64 ). The STaR project in Australia and the Collaborative Staging system in the USA are two other examples of national initiatives that enabled collection of consistent national-level stage data ( 11 , 34 , 44 , 49 ). However, both were short-lived projects and are no longer routinely used. Such temporal changes in population-based cancer staging have implications for comparability, as the stage data collected by one method for specific period is not necessarily comparable to data collected subsequently by other approaches. In the absence of national-level prioritisation, PBCRs in different jurisdictions may lack a consistent and comparable approach to population-based cancer staging, such as using different or multiple staging classification systems ( 48 , 52 ). Though the use of multiple different staging classification systems by a PBCR may be intentional to manage data gaps and issues with completeness of stage data ( 48 ). Of the 62 European PBCRs examined by Minicozzi et al. ( 48 ), 24 collected stage data using two or more of TNM, EoD, and condensed TNM staging classification systems. Considering this, there would be value in the development of an international tiered framework for the collection of staging that considers data sources, resources, collection and validation, and outputs. Appropriate detail should be included to facilitate the application of the framework at the jurisdictional level, and provide relevant context to the interpretation of staging data when undertaking benchmarking and comparative analysis.
Despite being the most common staging classification, the complex staging criteria of the AJCC/UICC TNM system poses well-described challenges to the collection of complete stage data by PBCRs and has driven the development of the various simplified staging systems ( 7 , 11 , 49 ). The use of different staging classification systems presents a major barrier to the comparability of stage data ( 5 , 7 , 43 , 45 , 53 ). To allow for comparisons, staging conversion algorithms have been developed and utilised to convert stage data from one staging classification system into another. Whilst more challenging to collect, individual TNM stage values capture highly detailed tumour information that can be more easily converted into other staging classification systems, through mapping of individual T, N and M categories against stage categories of other staging classification systems ( 5 ). Articles have developed and used algorithms to convert from AJCC T, N and M categories into Condensed TNM ( 48 ), SEER Summary Stage 2000 [as a surrogate for EoD stage groups ( 5 , 43 )], and EoD ( 53 ) for specific tumour types. Combining the individual TNM categories loses the more granular stage information, making it harder for AJCC TNM summary stage groups to be converted into other staging classification systems ( 5 ). The opposite conversion from less detailed simplified staging systems to more detailed staging systems cannot easily be done without the addition of supplementary stage information ( 5 , 53 ). As EoD captured less detailed tumour information, there were difficulties in converting to other more detailed staging systems (like TNM) in the absence of supplementary stage information ( 5 , 7 , 43 , 45 , 53 ). Though the CS system mapped the localised, regional, and distant groups of SEER Summary Stage with TNM stage groups, the algorithms required additional site-specific factors (such as tumour grade, hormone receptor status, and treatment response) that were not captured by EoD ( 5 , 53 ). As a result, other PBCRs have not been able to use the same algorithms to convert from EoD because they lack complete dataset collected by the CS system, which consists of data items required to determine both the AJCC TNM and SEER Summary Stage ( 5 , 53 ). Regardless of the staging systems involved, using stage conversion algorithms is associated with misclassification risks, which generally varies by tumour type and according to how well stage groups map to one another ( 5 ).
There was a lack of consistent terminology regarding cancer stage data collected for population-based analyses. This has consequences in not knowing if the similar terms have the same meaning or something else entirely ( 65 ). “Population-based cancer stage” is a term that has been used in the literature ( 35 , 43 ), however it is not utilised universally. “Stage at diagnosis” is another commonly used term ( 5 , 6 , 34 , 46 , 48 , 52 ) however without addition of terms like “population-based” or “population-level”, it can be unclear if this refers to the collection of individual-level stage at diagnosis. Two further terms that were outlined were “registry-derived stage” and “registry defined stage.” Due to its purpose to capture consistent population-level stage data across Australian PBCRs, RD stage is clearly defined as the “best estimate of summary TNM stage at time of diagnosis as derived by cancer registries from available data sources for use in population data analysis” ( 12 ). Since the project's conclusion, RD stage has been used to refer to the specific datasets collected during the STaR project ( 11 , 34 ) however it is not a term that is used outside of Australia ( 11 ). Registry defined stage is a similar term used by the NCRAS however it is not as commonly used or as clearly defined as RD stage ( 47 ). Due to the number of staging classification systems that exist, the name of the specific staging classification system may be used instead. However, the interpretation of terminology may be complicated as similar terms may be used in different contexts. For example, “extent of disease” is used to describe the anatomical extent of cancer growth and spread ( 7 , 10 ) in addition to its use to describe the locally developed Extent of Disease staging systems ( 5 , 43 , 48 , 53 ) and SEER's Extent of Disease coding system ( 51 ). Furthermore, some articles utilise four months as the period of data collection for population-based cancer stage whereas others use 120 days. Given the potential difference in number of days depending on the specific month, there is a need to clarify whether four months is equivalent to 120 days, based on a standard month of 30 days duration. A concept analysis is recommended to further review the clarity of terms and provide a clear definition for population-based cancer stage at diagnosis ( 65 – 67 ).
5. Strengths and limitations
This paper is the first published review looking into this area at the time of the literature search and there were relatively few articles included in the analyses, demonstrating a lack of literature in this field generally. Another strength was the use of comprehensive search strategy inclusive of peer-reviewed articles and grey literature. Additionally, this review provided a comprehensive summary of the approaches used for population-based cancer registry staging which will inform the development and implementation of population-based cancer staging into the WACR.
A rapid scoping review was conducted due to time constraints which resulted in simplification of processes. The identification of literature and data extraction was conducted by a single reviewer however a second reviewer assisted with screening a selection of articles by title and abstract and reviewed articles queried with the full-text review. The search for grey literature from websites was limited to a few websites and particularly for larger countries with established PBCRs and organisations with international collaborations, which risks missing literature from smaller countries and less-well established PBCRs. Grey literature was also limited to documents that are publicly available, whereas it is expected that a large amount of information on PBCR processes is not available publicly and stored internally. The inclusion of articles only in English potentially excluded evidence from eligible studies in other languages.
6. Conclusion
PBCRs used different approaches to determine cancer stage at diagnosis at a population-level. Variation exists between PBCRs in what data sources were available, how they used them, and what staging classification system was used. The lack of an internationally standardised methodology for determining population-based stage at diagnosis hinders the ability to make comparisons between different jurisdictions and countries. However, the approaches used by PBCRs to collect population-based cancer stage are limited by the resources available and the data sources that are accessible. This rapid scoping review adds to the limited literature that currently exists on the approaches used by PBCRs to determine cancer stage at diagnosis on a population-level. Additionally, it provides comprehensive background information on the collection of population-based cancer stage that will inform current work to integrate such processes into the WACR. Development of international guidelines, for example the development of a tiered framework for the collection of staging, would assist PBCRs with consistent collection of population-level stage at diagnosis and improve comparability of stage data, though consideration should be given to the significant amount of manual work that is required to collect population-based cancer stage.
Author contributions
RM, RT, KT, SS, LP conceived the study. LP, SS, RT, RM designed the study. KC consulted and provided guidance on using Research Screener. LP conducted literature searches, was the first reviewer who screened all articles, extracted, and analysed the data, and authored the draft paper. SS was the second reviewer who screened a portion of articles by title and abstract, involved in full-text queries, supervised and provided advice on the scoping review/rapid scoping review process and qualitative aspects, oversaw the analysis stages, consulted on the themes, and revised the draft paper. LP, SS, RT, RM reviewed articles of uncertainty. LP, SS, RM, RT, KT, KC, CGG, NH reviewed the themes and revised the manuscript critically for important intellectual content and provided approval for the final version and publication. All authors contributed to the article and approved the submitted version.
This work was supported by the Cancer Network WA.
Acknowledgments
The authors would like to acknowledge and thank Vanessa Varis, the librarian at Curtin University, for her consultation on the databases and key terms to use for the searches. The authors would like to thank Mary Tallon, Research Fellow at the School of Nursing at Curtin University, for peer-reviewing the initial protocol for the review process. Special thanks to Shantelle Smith, Research/Project Officer at Curtin University, for reviewing and formatting the manuscript.
Conflict of interest
KT is employed by Cancer Network WA, which funded the project. KT provided feedback on the draft of the paper but as the funder did not determine which results were reported. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
1. Cancer Australia. National cancer stage at diagnosis data: Cancer Australia (2018). https://ncci.canceraustralia.gov.au/features/national-cancer-stage-diagnosis-data
2. Parkin DM. The role of cancer registries in cancer control. Int J Clin Oncol . (2008) 13(2):102–11. doi: 10.1007/s10147-008-0762-6
PubMed Abstract | CrossRef Full Text | Google Scholar
3. Cancer Australia. A National Cancer Data Strategy for Australia (2008). https://www.canceraustralia.gov.au/publications-and-resources/cancer-australia-publications/national-cancer-data-strategy-australia
4. Coleman MP, Forman D, Bryant H, Butler J, Rachet B, Maringe C, et al. Cancer survival in Australia, Canada, Denmark, Norway, Sweden, and the UK, 1995-2007 (the international cancer benchmarking partnership): an analysis of population-based cancer registry data. Lancet . (2011) 377(9760):127–38. doi: 10.1016/S0140-6736(10)62231-3
5. Walters S, Maringe C, Butler J, Brierley JD, Rachet B, Coleman MP. Comparability of stage data in cancer registries in six countries: lessons from the international cancer benchmarking partnership. Int J Cancer . (2013) 132(3):676–85. doi: 10.1002/ijc.27651
6. Benitez-Majano S, Fowler H, Maringe C, Di Girolamo C, Rachet B. Deriving stage at diagnosis from multiple population-based sources: colorectal and lung cancer in England. Br J Cancer . (2016) 115(3):391–400. doi: 10.1038/bjc.2016.177
7. Piñeros M, Parkin DM, Ward K, Chokunonga E, Ervik M, Farrugia H, et al. Essential TNM: a registry tool to reduce gaps in cancer staging information. Lancet Oncol . (2019) 20(2):e103–e11. doi: 10.1016/S1470-2045(18)30897-0
CrossRef Full Text | Google Scholar
8. American Joint Committee on Cancer. AJCC Cancer staging manual. 8th edition ed . New York: Springer (2016).
9. Ruhl J, Callaghan C, Schussler N. Summary Stage 2018 Codes and Coding Instructions (2022). https://seer.cancer.gov/tools/ssm/SSM-2018-GENERAL-INSTRUCTIONS.pdf
10. Berrino F, Brown C, Möller T, Sobin L, Faivre J. ENCR Recommendations: Condensed TNM for Coding the Extent of Disease (2002). https://www.encr.eu/sites/default/files/pdf/extentofdisease.pdf
11. Lawrance S, Bui C, Mahindra V, Arcorace M, Cooke-Yarborough C. Assessing a modified-AJCC TNM staging system in the New South Wales cancer registry, Australia. BMC Cancer . (2019) 19(1):850. doi: 10.1186/s12885-019-6062-x
12. Cancer Council Victoria. Cancer Stage at Diagnosis: Definition of Registry Derived Stage for use by population-based cancer registries (2016).
13. Western Australian Department of Health. Cancer incidence, mortality and survival in Western Australia, 2017 (2020). https://ww2.health.wa.gov.au/∼/media/Files/Corporate/general-documents/WA-Cancer-Registry/Recent-reports/Incidence-Mortality-and-Survival-2017.pdf
14. Western Australian Department of Health. WA Cancer Plan 2020–2025 (2020). https://ww2.health.wa.gov.au/Articles/U_Z/WA-Cancer-Plan .
15. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol . (2018) 18(1):143. doi: 10.1186/s12874-018-0611-x
16. Mallett R, Hagen-Zanker J, Slater R, Duvendack M. The benefits and challenges of using systematic reviews in international development research. J Dev Eff . (2012) 4(3):445–55. doi: 10.1080/19439342.2012.711342
17. Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med . (2015) 13(1):224. doi: 10.1186/s12916-015-0465-6
18. Peters MD, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: scoping reviews (2020 version). 2020. In: JBI Manual for evidence synthesis . Aromataris E, Munn Z, editors. JBI (2020). https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-20-12
19. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med . (2018) 169(7):467–73. doi: 10.7326/M18-0850
20. Smith J, Nels A, Emery L, Stanley M. Exploring the use of photovoice in understanding the lived experience of neurological conditions: a scoping review and reflexive thematic analysis. Int. J. Qual. Methods . (2023) 22:16094069231156344. doi: 10.1177/16094069231156344
21. Cochrane Library. Cochrane Library (2021). https://www.cochranelibrary.com/
22. Vardell E, Malloy M. Joanna Briggs Institute: an evidence-based practice database. Med Ref Serv Q. (2013) 32(4):434–42. doi: 10.1080/02763869.2013.837734
23. Figshare. Figshare (2021). https://figshare.com/
24. Center for Open Science. Open Science Framework (2021). https://osf.io/
25. Alberani V, De Castro Pietrangeli P, Mazza AM. The use of grey literature in health sciences: a preliminary survey. Bull Med Libr Assoc . (1990) 78(4):358–63.2224298
PubMed Abstract | Google Scholar
26. Haddaway NR, Collins AM, Coughlin D, Kirk S. The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS One . (2015) 10(9):e0138237. doi: 10.1371/journal.pone.0138237
27. National Cancer Institute. Surveillance, Epidemiology, and End Results Program (2021). https://seer.cancer.gov/
28. National Cancer Registration and Analysis Service. Collecting and using data (2021). http://www.ncin.org.uk/collecting_and_using_data/
29. Canadian Partnership Against Cancer. Canadian Partnership Against Cancer (2021). https://www.partnershipagainstcancer.ca/
30. Cancer Research UK. ICBP International Cancer Benchmarking Partnership (2021). https://www.cancerresearchuk.org/health-professional/data-andstatistics/international-cancer-benchmarking-partnership-icbp
31. European Network of Cancer Registries. European Network of Cancer Registries, European Commission (2021). https://encr.eu/
32. International Agency for Research on Cancer. International Agency for Research on Cancer: World Health Health Organisation (2022). https://www.iarc.who.int/
33. Chai KEK, Lines RLJ, Gucciardi DF, Ng L. Research screener: a machine learning tool to semi-automate abstract screening for systematic reviews. Syst Rev . (2021) 10(1):93. doi: 10.1186/s13643-021-01635-3
34. Meng R, Venugopal K, Thomas H, D'Onise K. Cancer staging at diagnosis data comparisons in South Australia. Sci Rep . (2020) 10(1):1008. doi: 10.1038/s41598-020-57704-5
35. Threlfall T, Wittorff J, Boutdara P, Heyworth J, Katris P, Sheiner H, et al. Collection of population-based cancer staging information in western Australia – a feasibility study. Popul Health Metr . (2005) 3(1):9. doi: 10.1186/1478-7954-3-9
36. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev . (2016) 5(1):210. doi: 10.1186/s13643-016-0384-4
37. QSR. NVivo qualitative data analysis version 20. QSR International Pty Ltd. (2020).
38. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol . (2006) 3(2):77–101. doi: 10.1191/1478088706qp063oa
39. Adamo M, Groves C, Dickie L, Ruhl J. SEER Program coding and staging manual 2022. In: U.S. Department of health human services national cancer institute . Bethesda, MD: U.S. Department of Health Human Services National Cancer Institute (2021). p. 1–264
40. Aitken JF, Youlden DR, Moore AS, Baade PD, Ward LJ, Thursfield VJ, et al. Assessing the feasibility and validity of the Toronto childhood cancer stage guidelines: a population-based registry study. Lancet Child Adolesc Health . (2018) 2:173–9. doi: 10.1016/S2352-4642(18)30023-3
41. Aitken J, Youlden D, Ward L, Thursfield V, Baade P, Hallahan A, et al. Childhood cancer staging rules for population registries, based on the Toronto Paediatric Cancer Stage Guideline. Cancer Council Queensland: Brisbane, Australia (2016).
42. Bryan S, Masoud H, Weir HK, Woods R, Lockwood G, Smith L, et al. Cancer in Canada: Stage at diagnosis. Health Rep . (2018) 29(12):21–45.
43. Cabasag CJ, Arnold M, Piñeros M, Morgan E, Brierley J, Hofferkamp J, et al. Population-based cancer staging for oesophageal, gastric, and pancreatic cancer 2012-2014: international cancer benchmarking partnership SurvMark-2. Int J Cancer . (2021). 149:1239–46. doi: 10.1002/ijc.33679
44. Collaborative Staging Task Force of the American Joint Committee on Cancer. Collaborative Staging Manual and Coding Instructions, Version 01.04.00. (2007).
45. Cunningham R, Sarfati D, Hill S, Kenwright D. An audit of colon cancer data on the New Zealand cancer registry. N Z Med J . (2008) 121:46–56.18709047
46. Gupta S, Aitken JF, Bartels U, Brierley J, Dolendo M, Friedrich P, et al. Paediatric cancer stage in population-based cancer registries: the Toronto consensus principles and guidelines. Lancet Oncol . (2016) 17(4):e163–e72. doi: 10.1016/S1470-2045(15)00539-2
47. Henson KE, Elliss-Brookes L, Coupland VH, Payne E, Vernon S, Rous B, et al. Data resource profile: national cancer registration dataset in England. Int J Epidemiol . (2020) 49(1):16-h. doi: 10.1093/ije/dyz076
48. Minicozzi P, Innos K, Sánchez MJ, Trama A, Walsh PM, Marcos-Gragera R, et al. Quality analysis of population-based information on cancer stage at diagnosis across Europe, with presentation of stage-specific cancer survival estimates: a EUROCARE-5 study. Eur J Cancer . (2017) 84:335–53. doi: 10.1016/j.ejca.2017.07.015
49. Noone AM, Schussler N, Negoita S, Adamo M, Cronin K, Cyr J, et al. Availability of TNM staging data elements in the medical record and training needs assessment: results from the 2014 SEER training needs assessment for TNM study. J Registry Manag . (2015) 42:40–7.26360105
50. Public Health England. A guide to NCRAS data and its availability (2020). https://www.gov.uk/guidance/national-cancer-registration-and-analysis-service-ncras .
51. Ruhl J, Callaghan C, Schussler N. Extent of Disease (EOD) 2018 General Coding Institute (2021). https://seer.cancer.gov/tools/staging/eod/2018Extent_of_Disease_General_Instructions.pdf .
52. Siesling S, Kwast A, Gavin A, Baili P, Otter R. Availability of stage at diagnosis, cancer treatment delay and compliance with cancer guidelines as cancer registry indicators for cancer care in Europe: results of EUROCHIP-3 survey. Int J Cancer . (2013) 132(12):2910–7. doi: 10.1002/ijc.27957
53. Stevens W, Stevens G, Kolbe J, Cox B. Comparison of New Zealand cancer registry data with an independent lung cancer audit. N Z Med J . (2008) 121(1276):29–41.18574507
54. McCowan IA, Moore DC, Nguyen AN, Bowman RV, Clarke BE, Duhig EE, et al. Collection of cancer stage data by classifying free-text medical reports. J Am Med Inform Assoc . (2007) 14(6):736–45. doi: 10.1197/jamia.M2130
55. Baicry F, Molinié F, Plouvier S, Colonna M, Daubisse-Marliac L, Grosclaude P, et al. What is the most appropriate period to define synchronous cancers? Cancer Epidemiol . (2021) 71(Pt A):101900. doi: 10.1016/j.canep.2021.101900
56. Ellis DW, Srigley J. Does standardised structured reporting contribute to quality in diagnostic pathology? The importance of evidence-based datasets. Virchows Arch . (2016) 468(1):51–9. doi: 10.1007/s00428-015-1834-4
57. Srigley JR, McGowan T, Maclean A, Raby M, Ross J, Kramer S, et al. Standardized synoptic cancer pathology reporting: a population-based approach. J Surg Oncol . (2009) 99(8):517–24. doi: 10.1002/jso.21282
58. Gill AJ, Johns AL, Eckstein R, Samra JS, Kaufman A, Chang DK, et al. Synoptic reporting improves histopathological assessment of pancreatic resection specimens. Pathology . (2009) 41(2):161–7. doi: 10.1080/00313020802337329
59. ICCR. International Collaboration on Cancer Reporting. Publications (2022). https://www.iccr-cancer.org/publications
60. Soerjomataram I, Ervik M, Fox C, Hawkins S, Yeung K, Napolitano G, et al. Canstaging+: an electronic staging tool for population-based cancer registries. Lancet Oncol . (2021) 22(8):1069. doi: 10.1016/S1470-2045(21)00188-1
61. Cancer Institute NSW. Review of artificial intelligence use in cancer registries: Registry Automated Data Extraction (RADE) project, Version 1.0. (2018).
62. Nguyen AN, Lawley MJ, Hansen DP, Bowman RV, Clarke BE, Duhig EE, et al. Symbolic rule-based classification of lung cancer stages from free-text pathology reports. J Am Med Inform Assoc . (2010) 17(4):440–5. doi: 10.1136/jamia.2010.003707
63. Warner JL, Levy MA, Neuss MN, Warner JL, Levy MA, Neuss MN. ReCAP: feasibility and accuracy of extracting cancer stage information from narrative electronic health record data. J Oncol Pract . (2016) 12(2):157–8. doi: 10.1200/JOP.2015.004622
64. Wilson H. Celebrating 10 years of the International Cancer Benchmarking Partnership: An Impact Report: Stakeholder perspectives 2009-2019, Cancer Research UK (2019). https://www.cancerresearchuk.org/sites/default/files/icbp_phase_1_report.pdf
65. Smith S, Tallon M, Smith J, Angelhoff C, Mörelius E. Parental sleep when their child is sick: a phased principle-based concept analysis. J Sleep Res . (2022) 31(5):e13575. doi: 10.1111/jsr.13575
66. Smith J, Rapport F, O'Brien TA, Smith S, Tyrrell VJ, Mould EVA, et al. The rise of rapid implementation: a worked example of solving an existing problem with a new method by combining concept analysis with a systematic integrative review. BMC Health Serv Res . (2020) 20(1):449. doi: 10.1186/s12913-020-05289-0
67. Smith S, Mörelius E. Principle-Based concept analysis methodology using a phased approach with quality criteria. Int J Qual Methods . (2021) 20:16094069211057995. doi: 10.1177/16094069211057995
Keywords: rapid scoping review, registry-derived stage, population-based cancer stage, population-based cancer registries, cancer stage, stage at diagnosis
Citation: Pung L, Moorin R, Trevithick RW, Taylor K, Chai KEK, Garcia Gewerc C, Ha NT and Smith S (2023) Determining cancer stage at diagnosis in population-based cancer registries: A rapid scoping review. Front. Health Serv. 3:1039266. doi: 10.3389/frhs.2023.1039266
Received: 8 September 2022; Accepted: 3 February 2023; Published: 8 March 2023.
Reviewed by:
© 2023 Pung, Moorin, Trevithick, Taylor, Chai, Garcia Gewerc, Ha and Smith. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Stephanie Smith [email protected]
† ORCID Li Pung orcid.org/0000-0003-0239-7120 Rachael Moorin orcid.org/0000-0001-8742-7151 Richard Trevithick orcid.org/0000-0003-2620-7221 Karen Taylor orcid.org/0000-0001-9586-9427 Kevin Chai orcid.org/0000-0003-1645-0922 Cristiana Garcia Gewerc orcid.org/0000-0002-4730-2730 Ninh Ha orcid.org/0000-0002-2789-5604 Stephanie Smith orcid.org/0000-0002-1932-6516
Specialty Section: This article was submitted to Health Policy and Management, a section of the journal Frontiers in Health Services
Abbreviations PBCRs, Population-Based Cancer Registries; AJCC, American Joint Commission on Cancer; TNM, Tumour Node Metastasis; SEER, Surveillance, Epidemiology and End Results; ENCR, European Network of Cancer Registries; STaR project, Stage, Treatment, and Recurrence project; WACR, Western Australian Cancer Registry; PRISMA-ScR, Preferred Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews; JBI, Joanna Briggs Institute; NCRAS, National Cancer Registration and Analysis Service; ICBP, International Cancer Benchmarking Partnership; MDT, Multidisciplinary Team; DoS, Degree of Spread; CS, Collaborative Stage; NSW, New South Wales; UICC, Union for International Cancer Control; USA, United States of America; IARC, International Agency for Research on Cancer; EoD, Extent of Disease; FIGO, Fédération Internationale de Gynécologie et d'Obstétrique; WA, Western Australia.
This article is part of the Research Topic
Rising Stars in Health Policy and Management: 2022
IMAGES
VIDEO
COMMENTS
Types of Sources Primary Research Synthesizing Sources Conducting Primary Research What is Primary Research and How do I get Started? Ethical Considerations in Primary Research Common Pitfalls of Primary Research Interviewing Surveying Creating Good Interview and Survey Questions Observing Analysis Analyzing Your Primary Data
Academic and Trade Journals: Academic and trade journals contain the most up-to-date information and research in industry, business, and academia. Journal articles come in several forms, including literature reviews that overview current and past research, articles on theories and history, and articles on specific processes or research.
A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period. A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis.
Types of sources for a review... Primary source: Usually a report by the original researchers of a study (unfiltered sources) Secondary source: Description or summary by somebody other than the original researcher, e.g. a review article (filtered sources)
9.3. Types of Review Articles and Brief Illustrations. EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic.
Types of sources There are many different types of sources, which can be divided into three categories: primary sources, secondary sources, and tertiary sources. Primary sources are often considered the most credible in terms of providing evidence for your argument, as they give you direct evidence of what you are researching.
The study relies primarily on reports from extant literature, ranging from primary, secondary and tertiary data sources such as the newspapers, the internet and website sources (Levy and...
Literature review that cites the study: Examples of sources that can be primary or secondary. ... Some types of source are nearly always primary: works of art and literature, raw statistical data, official documents and records, and personal communications (e.g. letters, interviews). If you use one of these in your research, it is probably a ...
In your dissertation you will need to use all three categories of literature review sources: Sources for literature review and examples Generally, your literature review should integrate a wide range of sources such as: Books. Textbooks remain as the most important source to find models and theories related to the research area.
Systematic literature review can be divided into two categories: meta-analysis and meta-synthesis. When you conduct meta-analysis you take findings from several studies on the same subject and analyze these using standardized statistical procedures. In meta-analysis patterns and relationships are detected and conclusions are drawn.
Research summaries reported in textbooks, magazines, and newspapers are considered secondary sources. They typically provide global descriptions of results with few details on the methodology. Other examples of secondary sources include biographies and critical studies of an author's work.
Primary literature may also include conference papers, pre-prints, or preliminary reports. Also called empirical research. Secondary Literature Secondary literature consists of interpretations and evaluations that are derived from or refer to the primary source literature.
Literature Review is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.
A literature review is an integrated analysis-- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.
A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories.A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that ...
Following are a few acceptable sources for literature reviews, listed in order from what will be considered most acceptable to less acceptable sources for your literature review assignments: Peer reviewed journal articles. Edited academic books. Articles in professional journals. Statistical data from government websites.
Types of Reviews. Traditional or narrative reviews: This approach will generate a comprehensive, critical analysis of the published research on your topic. However, rather than merely compiling as many sources as possible, use this approach to establish a theoretical framework for your paper, establish trends, and identify gaps in the research.
When considering literature to review, it's important to understand that different types of information sources may be critical for particular disciplines. Below are examples of different types of information sources to consider. Depending on your discipline, you may need to consider more than one type of source.
The Literature Search. A systematic review search includes a search of databases, gray literature, personal communications, and a handsearch of high impact journals in the related field. ... See our list of recommended databases and gray literature sources on this page. ... A collection of six databases that contain different types of high ...
A literature review is a comprehensive summary of previous research on a topic. The literature review surveys scholarly articles, books, and other sources relevant to a particular area of research. The review should enumerate, describe, summarize, objectively evaluate and clarify this previous research.
Other common examples of primary sources include speeches, letters, diaries, autobiographies, interviews, official reports, court records, artifacts, photographs, and drawings. Galvan, J. L. (2013). Writing literature reviews: A guide for students of the social and behavioral sciences. Glendale, CA: Pyrczak. Persaud, N. (2010). Primary data source.
The research presented in this article is a systematic review of the literature on the assessment of mathematical modeling in the setting of mathematics education published in the previous five years. This research has compiled the current best information from around the world to offer an overview of the assessment of mathematical modeling for pre-service mathematics teachers or mathematics ...
However, types of SS [actual SS, perceived social support (PSS), and others], source of actual SS, and source of PSS were not significant moderators.ConclusionTo the best of our knowledge, this is the first systematic review and meta-analysis to quantitatively investigate the association between SS and FCR in Chinese cancer patients using β ...
The paper starts with a literature review, highly needed for the solid foundation of the study. The section is divided into three subthemes: entrepreneurial education and sustainable development, sources of funding, and preconditions for starting a business. ... as opposed to various types of funding sources. Based on the analysis of ...
Literature was included if peer-reviewed articles or grey literature sources used population-based cancer stage at diagnosis, and were published in English between 2000 and 2021. Literature was excluded if they were reviews or only the abstract was available. Database results were screened by title and abstract using Research Screener.