Conducting a Systematic Literature Review in Education: A Basic Approach for Graduate Students

Though essential for graduate students’ success, academic writing remains complex for a variety of reasons. Lack of institutional support and non-transparent writing practices leave graduate students in education to depend on the support of their academic supervisors. The aim of this paper is to familiarize graduate students with the genre of systematic literature review (SLR), as it is conducted in the field of education, by providing them with a self-paced approach to writing a SLR. This approach contains goals, explanations, and recommended time frames, while at the same time suggesting deliverables to be produced that would facilitate the writing of this important part of their research project.


Introduction
Academic writing, or writing done for academic purposes, is very complex (Lavelle & Bushrow, 2007), as it needs to be well-supported, carefully elaborated, logically sequenced, and rigorously reasoned (Fang, 2021;Graff & Birkenstein, 2018).As such, academic writing continues to confuse graduate students, while fears of failure often lead them to avoid it altogether for a variety of reasons (e.g., Al-Shboul & Huwari, 2015;Bloom, 1981).Some may lack experience with academic writing due to the nature of their field (Berdanier & Zerbe, 2018;Lax, 2002).Others may have high writing anxiety (Holmes et al., 2018) or low writing selfefficacy (Martinez et al., 2011).The latter may be especially true for many graduate students in the U.S. who are international students whose English is not their first language (Sidman-Taveau & Karathanos-Aguilar, 2015).One of the biggest reasons, however, might be the fact that novice writers are expected to decode writing practices that are universal to experienced academic writers but are rarely explicitly taught to students (Lambie et al., 2008;Paré et al., 2011).While some universities provide some support, such support is sparse (Okoli, 2015), leaving graduate students to rely mostly on the feedback they receive from their academic advisors.This is problematic, as it limits the experience of graduate students, whose disciplinary learning, academic success, career advancement, and social status in academia depend on the possession of academic writing skills (Fang, 2021;Kamler & Thompson, 2006).
To develop their expertise and scholarly identity, graduate students must know how to read, produce, and publish academic writing (Pearson & Brew, 2002).To have a chance to make scientific contributions, graduate students must build on and relate to existing knowledge, making a good-quality literature review the most relevant research method they need to master (Snyder, 2019).While there are many different types of literature reviews (e.g., meta-analysis, rapid review, scoping review; Grant & Booth, 2009), we believe that systematic literature review (SLR) is especially approachable for novice researchers as it follows a rigorous methodological procedure (Fink, 2019) that lends itself well to description.Existing articles that lay out the procedure on how to conduct SLRs (e.g., Kitchenham & Charters, 2007;Okoli, 2015;Shaffril et al., 2021;Xiao & Watson, 2019), however, do not focus on the field of education, while their approach is not simplified enough for graduate students to follow independently.
The aim of this paper, therefore, is to familiarize graduate students with the genre of SLR, as it is conducted in the field of education, by providing them with a self-paced approach.This approach includes a set of well-defined steps on how to conduct the review in addition to a checklist with suggested time commitments, goals, and topics for meetings with the academic supervisor.Our approach also contains a set of suggested and explained deliverables that a student should be able to produce by the end of their review, which will later help them as they write their own SLR.

Brock Education Journal 33 (1)
Conducting a SLR in Education SLR involves a rigorous methodological procedure (Fink, 2019), where every step requires careful documentation.The procedure is both exhaustive and extensive, while at the same time being replicable.By conducting such a review, students learn how to navigate enormous amounts of information (Walter & Stouck, 2020), making the process ideal for emerging scholars in education who are often encouraged to engage in meaningful inquiry and continue to develop expertise in their chosen education-related topic while at the same time identifying gaps in the literature.
While several articles engage in explaining the process of SLRs (e.g., Kitchenham & Charters, 2007;Okoli, 2015;Shaffril et al., 2021;Xiao & Watson, 2019), some articles (e.g., Okoli, 2015) lack the detail that would make them easily interpretable by graduate students with no prior experience with SLRs.Articles that do provide a more detailed description (e.g., Shaffril et al., 2021;Xiao & Watson, 2019) do not discuss resources, including academic support staff and/or faculty, who can provide help at different steps.Additionally, these resources do not include deliverables (e.g., written documents) that should accompany each step, and most importantly, they do not focus on the field of education, where different contextual factors (e.g., historical, economic, social, etc.) often significantly influence results (National Research Council, 2002).
Considering the nature of graduate work, which requires students to work independently, especially in connection with their thesis or dissertation work, we provide a 13-step approach accompanied by a simplified checklist that graduate students can use on their own.We also provide beginner-level explanations illustrated by examples from two recent SLRs in the field of education, and follow that with practical tips.Finally, unlike some scholars (e.g., Shaffril et al., 2021) who posit that only high-quality articles should be included in your SLR, we describe how grey literature (Paez, 2017) could be beneficial in certain cases.
It is important to understand that even though we recommend a certain order of steps in this paper, conducting a SLR is by no means linear.Instead, it is an iterative process during which students will need to stop and repeat steps before being able to move on to the next phase.

A Step-by-Step Approach for Graduate Students
Table 1 outlines the 13 action steps described in this paper, the goal for each step, the time recommended to execute it, and potential consultations that can help in the process.We also leave a blank column, so students can use it to set their own deadlines, as the checklist is designed to be easily printed and used by graduate students.In the text that follows the table, we define and describe each step, followed by examples from two SLRs published by the authors: Pantic and Clarke-Midura (2019) and Hamilton and Morgan (2022).Note.The process of conducting a search for a SLR is iterative.Steps marked with an asterisk (*) may need to be repeated several times before making a final decision.

Pantic & Hamilton
Step 1: Start a Breadcrumbs Document The first step for conducting a SLR is creating a document that we will refer to as a breadcrumbs document 1 .We use the term breadcrumbs document to refer to a running file in which students will write the dates and make notes on the decisions made along the way and why.This document should be located inside a folder in which students will save all the searches during this process.Hamilton and Morgan's (2022) breadcrumbs document, for example, included ideas discussed with colleagues and/or advisor(s), search terms utilized in database searches, notes, and any other related thoughts pertaining to the project at hand.
It is important for students to take careful notes of the whole process, as SLRs must be guided by careful protocols and standards (Shaffril et al., 2021).The idea is to create a literature review with a carefully described methodology so other scholars can replicate the process and get similar results.Considering the importance of the breadcrumbs document, we advise students to update this document every day they work on their literature review and save multiple copies of it to prevent losing important information.
Step 2: Decide on the Research Topic and Develop a RQ Brock Education Journal 33 (1) inclusion criteria, assess which articles qualify to be included in the SLR, and later helped with analysis.
Step 3: Locate and Examine SLRs Related to Chosen Research Topic During this step, students will want to locate SLR articles that relate to their existing topics, but any other articles in the field of education will also help them envision the main components of this genre.Students should skim through such articles to identify patterns and keep notes on these observed patterns in their breadcrumbs document.They should also remember to download these articles or save them in an area where they can easily reference them as they make decisions on the progress and direction of their SLR.Step 4: Determine Inclusion (and Exclusion) Criteria2 * According to Patino and Ferreira (2018), inclusion criteria describe features (e.g., demographics, geographic locations, etc.) of the target population that help conduct the review.
Such criteria should also include published language, a well-defined topic, and whether the articles included are exclusively peer-reviewed or not.On the other hand, exclusion criteria describe the features that might obstruct the study or increase the risk of undesired results (Patino & Ferreira, 2018).For example, studies published more than 5 years ago or studies that utilize a certain methodology might not be appropriate for inclusion.As an example, Hamilton and Morgan (2022) started with recent peer-reviewed articles but expanded to more earlier work due to the scarcity of research found in the initial search.Pantic and Clarke-Midura (2019), on the other hand, wanted a comprehensive and chronological literature review, which is why they included every article published on their defined topic.Students may wish to discuss their inclusion and exclusion criteria with their advisor(s), as they may help them refine the criteria at this point3 .

Pantic & Hamilton
The process of defining inclusion criteria is iterative.The students' original search may yield too many or too few articles 4 .Certain research topics may be emerging and/or understudied and therefore may only yield a small number of results.If that is the case, a discussion about the literature search with the student's advisor(s) and/or an academic librarian5 may be required at this point so the inclusion criteria can be readdressed.It could be that the student needs to expand the time span, or they may need to include grey literature in addition to peerreviewed articles.Paez (2017) defines grey literature as literature that has the potential to reduce publication bias as it is not published in commercial or high-impact journals.Such literature may include dissertations, theses, government reports, and so on.Expanding the literature search to include grey literature might prove worthwhile, as it could potentially yield more comprehensive results.On the other hand, students may have too many articles.If this is the case, they may need to adjust their inclusion criteria to include more recent publications, or they may only include articles from peer-reviewed journals.Students should make sure to write about any changes they make and explain the rationale for making them in their breadcrumbs document.
Step 5: Identify and Refine Search Terms * Next, students will want to identify the search terms they will utilize in their search.Search terms are terms related to each objective for the searching process of their chosen topic of study (Shaffril et al., 2021).They are entered into each browser's database to initiate the search.Some commonly used databases in education are ERIC, EPSCO, Education Source, Google Scholar, and Web of Science (as recommended by Xiao & Watson, 2019).To identify which search terms would be most fruitful, students can speak to their advisor, the librarian, and/or they might even look at related SLRs and identify the search terms that other authors used.A content librarian who specializes in a student's research area might be of great assistance because they can help identify additional search terms and or teach students how to use the databases.A thesaurus may also be helpful, as it may offer closely related words they may wish to include.
As an example, Hamilton and Morgan (2022) used the following search terms: "Indigenous," "making," "youth," "science," and "technology," while Pantic and Clarke-Midura (2019) used "women," "computer science," and "retention."Interestingly, Pantic and Clarke-Midura (2019) started their SLR by using the term "underrepresentation," but it quickly became clear that underrepresentation was a twofold problem.As a result, they had to go back a few steps and redefine the search terms and inclusion criteria, while at the same time refining the topic and narrowing down the research question.In other words, finding the right search terms took 4 Determining if results are too limited or too broad is at the discretion of the student and their advisor.
Brock Education Journal 33 (1) several iterations.Kitchenham and Charters (2007) recommend that scholars conduct the search in an iterative manner, starting with preliminary searches that would allow them to assess the volume of relevant studies and trial searches with various combinations 6 .In either case, students need to describe the details of their search explicitly (Okoli, 2015), first in their breadcrumbs document and later in their SLR paper, as those details are part of the rigour for this methodology.
Step 6: Conduct the Search * During this step, we advise students to create a folder that will include all the literature located during their search.We suggest they open a subfolder for each database.Students should conduct the same search in each one of the databases and keep track of the number of articles they locate.One advantage of database searches is that they are digital, and they can search various years of the same periodical at the same time (Behrens et al., 2012).However, databases are not the only choice they have.For example, Hamilton and Morgan (2022) explored article databases in the fields of learning sciences and Indigenous education, including Education Source, PsycINFO, and JSTOR, as well as journal databases such as the Association of Computing Machinery (ACM) and the Journal of American Indian Education (JAIE).
Their original search yielded 333 articles.
Pantic and Clarke-Midura (2019) started the search in the ERIC and Education Source databases, which resulted in 34 studies.To expand the list, they further looked at reference lists of identified literature reviews as well as articles that cited the reviews in question and found 48 more articles.Finally, Google Scholar and ACM Digital Library searches were conducted with the same descriptors and limitations.This final search provided 11,000 and 28,572 titles, respectively, which were sorted by relevance and inspected 10 Search Engine Results Pages (SERP) deep (a total of 100 titles each).This example demonstrates that sometimes students will have too many results in places like Google Scholar, which is why they need to make decisions on how many SERP they will inspect by ordering them by relevance.If they find repeated articles, it may be an indication that they started to reach saturation, which is a sign 6 Preliminary and trial searches will help students refine their inclusion criteria.Identifying search terms is also iterative and may be repeated several times before students are certain that they have the right combination of articles.After conducting the initial search, students may need to expand their search terms to be more inclusive and representative of certain populations.For instance, Hamilton and Morgan (2022) expanded their search terms to include many of the various identifiers scholars use to refer to Indigenous and/or Aboriginal peoples.
They performed this step by meeting with a content librarian, who helped them brainstorm a multitude of possible identifiers as well as examine key literature (e.g., seminal works) to identify the search terms scholars in their field readily use.
Pantic & Hamilton that they may be close to finishing the initial search.If, however, the initial search yields a substantial amount of literature, students may wish to consult their advisor(s) and/or the content librarian about additional inclusion criteria they can use that might help them further refine the search.If the initial search yields very few articles, they may have to expand upon the initial inclusion criteria so they can hopefully include additional related articles.
Step Step 8: Refine the List After conducting a thorough search, the next step should be to begin refining the list of literature students had previously located.There are likely to be duplicate articles that students will need to account for at this time.It is vital that they keep track of any articles they eliminate and the rationale for doing so.In other words, they need to write down how many articles they deleted and why.These steps need to be thoroughly described in the methods section to demonstrate rigour and systematicness in the process of conducting the SLR.Students need to ensure that others can perform the same steps and yield the same results.As mentioned above, Pantic and Clarke-Midura (2019) found 282 articles after their initial search.After comparing the folders of articles downloaded from six different databases, they found 12 duplicate articles.
Step 9: Read Article Abstracts For this step, students will read each of the article abstracts to make sure that they have the potential to answer their RQ.The goal here is to keep refining the list of articles students found, make sure they fit the inclusion criteria (Patino & Ferreira, 2018), and minimize time spent on irrelevant articles.If the article does not fit the criteria, we advise students to remove 270 articles.As they were reading, they kept referring to the RQ and the inclusion criteria they defined.This process resulted in the final number of 102 titles that got included in their SLR.
Step 10: Read Once the student is satisfied with their search terms, inclusion criteria, and the number of found articles, they should proceed to read the articles left.The process of reading, writing, and discussing this literature with their advisor(s) and colleagues will help them gain expertise in their chosen research topic.While students engage in the reading process, they should take notes in their breadcrumbs document on any ideas they have or patterns they observe.
Additionally, we advise students to begin several other documents (or deliverables) that will later help them with the writing process.One of the first documents is likely to be a matrix, a spreadsheet, or a "working bibliography" (Behrens et al., 2012, p. 304) where they will keep track of the bibliographic, methodological, and result information from each article.Some headings of this spreadsheet might include study population, age of participants, type of study (e.g., literature review, empirical study, report), year the study was conducted, instruments used to collect data, et cetera.The second document should be a document where students take notes related to the findings of each study and/or write down possible quotes they could later use in their SLR.This process is called mining.Behrens et al. (2012) define mining as the process during which you read and extract information and ideas that you can use in your paper.Students can do this inside the spreadsheet, but it is more practical to have a separate document, as long as they keep track of where the notes come from.
As an example, while conducting their SLR, Pantic and Clarke-Midura (2019) had four running documents: ideas for a problem statement, running notes on findings, and an Excel document (matrix) with bibliographic and methodological information on each article, in addition to the breadcrumbs document we mentioned above.Each document was helpful in retrieving information quickly during the writing process as they were digital, and we were able to search them for themes quickly and accurately.An example of a matrix might look something like Table 1 found in Hamilton and Morgan's (2022) article, in which they provide a summary of settings, technologies used, and learning artifact(s) created from the studies they located in their SLR (p.419).
Step 11: Search for a Theoretical Framework SLR findings are usually synthesized using a theoretical framework.Grant and Osanloo (2014) define the theoretical framework as a "blueprint" on which researchers can build their study, a Pantic & Hamilton guide that provides the structure with which they can approach their study philosophically, epistemologically, methodologically, and analytically.To start the search for a theoretical model, it may be useful to locate the SLR(s) and identify how they structured their findings (e.g., tables or figures).Seeing such a structure may help the student with their own organizational structure.Students may wish to ask themselves: How did the author(s) expand upon their chosen theoretical models and/or frameworks?Additionally, students may search for seminal works related to their research topic to find a good framework that will help them synthesize literature.Any observations about the commonalities and differences students find, they should write about in the breadcrumbs document.Next, students may write about how these candidate frameworks might (not) work in their SLR or check in with their advisor(s) to see if they are headed in a promising direction.
As an example, when Pantic and Clarke-Midura (2019) worked on their SLR, they came across Tinto's (1993) model of institutional departure, which explained why students left universities.
They used the model as classification criteria to organize the literature in their SLR.When Hamilton and Morgan (2022) conducted their SLR, they organized their findings according to Barajas-López and Bang's (2018) principles of Indigenous making and sharing.
Step 12: Chart Themes In this step, students should begin thinking about how the papers they included in their SLR relate to and/or expand on the theory they have chosen.They should begin identifying ways they can organize the findings from their SLR while using the framework as classification criteria.They should look for patterns that emerge from their findings.This step can be done on paper, or students could consider using qualitative analysis software such as MaxQDA (https://www.maxqda.com/),Dedoose (https://www.dedoose.com/),or NVivo (https://lumivero.com/resources/support/getting-started-with-nvivo/).Students must remember to write any conclusions in their breadcrumbs document8 .
For example, Pantic and Clarke-Midura (2019) printed all the notes and used the whiteboard in their office to write down all the main concepts from the findings.They then used Venn diagrams to group the concepts on the board based on the theoretical framework.This process helped the authors organize their thoughts and start writing the results section of their SLR.
Meanwhile, Hamilton and Morgan (2022) wrote about their findings in their shared breadcrumbs document and organized them into a shared matrix while performing frequent check-ins with each other throughout the writing process.Regardless of which tool they decide to use, students should not write without having a clear idea of how the findings from their SLR go together.
Step 13: Write A Sequence for Academic Writing.Roschelle (2016) is another great resource when it comes to academic writing9 .

Conclusion
Academic writing is rarely explicitly taught to graduate students (Lambie et al., 2008;Paré et al., 2011), while support systems at universities are scarce (Okoli, 2015).This is problematic, as such skills are crucial for students' disciplinary learning, academic success, career advancement, and social status in academia (Fang, 2021;Kamler & Thompson, 2006).What is more, graduate students' participation in academic writing is surrounded by confusion and fear of failure (Al-Shboul & Huwari, 2015;Bloom, 1981), which is not supporting the development of students' expertise and academic identity.
Instead of having students depend on advisors' limited time and feedback, we need to provide writing resources for students that may drive one-on-one interactions with faculty (Holmes et al., 2018).In this paper, we present one such resource that has the potential to expand the landscape of academic writing in academia by (a) training students in conducting systematic research of existing literature (aka SLR) on a topic of interest and (b) teaching them how to make theoretical and empirical contributions by building on or relating to existing scientific knowledge (Snyder, 2019).While conducting SLRs can be difficult and time-consuming, the effort and time spent can prove worthwhile when it comes to building graduate students' expertise and academic identity.By learning how to conduct a SLR, students can build the foundation of disciplinary knowledge while developing their academic writing skills.
In our opinion, a SLR is an ideal place to start students' training in academic writing as it lends itself well to detailed description due to its rigorous procedure.Therefore, the main contribution of this paper is the resource it provides for students to use on their own or with the help of the advisor.The resource approaches the writing of SLR in 13 steps, which are defined, described, and illustrated with examples from two SLRs published by the authors of this paper.We believe that such resources will motivate students to work independently with less fear and confusion and to use their time with faculty in a more constructive manner.

it 7 .
If an article answers the RQ, students should keep it and read it in Step 10.At this point, students should explain their rationale for keeping and/or eliminating each article in their breadcrumbs document.As an example, Pantic and Clarke-Midura (2019) read the abstracts of Brock Education Journal 33 (1)

Table 1
Approach for Conducting SLR: Checklist for Graduate Students in Education Hamilton and Morgan (2022)2019)andHamilton and Morgan (2022)started their process of writing the SLRs by consulting journals such as Review of Educational Research (https://journals.sagepub.com/home/RER)and Educational Research Review (https://www.sciencedirect.com/journal/educational-researchreview),as well as the International Database of Education Systematic Reviews (IDESR) (https://idesr.org/).
7: Repeat Steps 4-6 As we mentioned throughout the text, defining the right scope of the SLR might require students to conduct several rounds of refinement.Students are advised to repeat Steps 4, 5, and 6 if they need to refine the inclusion criteria, search terms, and or the search itself before moving onto Step 8. Students should move to Step 8 once they are certain that the search for articles is complete.For more specific advice on how to recognize when a step needs to be repeated, how to should conduct each iteration, and who can support you during that process, please revisit Steps 4, 5, and 6 in this paper.
Graff and Birkenstein (2018)19 have enough material to write a high-quality SLR.Students should refer to their notes often as they write, but they should also refer back to any charts they made to keep themselves organized and focused.Most SLRs have tables that offer a full list of articles per identified theme (seePantic and Clarke-Midura, 2019for examples).It might be helpful to start by building those tables first.As they write, students should make sure they are systematically synthesizing findings across different studies(Snyder, 2019), and not focus on one study at the time.AsGraff and Birkenstein (2018)state, learning how to synthesize is art on its own; therefore, students should not hesitate to ask for help.Most universities have writing workshops or on-campus writing centres that students could take advantage of.Some books can also be very beneficial in teaching the mechanics of academic writing.To that end, we recommend two books that we found useful during this process: Graff and Birkenstein's (2018) They Say / I Say: The Moves That Matter in Academic Writing,and Behrens et al.'s (2012)