This paper is based on a presentation at the Health Libraries Australia and Health Libraries Inc. National Conference on October 19th, 2023 by the authors.
Abstract
At Federation University Australia, the number of queries received by the library about conducting scoping and systematic reviews from academics and Higher Degree by Research students has increased. In response, Liaison Librarians developed an online subject guide and ran a series of webinars. The webinars were then adapted to a series of longer workshops with more hands-on activities. Currently, an online open textbook with interactive elements is being written to allow people to work through content at their own pace. A further series of activity based digital workshops in an online flipped classroom model is being planned, along with other initiatives such as a Community of Practice. To be successful, this project required librarians to develop their own digital skills and knowledge, as well as the participants’. The digital content and workshops have increased equity for participants as they can access it no matter their location or other commitments. Cross-campus collaboration has increased and the library is continuing to evolve and adapt to meet the needs of academics and students.
Keywords
Systematic reviews, Open Educational Resource, Digital, Equity, Collaboration
Introduction
In 2020, the Liaison Librarians across Federation University Australia’s four campuses noticed a sharp increase in demand for support with complex search strategies for scoping and systematic reviews. These requests came from academics, Masters and Higher Degree by Research (HDR) students, with many generating individual appointments with librarians held face-to-face or online. Many students informed us that when they asked their supervisors for support, they were simply referred to the library. In addition to help with searching, they were also requesting assistance with review methods outside the library's scope, such as data extraction, indicating a lack of understanding about what the library's role is in supporting reviews. It was at this point we realised that while supervisors are experts in their discipline, this does not necessarily mean they are also experts in library research areas such as developing complex search strategies, using databases or screening programs. In addition, there seemed to be some confusion about the difference between systematic reviews and searching systematically. Consequently, we realised we needed to rapidly evolve as a team if we were going to deliver the specialised support required.
First steps
Subject guide
We first developed a subject guide about reviewing the literature in a systematic way. Initially we referred queries to guides from other institutions, but we felt we needed a Federation University guide with content relevant to our students and academics. This meant people could access support and guidance from us at any time, from anywhere, and were not limited to librarian availability. This was particularly important as Federation’s typical student is older, and therefore more likely to have work or caring responsibilities (Stone & O'Shea, 2019). Additionally, as we are a regional university, they are located across Victoria. We structured the guide according to the stages of a review, and included the most common review types we were asked about. As we are not experts in academic writing, we collaborated with Learning Skills Advisors. Their role is to develop students’ academic writing skills and they provided content about the differences between traditional literature reviews and systematic reviews, and the style and structure of writing.
Creating this guide required us to upskill and learn rapidly, as we needed to ensure we provided accurate advice. We recognised we had a great deal to learn about conducting reviews as we were new to this area and had limited previous experience. We attended professional development webinars and watched videos from other institutions. We also attended the five-day annual course conducted by the Australian Evidence Based Practice Librarians’ Institute. Finally, we read extensive journal articles and books, frequently discussing what we had learned, and seeking feedback from each other on search strategies we were developing or had found in the literature.
The guide was launched at the Federation University Higher Degree by Research Conference in July 2022. We ran a one-hour in person session to showcase the guide and inform students about the content included, and we received many questions and comments. As of November 2023, the guide has received over 4800 views. It has been edited and updated as required to reflect the changing needs of our academics and students, such as adding in more types of reviews or online tools that we feel may be helpful.
Webinars
We then delivered two series of webinars for HDR students and academics. These allowed participants to ask questions as they arose and follow along with content and skills we demonstrated. In the first series, we ran a webinar on each stage of the review process, again collaborating with Learning Skills Advisors on the content about writing. Running them as a series allowed people to attend those of most interest. We recorded them and made the videos available via a page on the subject guide. These sessions were primarily focused on delivering information to participants through demonstrations and content on the subject guide, for example searching databases or using reference management software. The attendance for each webinar was between five to ten attendees and while they were well received, participants expressed a desire for opportunities to have hands-on practice in addition to the content.
Our next webinar series consisted of three longer workshop style events of half a day each. These were targeted at HDR students and promoted through the Graduate Research School and the University’s Research Centres. In response to feedback from the first series, we planned to include hands-on activities that used some of the techniques and tools we demonstrated in addition to delivering content. For example, in the session on screening we placed attendees into break out rooms with a research question and the titles and abstracts of three relevant articles. We asked them to discuss if they would include or exclude these studies and present their reasons to the whole group. These workshops were not recorded, as we felt there would be little benefit to watching recordings due to the hands-on nature of the sessions and we wanted to encourage people to attend live. While the workshops were long, we felt the inclusion of activities justified the length and would help break the sessions up. However, there were still some activities we ran out of time for. The workshops had excellent attendance with approximately 45 students coming to each, and we received positive feedback about them, particularly the activities.
Student appointments
This year, Masters of Health students undertaking a research unit had been assigned an assessment that included a scoping review component. While the assessment was titled a ‘scoping review’, the only guidance provided to students was the article on developing a methodological framework for scoping studies by Arksey and O’Malley (2005), and the PRISMA-Sc was not used to guide their reporting, as is best practice. Exemplar reviews given to students by supervisors often had poor quality search strategies reported. These students made many individual appointments, were highly stressed, and confused about what was required for the assessment. We also saw third-year nursing students who were assigned a heavily modified version of a scoping review, in which they were required to search systematically but not follow a formal screening process, and then produce a modified PRISMA flowchart. Naturally, these students were extremely distressed about their assignments.
To manage the time commitment of the large number of appointments, we reached out to the coordinators of these units and offered to run classes to support students in a more sustainable way. This was met with enthusiasm by the academics, and we ran online classes on topics such as the structure of a scoping review, searching systematically, and using reference management software. For one unit, we were even asked to teach tutorials by ourselves for two weeks due to staffing issues (but without the pay of a sessional tutor!). However, while students attended these classes, there was minimal engagement. They did not contribute to the planned discussions, even in the chat function of the online meeting platform. This lack of engagement meant we had to discuss the topics between ourselves to explore the concepts and tease out solutions to some of the issues. These classes did not have the effect we were hoping for, as we still received many students booking individual appointments.
We then had extensive discussions between ourselves, the Learning Skills Advisors, and Learning Designers about the issues we saw in these assessments and how best to manage these while providing much needed support for students. Together, we provided feedback to coordinators and discipline leads on the issues we identified and the confusion and stress of the students.
Current support
At the time of writing in November 2023, we are delivering a series of ten half hour webinars on searching systematically covering the following topics:
• Research question frameworks, keywords, and synonyms
• Seed papers
• Subject headings and keywords
• Combining terms and advanced operators
• Searching databases to test the search strategy
• Evaluating the search strategy
• Translating the search strategy
• Grey literature
• Citation tracking
• Reporting the search strategy
These prioritise hands-on activities and active learning to promote knowledge retention and student engagement (Rossi et al., 2021), while providing opportunities for discussion to enable student reflection on their understanding (Hokanson et al., 2019). The multiple shorter webinars reduce cognitive fatigue (Shail, 2019) while allowing us to cover the content comprehensively. These changes were informed by feedback from the participants of previous webinars. At the conclusion of the series, we will ask participants what further content about reviews they are interested in and how we could improve sessions to help plan our future offerings. In response to feedback already received, we plan to make future sessions longer to allow more time for activities and discussions.
Finally, as part of a realignment at Federation University Library, the Liaison Librarians have been assigned specialisations according to emerging needs, one of which is a Review Protocol specialist. While these roles and their specific responsibilities are currently being developed, this role will provide in depth support for students and academics conducting reviews.
Future plans
In response to feedback from students and academics we are developing an online open Creative Commons (CC-BY) licensed textbook. This Open Educational Resource (OER) is aimed at conducting systematic and scoping reviews and developing practical search skills using interactive elements, such as videos, H5P activities, and quizzes. This text will be an exemplar in our university wide OER project run through the Council of Australian University Librarians. We will also run a series of activity based workshops using a flipped classroom model where students will work through material from the OER before attending (Reidsema, Hadgraft, & Kavanagh, 2017). It is hoped this will better prepare students for the workshops, increase engagement and enable them to clarify any issues to reinforce their learning (Cuetos, 2023). Finally, given our students are spread across the state studying on campus or online, we believe the flipped classroom model will be a more equitable learning experience (Willis, 2017).
After we publish the OER and conduct the online workshops, we plan to gather feedback from HDR students and academics. We will refine how we conduct webinars and review and update the subject guide. Peer learning sessions with our Liaison Librarian colleagues are also a priority to increase their knowledge and confidence when supporting students and academics with complex search strategies (Stark & Aiello, 2021).
We are also collaborating with unit coordinators to consider how to meet the needs of Masters students conducting reviews. We need to promote student engagement and understanding while maintaining a realistic time commitment by librarians. Some ideas we have so far include changing the assessment from a scoping review to a rapid review, which can be conducted in a shorter timeframe (Garrity et al., 2020), creating a Community of Practice for students to empower them to learn from and with each other (Mercieca, 2017), or providing professional development for academics regarding the stages and process of a quality review. Any of these potential initiatives will require reflection on their effectiveness for student learning.
Equity and inclusion
Throughout this process, we aimed to increase equity and inclusiveness for students and academics. Students could access webinars and support regardless of their location or other commitments. This was an important consideration, as Federation is a regional university with students located across Victoria. Many also have work and caring responsibilities, so online classes are more accessible and create greater equity (Stone, 2022).
While digital content and classes are beneficial for student equity, it created some challenges such as retaining student engagement and interaction, which is a common issue (Tertiary Education Quality and Standards Agency, 2020). It was important that students did not get an inferior experience due to classes only running online, so we trialled various ways of maintaining interaction, connections, and participation. Managing this is an ongoing learning process for us as we take in feedback and experiment with ways of creating interactive content and experiences.
Open Educational Resources help promote education as a human right and enable equity, inclusion and lifelong learning (United Nations Educational Scientific and Cultural Organization, 2019). They disrupt the commercial academic publishing model, as profits and sales are not a consideration, but information is freely shared with all who need it (Hollich, 2022). Our students will have access to the information for no cost, even after graduation, as will anyone in the world with an internet connection. Other libraries and academics will also be able to reuse and adapt our content to suit their own context. Our OER has the potential to increase equity across the wider university, as we are promoting their use, creation, and adaptation to all our academics. Leading by example means we can demonstrate what is possible and the benefits to students, and we hope they will be inspired to create and adapt OERs as well.
While we work to create equity and inclusion in our support for conducting reviews, offering online only content means we need to be mindful of the potential digital divide. People in very remote areas and on a low income are more likely to be digitally excluded, and those with a disability or not working can find the cost of reliable internet or technologies challenging (Thomas et al., 2023). As Federation University is regional, our students may live in remote areas with unreliable internet, and as students they may also struggle to afford the technology or internet required to adequately participate in webinars. We need to keep this in mind when developing our support in the future.
Lessons learnt
What have we learnt that could be useful for librarians in a similar situation to our own? Federation University Library is small but mighty, and we know there are others like us with big ambitions and limited resources.
• Upskill! If, like us, you work in a small institution, you may be limited in the amount of peer learning available. However, this does not mean you cannot learn. Attend any webinar or professional development available in your budget, read all the articles and books on conducting reviews you can find, look at material such as subject guides or videos created by experts, and if possible, try to find a mentor from another institution. You will have a steep learning curve, and it will take time, but it is worth it for the increase in your confidence and understanding.
• Collaborate! In small institutions like ours, librarians do a bit of everything, meaning we are all busy and pressed for time. Working with other interested colleagues means the workload is spread more equitably and doesn’t impose an undue burden on one person. Furthermore, seek out other support staff or academics who might be interested in working with you, as people from different areas often look at things in ways you had not thought of. If you have multiple campuses, meet online. We learnt during lockdowns that being based at different locations is no barrier to collaboration, so make use of this now.
• Boundaries! Any healthy relationship needs clear boundaries, and the relationship between librarians and academics is no different. Setting expectations about what is in and out of the library’s scope can avoid misunderstanding and requests to provide services and support that are not your area of responsibility. Consider what is feasible and manageable, especially if you have a small staff. This could be documented and made available to academics before commencing formal support. Learn from our mistakes!
The process of developing support for conducting reviews has led us to continually evolve and adapt. As our academic and student needs have changed, we have altered what we do to ensure they are equitably supported. The content we teach, the way we deliver it and our support for individuals and teams is continually evolving as we meet with them and receive feedback, and as we ourselves learn new skills. We are supporting the curriculum and courses to evolve, as we provide feedback in turn to academics based on what we observed. In a constantly changing higher education and research ecosystem, one thing we are sure of is that our support services for conducting reviews will be in a constant state of adaptation and evolution.
References
Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32. https://doi.org/10.1080/1364557032000119616
Cuetos, M. J. (2023). Application of the flipped classroom model to stimulate university students' learning with online education. Journal of Technology and Science Education, 13(1), 368-380. https://doi.org/https://doi.org/10.3926/jotse.1806
Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., Affengruber, L., & Stevens, A. (2021). Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. Journal of Clinical Epidemiology, 130, 13-22. https://doi.org/10.1016/j.jclinepi.2020.10.007
Hokanson, S. C., Grannan, S., Greenler, R., Gillian-Daniel, D. L., Campa, H., & Goldberg, B. B. (2019). A study of synchronous, online professional development workshops for graduate students and postdocs reveals the value of reflection and community building. Innovative Higher Education, 44(5), 385-398. https://doi.org/10.1007/s10755-019-9470-6
Hollich, S. (2022). The unrealised promise of OER: An exploration of copyright, the open movement, and social justice. In C. J. Ivory & A. Pashia (Eds.), Using Open Educational Resources to promote social justice (pp. 3-22). Association of College and Research Libraries. https://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/booksanddigitalresources/digital/9780838936771_OA.pdf
Mercieca, B. (2017). What is a community of practice? In J. McDonald & A. Cater-Steel (Eds.). Communities of practice: Facilitating social learning in higher education. (pp. 3-25). Springer. https://doi.org/10.1007/978-981-10-2879-3_1
Reidsema, C., Hadgraft, R., & Kavanagh, L. (2017). Introduction to the flipped classroom. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), The flipped classroom practice and practices in higher education (pp. 3-14). Springer. https://doi.org/10.1007/978-981-10-3413-8
Rossi, I. V., de Lima, J. D., Sabatke, B., Ferreira Nunes, M. A., Ramirez, G. E., & Ramirez, M. I. (2021). Active learning tools improve the learning outcomes, scientific attitude, and critical thinking in higher education: Experiences in an online course during the COVID-19 pandemic. Biochemical and Molecular Biology Education, 49, 888-903. https://doi.org/https://doi.org/10.1002/bmb.21574
Shail, M. S. (2019). Using micro-learning on mobile applications to increase knowledge retention and work performance: A review of literature. Cureus, 11(8), e5307. https://doi.org/10.7759/cureus.5307
Stark, R. K., & Aiello, M. T. (2021). Solo but not alone: Self-directed online peer education as a way to increase professional knowledge and skills. Journal of Hospital Librarianship, 21(1), 1-10. https://doi.org/10.1080/15323269.2021.1860448
Stone, C. (2022). From the margins to the mainstream: The online learning rethink and its implications for enhancing student equity. Australasian Journal of Educational Technology, 38(6), 139-149. https://doi.org/10.14742/ajet.8136
Stone, C., & O'Shea, S. (2019). Older, online and first: Recommendations for retention and success. Australasian Journal of Educational Technology, 35(1). https://doi.org/10.14742/ajet.3913
Tertiary Education Quality and Standards Agency. (2020). Foundations for good practice: The student experience of online learning in Australian higher education during the COVID-19 pandemic. https://www.teqsa.gov.au/sites/default/files/student-experience-of-online-learning-in-australian-he-during-covid-19.pdf?v=1606442611
Thomas, J., McCosker, A., Parkinson, S., Hegarty, K., Featherstone, D., Kennedy, J., Holcombe-James, I., Ormond-Parker, L., & Ganley, L. (2023). Measuring Australia's digital divide: Australian Digital Inclusion Index 2023. ARC Centre of Excellence for Automated Decision Making and Society, RMIT University, Swinburne University of Technology, & Telstra. https://www.digitalinclusionindex.org.au/wp-content/uploads/2023/07/ADII-2023-Summary_FINAL-Remediated.pdf
United Nations Educational Scientific and Cultural Organization. (2019). Recommendation on Open Educational Resources (OER). https://www.unesco.org/en/legal-affairs/recommendation-open-educational-resources-oer
Willis, L.-D. (2017). Implications for pedagogy: Flipping the classroom to engage pre-service teachers. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), The flipped classroom practice and practices in higher education (pp. 273-287). Springer. https://doi.org/10.1007/978-981-10-3413-8