Evadne Lewis | Hunter New England Local Health District

 https://training.cochrane.org/interactivelearning

Standard rate for a 1-year individual subscription is US$257 with a 45% discount for Cochrane members. A complete institutional price schedule is included in Appendix 2.

Module 1 is included FREE for all registered users.

Contact CILsubscriptions@cochrane.org for institutional licensing

Cochrane authors, full-time Research4Life authors from group A and B countries, and other specific Archie-based roles have free access to all Cochrane Interactive Learning modules.

https://training.cochrane.org/interactivelearning/purchase

Introduction

Systematic reviews are characterised by replicable methodology and presentation with the aim of informing clinical practice. Terms like ‘rigorous standards’, ‘unbiased results’, and ‘robust training’ (Ghezzi-Kopel, 2018)are constantly used. But how does a new researcher know which methodology to follow when their obvious first choice – to google “How to do a systematic review?” – turns up so many results. Hopefully they turn to their health librarian for direction. But as the desire to complete a systematic review continues to grow, even librarians are feeling the pressure (Demetres, Wright, & Derosa, 2020). What they both need is a tool with solid methodology created by a trusted source… 

Cochrane Interactive Learning (CIL) is a modular, self-directed training course that leads you through all steps required to conduct a complete systematic review following Cochrane methodology. It is aimed at both new review authors and experienced authors who want to update their knowledge. The course is based around the ‘Cochrane Handbook for Systematic Reviews of Interventions’, 2^nd ed. (Higgins et al., 2020).

Cochrane authors have free access to all CIL modules, while non-Cochrane authors can purchase the full course at the standard rate of US$257 for a 1-year individual subscription. Cochrane members receive a 45% discount on the standard rate, and module 1 is free for all registered users. A complete institutional price schedule is included in Appendix 2.

Although not specifically aimed at librarians, the course provides an overview that would be relevant to anyone collaborating on a systematic review.

Module Breakdown

Module 1 is freely available – only requiring registration for access. It introduces you to the systematic review as a research design, outlining the elements of a well-defined review question (PICO/S), and explaining the importance of using rigorous methods to conduct a systematic review.

Module 2 explains why a review protocol is crucial to planning and delivering a systematic review. It details the components of a protocol, and how to define eligibility criteria using the PICO format.

Module 3 teaches you how to plan and structure a search using a rigorous and systematic approach, choose sources and run searches to identify published and unpublished studies, how to download results and report the process. This is a really useful overview for anyone working on a systematic review - whether they be researchers or librarians – and probably the module I found most interesting and relevant.

Module 4 describes how to select studies to include in a review - explains the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram, how using the diagram ensures rigor and transparency in documentation, and introduces software tools that can assist to extract and collect data.

Module 5 highlights the importance of determining the quality of included studies – detailing the varied sources of bias that can exist, and identifying ways of incorporating ‘Risk of bias’ assessment into the analysis – with examples of useful tools.

Module 6 shows you how to recognise characteristics of different types of data; identify non-standard study designs; manage heterogeneity between included studies; and undertake a meta-analysis. It is very long and detailed – and would be extremely useful to researchers who needed to complete statistical analysis as part of their review – but the level of detail feels excessive for a librarian participating in a systematic review. It would act as a valuable reference for future questions.

Module 7 demonstrates how to interpret the results of your statistical analysis – explaining confidence intervals, standardized mean difference, reporting bias, and how to use the GRADE approach to determine the certainty of evidence.

Module 8 presents the elements of effective review reporting: how to discuss evidence, draw conclusions and write up a review – with particular emphasis on creating ‘Summary of findings’ tables, appropriate vocabulary for ‘Plain Language Summaries’ (PLS), and how best to present your findings so it can inform healthcare or policy decisions.

Module 9 introduces health economics, identifying factors of resource use relevant to decision making so you can understand the role of economic evidence in a Cochrane review.

Module 10 discusses network meta-analysis (NMA) in the context of a systematic review of randomized trials – the assumptions required, how to plan an analytical strategy, and understanding the results.

Module 11 presents health equity considerations when planning and conducting a systematic review – how to include equity in your systematic review and consider equity in knowledge translation.

Module Pros and Cons

I found working through the modules more helpful than simply reading a book on the topic. The information is broken into chunks – each module and section building on the previous one, providing background and context to your understanding of what is involved with a systematic review. 

Each section is on a specific topic. As you finish a topic it is marked completed with a filled in square – which is a useful place marker, indicating your progress. Unfortunately every time you finish a topic you are sent to the beginning of the module page and have to navigate back to the last finished place marker. This becomes annoying when you try to complete more than one topic in a session – surely a more user-friendly alternative would send you to the next topic.

There are downloadable resources relevant to each module (examples of data collection forms, cheat sheets, and checklists), as well as links to standards, relevant chapters of the Cochrane handbook, websites with further information, additional training modules, videos and tips that add vital content. Under each module there is an indication of the time it will take to complete. 

Along the way you complete quizzes that help you engage with the content, and test your understanding. You can stop and restart at any point – and mark pages as ‘favourites’ for later referral. 

There is a search filter, so you can search the entire course by favourites, complete/incomplete pages, and topic tags. This is helpful for returning to specific sections of the course when you have questions.

Each module ends with self-assessment. You get 2 attempts at each question and need to get 100% to ‘pass’. After successful completion of a module, you can download a personalised certificate. I found some of the questions confusing, and the provided feedback response ‘partly correct’ wasn’t always helpful – especially when the answers are multiple choice. Although you don’t need to ‘pass’ to continue completing modules, after a few wrong attempts – you end up frustrated, second-guessing which aspect you misunderstood. Provision of a more comprehensive response, clarifying concepts, after several failed attempts – rather than just sending you back to a section to review your understanding – would be really helpful.

There are diagrams and tables throughout each module containing additional explanation points that open when you click on a pin. This allows you to access a lot of detail within the one small space – but it would be more useful if the pop-up explanation didn’t open in front of the diagram, covering the thing it was explaining – especially on the more complex statistical topics.

I found several linking errors – either links didn’t work or went to archival content – and the PRISMA link doesn’t go to the latest version. This is probably due to the fact that the whole course was last updated in December 2019. Several modules state the “next planned review date is Quarter 4, 2020” which has obviously passed. This is disappointing as the course is advertised as a “continually updated resource”.

According to the ‘Help’ page (Cochrane Interactive Learning), the modules have been designed to comply with Web Accessibility Initiative (WAI) guidelines to Level AA, and all video/audio content is accompanied by written transcripts.

At least two other introductory systematic review courses exist, for example “Introduction to Systematic Review and Meta-Analysis” (Coursera) or “Comprehensive Systematic Review Training Program” (JBIGlobal) - and many libraries offer their own online training courses. The trusted Cochrane brand and option for a yearly subscription are points in favour of CIL, but not every researcher – or every journal they wish to publish in – will require the same high standards.

Thoughts

I found the modules helpful and well written, and have developed some confidence in my understanding of relevant terminology, and the purpose behind each stage of the systematic review process – which are good building blocks and will be helpful long-term. But as a library technician, with limited experience of the systematic review process, I don’t think I could now go away and feel comfortable assisting a researcher to tackle one – even after successfully completing all the assessments. I would need to continually dip in and out of the modules as questions surfaced throughout the review – which would require continued access to the resource.

Which brings me to questions of resource cost and who is the audience? Would we buy it for library staff training – so we can improve our skills to assist/collaborate with staff researchers, or as a stand-alone resource for the education of staff researchers? Each individual user requires an account – all tagging, and recording of outcomes is tied to an individual subscription – so it would not be possible to easily share the resource. Individual subscriptions would need to be purchased for each interested researcher or library staff member, and at US$257 for a one year individual subscription – I’m not sure it would be within our budget or that of many Australian health libraries.

I acknowledge that the level of support provided for the review process varies between libraries and organisations, but felt that the promotion of librarian involvement throughout the systematic review process could have been stronger. Module 3 highlights the importance of working with an ‘Information Specialist’ in the ‘search’ aspect of the review process, but the inclusion of a librarian in the review team is not mentioned anywhere else. This is especially disappointing as the course aims to educate review authors (both new and experienced) on best practice. As noted by Ghezzi-Kopel in her excellent 2018 review of CIL (Ghezzi-Kopel, 2018), this is a missed opportunity to promote the inclusion of a librarian on a larger scale. Her suggestion that “a separate section on working with a librarian would be a helpful addition” is still relevant even though it hasn’t been taken up yet (latest update December 2019).

Summary

Cochrane Interactive Learning is an engaging, well-designed course, founded on strong methodology – but I question whether it is actually accessible to beginners (“new authors”). It is not an introduction in any sense - it is a full course, and extremely content heavy. It is also organised according to Cochrane standards – which may be too much methodology for the non-Cochrane researcher. 

But there are not too many quality training options available, so while the level of detail might at first seem overwhelming to the beginner, the ability to pop in and out of the course throughout your subscription period – allowing you to steer your learning in your desired direction – and repeat any aspect, really appeals.

If you are already involved in writing reviews – or providing assistance to researchers who are – and want to hone your knowledge and skills, it would be a great tool to have on hand. The course provides clear direction and examples to develop an in-depth understanding of the process of conducting a systematic review – and would leave you feeling confident in the quality of your review.

References

Cochrane Interactive Learning. Help. Retrieved from https://training.cochrane.org/interactivelearning/help

Coursera. Introduction to systematic review and meta-analysis. Retrieved from https://www.coursera.org/learn/systematic-review

Demetres, M. R., Wright, D. N., & Derosa, A. P. (2020). Burnout among medical and health sciences information professionals who support systematic reviews: an exploratory study. Journal of the Medical Library Association, 108(1), 89-97. doi:10.5195/jmla.2020.665

Ghezzi-Kopel, K. (2018). Cochrane Interactive Learning. Journal of the Medical Library Association, 106(4), 577-579. doi:10.5195/jmla.2018.516

Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2020). Cochrane handbook for systematic reviews of interventions (Second edition. ed.). Hoboken, NJ :: Wiley-Blackwell.

JBIGlobal. Comprehensive Systematic Review Training Program. Retrieved from https://jbi.global/education/systematic-review-training

Appendix 1 | Error examples

In Module 6 you are referred to ‘Chapter 9’ in the resources. “For more details on the formula used to calculate the SMD, see Chapter 9 of the Cochrane Handbook in Resources.” But there is nothing labelled ‘Chapter 9’ – I think they’re referring to ‘Calculations with continuous outcome data’.

Module 8 (Plain Language summaries) – link doesn’t work: https://methods.cochrane.org/sites/default/files/public/uploads/PLEACS.pdf

A link in Module 9 takes you to an archived version of a clinical event pathway. This page has a forwarding link – but it doesn’t take you to a ‘clinical event pathway’:

http://handbook-5-1.cochrane.org/index.htm#chapter_15/figure_15_2_a_clinical_event_pathways.htm - 

Module 10 (Resources link) PRISMA extension for NMA reporting guideline – doesn’t work.

Appendix 2 | Institutional Price Schedule