Erica Hateley | Northern Health

This paper begins with PICO and its place in health librarianship, before moving to discuss a model of teaching knowledge that may offer some new ways of thinking about PICO (or, perhaps, leaving PICO behind). My two main points of contention with the ubiquity of PICO in health librarianship are: 

  1. That it is only one among many approaches to question formulation; and 
  1. That question formulation strategies are treated ‘as’ search formulation strategies. 

I am particularly interested in PICO’s ubiquity as a teaching tool for advanced information-seeking within health libraries, and the possible division between what health librarians are teaching and what they do when moving from question formulation to designing search strategies.  

 

The Rise of PICO

The PICO question formulation tool is a real success story of the evidence-based practice (EBP) movement. First described in 1995 as part of a short editorial advocating for better–that is, more answerable–clinical questions, the authors pose and answer a question of their own: 

What makes a clinical question well built? First, the question should be directly relevant to the problem at hand. Next, the question should be phrased to facilitate searching for a precise answer. To achieve these aims, the question must be focused and well articulated for all 4 parts of its “anatomy”: 1) the patient or problem being addressed; 2) the intervention or exposure being considered; 3) the comparison intervention or exposure, when relevant; 4) the clinical outcomes of interest. (Richardson et al., 1995, p. A-12) 

 

Since 1995 PICO has become almost synonymous with EBP. Any clinician or health information professional who has had even a basic introduction to EBP is likely to have been taught PICO as a foundation.  

 

I will stipulate that PICO is extremely useful when used for its original purpose: the formulation of answerable clinical questions, especially when those questions address a specific intervention or therapy. One does not need to look far before finding areas where PICO is less useful: “the PICO framework is best suited for representing therapy questions, and considerably less-suited for diagnosis, etiology, and prognosis questions” (Huang et al., 2006, p. 363). It is not suited for many allied health, quality improvement, qualitative etc. questions, which is okay because it was not designed for such contexts. This is why there are multiple question formulation frameworks that have been developed in the past few decades; and, clinical librarians know that PICO is just one of many such possibilities (see the supplement to Booth et al. 2019 for a list of 38 such alternatives). For anyone new to question formulation in the health sciences PICO can be helpful: it is easy to learn, easy to remember, and easy to use. 

 

Presumably that accounts for any number of university and/or hospital library resources (e.g. LibGuides) on EBP placing PICO front-and-centre. Similarly, any number of books, articles, and websites devoted to instruction in EBM suggest PICO – good old familiar, “universal”, unquestioned PICO – as an essential component. It makes a kind of sense that an EBP clinician or health information professional called upon to find evidence, or deliver training in finding evidence, might default to using and/or teaching PICO.  While I hope to complicate that, I am even more interested in challenging any default use of PICO for developing search strategies. 

 

Making the move from asking to finding 

The fact that Richardson et al. (1995) mentioned “facilitate searching” as part of their rationale may account for its persistence in not just question formulation discourse but literature searching discourse today. Indeed, there is a slippage or blurring legible when one looks for PICO in the realm of search.  It is well and truly in the literature by 2011 (Arguelles, 2011; Davies, 2011), and by 2017 one paper asserts that “The PICO Framework should also be used to develop the search terms that are informed by the PICO question” (Considine et al., 2017, p. 79). However, even when PICO is the appropriate framework for question formulation of the issue at hand, health librarians leave it behind for searching because we know it can be inappropriate or just plain not work. 

 

Top-tier guidance affirms that we should set PICO aside as we move from asking to searching. The NHMRC’s guidance on producing evidence-based clinical guidelines suggests leaving “the outcome field blank ensures that studies presenting all relevant outcomes are identified” (National Health and Medical Research Council, 2019, online). The Cochrane Handbook for Systematic Reviews of Interventions (Lefebvre et al., 2022) advises against including outcomes in systematic review search strategies, even when PICO has been useful in developing the review question: 

The structure of search strategies in bibliographic databases should be informed by the main concepts of the review, using appropriate elements from PICO and study design. It is usually unnecessary, however, and may even be undesirable, to search on every aspect of the review’s clinical question. Although a research question may specify particular comparators or outcomes, these concepts may not be well described in the title or abstract of an article and are often not well indexed with controlled vocabulary terms. (Lefebvre et al., 2022, sec. 4-4-2) 

 

Intra-health-libs discourse generally does not necessarily advocate or even imply PICO as a search-formulation tool. When clinical librarians are talking to each other PICO does not come up (Burns, 2015), or is treated as one of several options (Eldredge (2008), Ballantyne Scott et al. (2022)). That may be because we know that when it comes to searching, PICO can be distinctly unhelpful. 

 

In 2000, Booth et al. described trying “EBM-structured forms” (basically PICO forms) for literature search requests, and the librarians ended up rating “minimally structured forms more highly than EBM-structured forms” (p. 239) because they generally offered more context for the information need. The lack of helpfulness runs the gamut of enquiry types that health librarians might be answering. From direct clinical queries, where it seems that “PICO queries do not result in better recall or precision in time-limited searches” (Hoogendam et al., 2012, p. 121), to the more complex searches required for systematic reviews: “when using PICO as a search strategy tool, searching on all of the PICO elements will result in a lower recall [...] of especially the outcome element” (Frandsen et al., 2020, p. 73). It should be noted that a follow-up study noted that excluding outcomes can lead “to an enormous increase in the number of retrieved records” (Frandsen et al., 2022, p. 85), but that this may be connected with the prevalence of single-stranded search strategies (p. 87). 

 

Teaching the move from asking to finding 

To the extent that PICO is a useful and influential framework, health information professionals need to know it, understand it, and apply it. There is a strong chance we will also need to know how to teach it. My two concerns are that we may be teaching PICO in such a way as to imply that it is the only question formulation tool, and that we may be (intentionally or not) teaching that PICO is a tool for search strategy formulation. 

 

There are PICO filters and benchmarks in emerging AI tools for developing search strategies, and major databases and platforms incorporate PICO search-building tools (e.g. https://pubmedhh.nlm.nih.gov/pico/index.php ) Literature refers to such examples and describes teaching techniques such as assessing students for “PICO conformity” (Philbrick et al., 2019), making PICO fun  with a “PICO pal” (Waszak et al., 2022) or the “PICO game” (Milner and Cosme, 2017). Anecdotal evidence is not rigorous, but I have myself attended training sessions where “PICO” was used as a synonym for “question”, even for topics where a PICO-style question would be, at best, limiting. 

 

Resources and examples such as these are not exclusively for health librarians, but as part of their own evidence-based practice librarians will find such resources and may well be persuaded by the ubiquity of PICO discourse. It is worth remembering that the EBP model includes not only available evidence but also the practitioner’s knowledge and skills.  

 

Teacherly knowledge and practice 

Many health librarians have both experience and expertise in question formulation, search strategy development, and the relationship between the two activities. Many health librarians deploy these skills as part of their regular professional practice, and also have to teach these skills to other health professionals. In order to reflect upon these aspects of health librarianship I am drawing on a distinction between expert knowledge and the teaching of expert knowledge. Emerging in the literature of teacher education in the 1980s, work by Lee Shulman and others influentially defined and described content knowledge and pedagogical content knowledge. I offer it here as a useful lens for understanding both the interdependence and differences between librarians’ practice and teaching. 

 

Content knowledge addresses the need for deep and detailed knowledge of what is being practiced: “The teacher need not only understand that something is so; the teacher must further understand why it is so, on what grounds its warrant can be asserted, and under what circumstances our belief in its justification can be weakened and even denied” (Shulman, 1986, p. 9). In the case of PICO, health librarians must understand what it is, what it is for, and when, where, and how it may or may not be useful. Such knowledge will strengthen their practice as they make informed decisions about how best to formulate answerable questions. Ideally, this knowledge will be sufficiently thorough to enable explaining it to other people. 

 

The delivery of such knowledge via training, teaching, or education requires the development of pedagogical content knowledge: “the most regularly taught topics in one’s subject area, the most useful forms of representation of those ideas, the most powerful analogies, illustrations, examples, explanations, and demonstrations—in a word, the ways of representing and formulating the subject that make it comprehensible to others” (Shulman, 1986, p. 9). Again, in the case of PICO, this would mean health librarians develop a number of ways to represent what it is, adjusting or amending those representations or explanations to meet the specific contexts, capacities, and experiences of those they are teaching. 

 

Shulman’s terms can help us distinguish between health librarian teaching and practice in order to advocate for librarians to take themselves seriously as expert educators and practitioners when it comes to formulating questions and search strategies. When I think about health librarians who use their knowledge and experience to use PICO for question formulation when it is appropriate, and then leave the PICO framework behind for designing search strategies to locate literature that may help answer their question, I think of them as experts. I think of them as evidence-based practitioners. I think of them as professionals who do, can, or will translate their content knowledge into pedagogical content knowledge. 

 

Educators of health professionals, unite! 

I invite health information professionals to reflect upon their own use (or non-use) of PICO in their own practice, especially in terms of content knowledge and pedagogical content knowledge. If they are uneasy with the PICO status quo, they may take comfort in an emerging critique of the current state of affairs, and feel reassured that it is thus far coming from health educators and librarians! 

 

Recent work by Jonathan D. Eldredge (who has written about and advocated for evidence-based health librarianship for many years) has critiqued PICO as a teaching tool. Writing about training medical students and physician assistant students in question formulation, he and his co-authors note that a “growing body of evidence points to the limitations to PICO despite its common use” (Eldredge et al., 2021, p. 69); and, further that their “study supports mounting evidence that we need to reconsider the utility of PICO” (Eldredge and Nogar, 2022, p. 49). 

 

The most trenchant recent critiques have been made by nurse educators: 

In our experience, PICO often seems to represent or signify EBP in students’ minds; that is, the rote activity of formulating a structured clinical question has come to engulf the much more complex and rich process of critical decision-making in care delivery that EBP is meant to be. As a pedagogical tool, PICO is not only ineffective, but it has mistakenly come to reify EBP, making it impractical and even detrimental to the teaching of EBP curricula. (Schiavenato and Chu, 2021, p. 2) 

 

It is possible that many health librarians may share these concerns and frustrations, perhaps even feeling that “PICO is overused, outdated, and flawed” (Cullen et al., 2023, p. 517). If so, this begs the question of what to do when it comes to PICO? 

 

Teaching nuance is an act of kindness 

I suggest starting with one’s content knowledge and pedagogical content knowledge – where, when, and how are we using and teaching PICO? Why are we using and teaching it in those particular ways? Schiavenato and Chu explicitly argue for moving away from PICO-by-default in our teaching: “an obvious start is with an educational de-emphasis on structured question formation. That is, teaching what PICO is, and what it is not; its narrow application, as well as its considerable limitations” (2021, p. 2). This can very much be seen as a call to reconsider how we translate our content knowledge into pedagogical content knowledge! 

 

It is also important to attend to the work of librarians who offer kind, nuanced, and empathetic approaches to teaching question formulation. Kloda and Bartlett (2013) offer a particularly useful reminder that many of our colleagues and/or learners may be stressed or anxious by the time they meet with us. In this light, taking a nuanced approach to PICO is an act of kindness: 

PICO, and other structures, might best be thought of as guides or recommendations rather than strict formulations to be followed. Information needs should not be thought of in terms of restricted structures, as articulating and communicating them can already be difficult and linked to feelings of anxiety for the individual asking the question. (Kloda and Bartlett, 2013, p. 59) 

 

Taking the time to learn about PICO’s history and meaning can empower health librarians to explain and contextualise it for their colleagues, library users, and learners. Such work to enhance one’s expertise alongside one’s experience can also empower health librarians not to be rigidly prescriptive: to use or suggest PICO when relevant, but be confident and competent to suggest when it is not relevant. Throwing out PICO in no way means throwing out structure or rigour. 


What I am advocating for 

I invite and encourage my fellow health librarians to reflect on their PICO mindset. I believe and argue that we need to: 

 

References 

Arguelles, C. (2011). Evidence-Based Practice Mentors: Taking Information Literacy to the Units in a Teaching Hospital. Journal of Hospital Librarianship, 11(1), 8-22. https://doi.org/10.1080/15323269.2011.537989   

Ballantyne Scott, B. (2022). Developing a code of practice for literature searching in health sciences: a project description. Journal of the Canadian Health Libraries Association / Journal de l'Association des bibliothèques de la santé du Canada, 43(1). https://doi.org/10.29173/jchla29409 

Booth, A., Noyes, J., Flemming, K., Moore, G., Tunçalp, Ö., & Shakibazadeh, E. (2019). Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ global health, 4(Suppl 1), e001107. https://doi.org/10.1136/bmjgh-2018-001107 

Booth, A., O'Rourke, A. J., & Ford, N. J. (2000). Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bulletin of the Medical Library Association, 88(3), 239-247.  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC35232/  

Burns, E. C. (2015). Being a Solo Librarian in Healthcare: Pivoting for 21st Century Healthcare Information Delivery. Chandos.  

Considine, J., Shaban, R. Z., Fry, M., & Curtis, K. (2017). Evidence based emergency nursing: Designing a research question and searching the literature. International emergency nursing, 32, 78-82. https://doi.org/10.1016/j.ienj.2017.02.001    

Cullen, L., Hanrahan, K., Tucker, S., Edmonds, S. W., & Laures, E. (2023). The Problem with the PICO Question: Shiny Object Syndrome and the PURPOSE Statement Solution. Journal of PeriAnesthesia Nursing, 38(3), 516-518. https://doi.org/10.1016/j.jopan.2023.01.024 

Davies, K. S. (2011). Formulating the Evidence Based Practice Question: A Review of the Frameworks. Evidence Based Library & Information Practice, 6(2), 75-80. https://doi.org/10.18438/B8WS5N   

Eldredge, J. D. (2008). Evidence-Based Practice. In M. S. Wood (Ed.), Introduction to Health Sciences Librarianship (pp. 241-270). Routledge.  

Eldredge, J., & Nogar, C. (2022). Physician Assistant Student Training in Question Formulation: A Quasi-Experiment. The journal of physician assistant education 33(1), 47-50. https://doi.org/10.1097/JPA.0000000000000409    

Eldredge, J., Schiff, M. A., Langsjoen, J. O., & Jerabek, R. N. (2021). Question formulation skills training using a novel rubric with first-year medical students. Journal of the Medical Library Association, 109(1), 68-74. https://doi.org/10.5195/jmla.2021.935     

Frandsen, T. F., Nielsen, M. F. B., & Eriksen, M. B. (2022). Avoiding searching for outcomes called for additional search strategies: a study of Cochrane review searches. Journal of clinical epidemiology, 149, 83-88. https://doi.org/10.1016/j.jclinepi.2022.05.015 

Frandsen, T. F., Nielsen, M. F. B., Lindhardt, C. L., & Eriksen, M. B. (2020). Using the full PICO model as a search tool for systematic reviews resulted in lower recall for some PICO elements. Journal of clinical epidemiology, 127, 69-75. https://doi.org/10.1016/j.jclinepi.2020.07.005 

Hoogendam, A., de Vries Robbé, P. F., & Overbeke, A. J. P. M. (2012). Comparing patient characteristics, type of intervention, control, and outcome (PICO) queries with unguided searching: a randomized controlled crossover trial. Journal of the Medical Library Association, 100(2), 121-126. https://doi.org/10.3163/1536-5050.100.2.010 

Huang, X., Lin, J., & Demner-Fushman, D. (2006). Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annual Symposium Proceedings, 359-363. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839740/  

Kloda, L. A., & Bartlett, J. C. (2013). Formulating Answerable Questions: Question Negotiation in Evidence-based Practice. Journal of the Canadian Health Libraries Association (JCHLA), 34(2), 55-60. https://doi.org/10.5596/c13-019 

Lefebvre, C., Glanville, J., Briscoe, S., Featherstone, R., Littlewood, A., Marshall, C., Metzendorf, M.-I., Noel-Storr, A., Paynter, R., Rader, T., Thomas, J., & Wieland, L. (2022). Chapter 4: Searching for and selecting studies. In J. Higgins, J. Thomas, J. Chandler, M. Cumpston, G. Li Bassi, M. Page, & V. Welch (Eds.), Cochrane Handbook for Systematic Reviews of Interventions (ver. 6.3). Cochrane. https://training.cochrane.org/handbook/current/chapter-04#section-4-4-2   

Milner, K. A., & Cosme, S. (2017). The PICO Game: An Innovative Strategy for Teaching Step 1 in Evidence-Based Practice. Worldviews on evidence-based nursing, 14(6), 514-516. https://doi.org/10.1111/wvn.12255 

National Health and Medical Research Council. (2019, 6 September, 2019). Guidelines for Guidelines: Identifying the evidencehttps://www.nhmrc.gov.au/guidelinesforguidelines/develop/identifying-evidence  

Philbrick, A. M., Hager, K. D., Lounsbery, J. L., Moon, J. Y., Pereira, C., Undeberg, M. R., Westberg, S. M., & Reidt, S. (2019). Educational Prescriptions to Document Evidence-Based Medicine Questions in Ambulatory Care Advanced Pharmacy Practice Experiences. American journal of pharmaceutical education, 83(8), 7299. https://doi.org/10.5688/ajpe7299 

Richardson, W. S., Wilson, M. C., Nishikawa, J., & Hayward, R. S. A. (1995). The well-built clinical question: a key to evidence-based decisions. ACP Journal Club, 123(3), A12-A13. https://doi.org/10.7326/ACPJC-1995-123-3-A12    

Schiavenato, M., & Chu, F. (2021). PICO: What it is and what it is not. Nurse education in practice, 56, 103194. https://doi.org/10.1016/j.nepr.2021.103194    

Shulman, L. S. (1986). Those Who Understand: Knowledge Growth in Teaching. Educational Researcher, 15(2), 4-14.  https://doi.org/10.3102/0013189X015002004  

Waszak, D. L., Stec, M., Martini, K., & Mihal, C. C. (2022). An Innovative Approach to Teaching Evidence-Based Practice: Use of PICO Pal. Nurse educator, 47(6), 332-335. https://doi.org/10.1097/NNE.0000000000001226