Multiple gaps exist in health care quality and outcomes for children, who receive <50% of recommended care. The American Board of Pediatrics has worked to develop an improvement network model for pediatric subspecialties as the optimal means to improve child health outcomes and to allow subspecialists to meet the performance in practice component of Maintenance of Certification requirements. By using successful subspecialty initiatives as exemplars, and features of the Institute for Healthcare Improvement’s Breakthrough Series model, currently 9 of 14 pediatric subspecialties have implemented collaborative network improvement efforts. Key components include a common aim to improve care; national multicenter prospective collaborative improvement efforts; reducing unnecessary variation by identifying, adopting, and testing best practices; use of shared, valid, high-quality real-time data; infrastructure support to apply improvement science; and public sharing of outcomes. As a key distinguisher from time-limited collaboratives, ongoing pediatric collaborative improvement networks begin with a plan to persist until aims are achieved and improvement is sustained. Additional evidence from within and external to health care has accrued to support the model since its proposal in 2002, including the Institute of Medicine’s vision for a Learning Healthcare System. Required network infrastructure systems and capabilities have been delineated and can be used to accelerate the spread of the model. Pediatric collaborative improvement networks can serve to close the quality gap, engage patients and caregivers in shared learning, and act as laboratories for accelerated translation of research into practice and new knowledge discovery, resulting in improved care and outcomes for children.
- quality improvement
- improvement networks
- pediatric care
- Rapid Learning Networks
- Learning Healthcare System
- ABP —
- American Board of Pediatrics
- BTS —
- Breakthrough Series
- CFF —
- Cystic Fibrosis Foundation
- CME —
- continuing medical education
- COG —
- Children’s Oncology Group
- IT —
- information technology
- LHS —
- Learning Healthcare System
- MOC —
- Maintenance of Certification
- NNECDSG —
- Northern New England Cardiovascular Disease Study Group
- QI —
- quality improvement
- QPSC —
- Quality in Pediatric Subspecialty Care
Multiple gaps and disparities in health care quality and outcomes for children exist.1,2 In fact, children receive <50% of recommended care.3,4 This stark fact reveals the chasm between evidence and practice and highlights continuing deficiencies in pediatric safety and quality. The Institute of Medicine has noted that it takes an average of 17 years for new knowledge generated by randomized controlled trials to be incorporated into practice,5 and even then application is highly uneven. In this article, we describe how pediatric collaborative improvement networks can serve to close the quality gap and accelerate the translation of evidence into practice, resulting in improved care and outcomes for children.
In the 1990s, efforts to improve health care quality often focused on changing provider behavior, particularly through various types of continuing medical education (CME). At the time, various adaptations to the typical lecture-based CME format were being evaluated. In a series of reviews, Davis et al6,7 summarized attributes of effective continuing education programs: the use of a needs assessment, longitudinal activities, and support with enabling tools.
In 1995, approximately the same time that Davis et al’s first review on CME was published in the Journal of the American Medical Association, the Institute for Healthcare Improvement developed and launched the Breakthrough Series (BTS), a collaborative model for achieving “breakthrough” improvement. On the basis of the improvement science principles of W. Edwards Deming, the BTS engages multidisciplinary teams from various clinical sites in a collaborative structure, uses data-driven feedback, encourages frequent small tests to assess implementation of changes, and provides ongoing coaching and support to assist teams to achieve results.8 The BTS model is longitudinal and time-limited, usually 12 to 18 months. The BTS focuses on a team and the practice system in which the team works rather than on the individual clinician, uses measurement to understand and reduce variation, and emphasizes a learning and collaborative environment among improvement teams consisting of both clinicians and nonclinical staff.
Much of what has been reported to date about the impact of collaborative improvement using the BTS model suggests that factors that facilitate successful team participation in quality improvement (QI) collaboratives include supportive senior leadership, reliable data and measurement, alignment of collaborative and institutional goals, and a focus on key outcomes and care processes.9–14
Developing a Framework for Pediatric Subspecialty Maintenance of Certification
In 2002, the American Board of Pediatrics (ABP) chartered Quality in Pediatric Subspecialty Care (QPSC), a workgroup of >45 representatives from the pediatric subspecialties, and engaged QI design experts. QPSC was asked to develop a framework for the performance in practice component of Maintenance of Certification (MOC) requirements for subspecialty practice with the goal of improved child health outcomes. Several assumptions emerged on the basis of an environmental scan and interviews with key subspecialty and QI leaders. For example, a multisite effort was believed to be especially important because the small number of pediatric patients in a subspecialty at any one center makes it difficult to learn what works best. Also, making the substantial system change to improve child health outcomes requires more than short-term efforts on the part of a group of self-selecting organizations or centers; persistent and intensive work as part of an ongoing sustained network is needed. Three successful subspecialty initiatives were considered as models.
The Children’s Oncology Group
The Children’s Oncology Group (COG) currently involves >200 children’s hospital and cancer centers and >8000 experts in childhood cancer.15 More than 90% of 13 500 children and adolescents diagnosed with cancer each year in the United States are cared for at a COG institution. COG uses a cooperative network for children with cancer, including systematic and uniform data collection that engenders a community standard of care. This cooperative group system for clinical research began in 1955 with a consortium focused on childhood cancer research. In 2000, 4 distinct research groups focused on pediatric cancer voluntarily merged efforts to create COG. COG’s activities to address pediatric leukemia include efforts at standardization of care and repetitive tests of change, activities that are characteristics of a collaborative improvement network. Joseph V. Simone, MD, the first director of COG, noted in a white paper for the Institute of Medicine:
“…the cure rate for Acute Lymphoid Leukemia rose from about 40% in the early l970s to about 70% in the mid-1990s without a single new frontline agent….improvements came largely from the trial-and-error adjustments of therapeutic doses and schedules made possible by the large pool of patients.…”16
The COG research has turned children’s cancer from a virtually incurable disease 50 years ago to one with a combined 5-year survival rate of 80% today.17
Northern New England Cardiovascular Disease Study Group
The Northern New England Cardiovascular Disease Study Group (NNECDSG) began in 1987 as a regional clinical database to allow clinicians to track outcomes after cardiac interventions and then transformed to a collaborative improvement effort. The purpose of the NNECDSG is to use its database and organizational structure to seek best practices and disseminate information aimed at improving results for patients undergoing cardiovascular interventions. This voluntary regional collaborative of clinicians, hospital administrators, and health care research personnel has tracked consecutive cardiovascular interventions performed throughout northern New England and reported its findings to the clinicians. Collaboration between NNECDSG institutions has led to progressive refinements in the clinical database, institutional site visits, efforts to understand and standardize ideal processes of care, risk-stratification tools to aid in decision-making, and most recently, tools to track and report on appropriateness of interventions on the basis of national criteria. Working together as a multiinstitutional, regional effort to improve surgical care for coronary artery bypass grafting surgery, the NNECDSG has reduced care variations between centers and has improved morbidity and mortality.18–20
The Cystic Fibrosis Foundation
The Cystic Fibrosis Foundation (CFF) provided an excellent example of how to use data from a national registry to spur improvement. Like the NNECDSG, the CFF’s initial focus was on the use of its comparative database. As a result of its transformation to a collaborative QI initiative in the early 2000s, the CFF posted individual center data on patient care and outcomes on its public Web site. This unique commitment to public data transparency facilitated discussion among sites about how to learn from one another to improve care. The CFF now supports a national care network of clinical centers that includes QI initiatives, basic and clinical research, and robust patient and family engagement.
The ABP, with support from the QPSC workgroup, developed and promoted a network model for pediatric subspecialties as the optimal means to improve pediatric health outcomes and to allow subspecialists to meet MOC requirements. The model’s key components for a pediatric network are as follows:
a common aim to improve care;
national multicenter prospective collaborative improvement efforts;
reducing unnecessary variation by identifying, adopting, and testing best practices;
shared, valid, high-quality real-time data;
infrastructure support to apply improvement science; and
public sharing of outcomes to identify best practices.
This ABP model has catalyzed collaborative improvement efforts in pediatrics. Three pediatric collaborative improvement programs have been supported, in part, with seed money from the ABP Foundation (ImproveCareNow, the Children’s Hospital Association Quality Transformation Network, and the American Academy of Pediatrics’ Chapter Quality Network). Currently, 9 of 14 pediatric subspecialties have implemented collaborative network improvement efforts.
Additional Support for the Value of the Network Model
The Importance of Time
Additional evidence from within and external to health care has accrued to support the network model proposed by the ABP in 2002. In particular, the need to support efforts over time to achieve improved results has been recognized. For example, a retrospective, 9-year study in 8 NICUs that participated in a QI collaborative to reduce lung injury in very low birth weight infants revealed that participating NICUs had significant and sustained improvement in clinical care practices and improved survival and nosocomial infection rates. However, a number of these improvements took longer than the 1- to 2-year duration of a typical collaborative.21 The experience of pediatric clinical research networks provides additional evidence that longevity is associated with productivity.22
A commentary published as a companion to reports of 3 collaborative QI efforts to reduce bloodstream infections in NICUs concluded that “reliably performing the best quality-of-care practices is hard work,” and noted that there are “no quick fixes.”23 Solberg24 observed that rather than single-topic, time-limited QI collaboratives, improvements and care transformation in an entire region are much more likely to occur via long-term efforts, led and supported by the leaders of individual member organizations, with a variety of ongoing communication and meeting efforts that go beyond a particular quality initiative.
Examples External to Health Care
Recent evidence from the nonmedical literature supports the value and potential impact of collaboration and sharing of information among independent sites. Benkler25,26 emphasizes that networks build on the natural instincts of individuals for collaboration. He suggests that if we design systems that foster cooperation and serve the collective good, this will naturally lead to large-scale collaborations with transformative impact (eg, Wikipedia, Linux). Burt and Ronchi27,28 have described the value of social networks in bridging “structural holes.” People focus on activities inside their own group, which creates holes in the information flow between groups, or more simply, structural holes. In pediatric health care, clinical centers within academic institutions may not be aware of how other centers are organized or of their unique care innovations. Connecting structures such as networks that successfully bridge these structural holes or “gaps” between groups will catalyze earlier access to a broader diversity of information and experience in translating information across groups. Thus, social networks have an advantage in detecting and developing good ideas. They are able to “see early, see more broadly, and translate information across groups.”29
Proposed National Health Care Models
The Institute of Medicine has described a vision for a Learning Healthcare System (LHS)30 that will transform systems of care and improve outcomes for patients. The LHS is a system “in which knowledge generation is so embedded into the core of the practice of medicine that it is a natural outgrowth and product of the healthcare delivery process and leads to continual improvement in care.”31 An LHS comprises clinicians, patients, and scientists who use data to learn from each clinical encounter to improve patient outcomes. More recently, one of the top recommendations from the Patient Centered Outcomes Research Institute Methodology Committee was using Rapid Learning Networks to improve outcomes and to advance knowledge for patients, clinicians, and researchers.32
How Pediatric Collaborative Improvement Networks Work
In part due to the ABP’s emphasis on the collaborative network model, a number of multisite, practice-based clinical networks have emerged over the past several years. Many of the features of a pediatric collaborative improvement network derive at least in part from the BTS model, as follows:
focus on a high-impact condition, health topic, or safety issue that can affect patient outcomes, engage caregivers, addresses a key population group, and is a widely recognized problem;
support from both clinical content and QI experts who provide evidence-informed ideas, guidance on appropriate methods, training on improvement science concepts/principles, and ongoing coaching and support;
use of the Model for Improvement33 focusing on setting clear aims, measurable targets, using data for feedback, and testing changes quickly on a small scale to learn by doing;
infrastructure support that includes monthly data collection, analysis, and reporting; project management; and QI coaching;
a series of defined collaborative activities (eg, learning session workshops, monthly webinars, a listserv or other shared communication platform, and shared tools); and
a critical mass of multidisciplinary teams from multiple sites involving front-line staff: physician engagement and nursing and administrative staff (and key others, as appropriate, eg, infection control, hospital quality leads, dietitians).
A key difference between pediatric learning collaboratives and pediatric collaborative improvement networks is the time frame and outcomes/results-oriented focus of networks. Networks begin with a plan to persist until aims are achieved and improvement is sustained. It is recognized that there are no short cuts to reliable change, and so an end date cannot be predetermined.
The “open ended” network model offers several advantages compared with a short-term collaborative. For example, all multisite improvement efforts need specific infrastructure elements that require a significant initial investment of resources (eg, database development, recruitment of participating teams, and human subjects approvals). For a network, this investment is leveraged as it provides a foundation for ongoing improvement and research involving the same sites and target patient population. In contrast to a short-term collaborative, networks typically need a more robust information technology (IT) system to deal with a large volume of data, from disparate sources, over a long period of time, to provide the capability to answer complex research questions. For these reasons, networks typically have a stronger level of sophistication in data management, analysis, and reporting, with resulting complex and resilient IT systems that can leverage this information for both improvement and research efforts.
As part of its work to support networks, the Learning Networks Core of the Anderson Center for Health Systems Excellence at the Cincinnati Children’s Hospital Medical Center has grouped the systems and capabilities required for collaborative improvement networks into 7 main areas (Fig 1)34:
Evidence-informed content: the involvement of content experts as network faculty and the use of evidence-informed content to ensure that clinical teams have the knowledge and skills to improve care and achieve results
Quality improvement: methodology and tools to support improvement projects executed within the network as well as the capabilities to teach improvement science concepts and principles (including site-specific coaching and training)
Project management: processes and capabilities to manage the resources of the network, plan and execute research and improvement projects, support longer-term planning and stakeholder management activities, and communicate effectively across the network
IT infrastructure: systems and tools to allow effective electronic collection, storage, analysis, and reporting of QI and research data within the network
Data management: systems and practices to ensure the data used to support improvement and research efforts within the network is valid and protected
Analytics: capabilities and tools to support analytical needs of quality improvement and research efforts within the network
Research: capabilities and tools to support the discovery of new knowledge related to improving the health care system for a targeted population
Networks that plan and develop these capabilities are well positioned to provide a strong foundation for improvement, engage patients and caregivers in shared learning, and serve as laboratories for translating research into practice and discovering new knowledge.
Additional Needs of Mature Networks
As a network matures, additional aspects evolve. After initial start-up, networks will need to develop an infrastructure for governance, decision-making, and strategic planning as well as specific workgroups (eg, data, research). Project management must have the depth to manage multiple, concurrent improvement and research projects as a robust database engages interest and spawns new questions. Keeping sites active and enthusiastic requires attention to team engagement, celebrating success, and the timely development and implementation of new topics. In addition, the ongoing nature of a network provides robust opportunities for long-term involvement and learning from the voices of patients and families; achieving full patient engagement requires specific network communications and support. Also, as networks demonstrate results and receive increased attention, the network is likely to attract new sites that are interested in joining. As a result, a disparity in QI and content knowledge may develop among teams. This means that to bring new teams “on board,” systems must be developed that ensure optimal educational processes for medical content and QI so that every team member has the knowledge and skills needed to improve care. In addition, data analyses must be modified to take into account new teams (eg, should results be evaluated per cohort, based on time in the network?). Finally, to ensure their longevity and success, networks must address issues of sustainability, particularly related to funding.
Start-up funding is required to develop and initiate a collaborative improvement network’s personnel and infrastructure. As the network grows and matures, additional funding is needed to support ongoing operational costs as well as innovation and research. Networks use a variety of funding mechanisms: participation fees, foundation grants, state and federal contracts, and awards. All present certain challenges. Grants are usually time-limited. The “pay to play” concept of participant fees may be a barrier for “low revenue” subspecialties (eg, rheumatology, adolescent medicine) or for hospitals or health systems that want to support participation in multiple improvement networks at the same time. Because monies spent on child health, even for children with complex chronic disease, are only a small percentage of total insurance costs, pediatric issues have not been a high priority for many insurers, and it can be difficult to focus their attention on the value of pediatric networks. This situation is a challenge, because the sustainability of networks is likely to increasingly depend on the ability to align with external forces (eg, payor incentives) and to document value in improved outcomes and patient experience at reduced costs.
The pediatric collaborative improvement network model developed and promoted by the ABP was designed to accelerate the translation of evidence into practice, resulting in improved care and outcomes for children, and to serve as a standard for the performance in practice component of MOC. Additional evidence from within and external to health care has accrued to support the model since it was proposed in 2002. ABP leadership, in collaboration with pediatric QI thought leaders, has helped to spur the development of multiple collaborative improvement efforts in children’s health care. As a result, the infrastructure systems and capabilities for networks have been delineated and can be used to accelerate the spread of this innovative and important model.
We gratefully acknowledge the contributions to the Learning Networks Core by Brooke Mullett MBA, Sandra Fuller MEd, Stacy Kramer, MPH, Jessica McAuliffe, the late Divvie Powell RN MS, and James Rile MBA, of the Anderson Center for Health Systems Excellence at Cincinnati Children's Hospital Medical Center.
- Accepted February 27, 2013.
- Address correspondence to Carole M. Lannon, MD, MPH, The James M. Anderson Center for Health Systems Excellence, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229. E-mail:
The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
FINANCIAL DISCLOSURE: The authors indicated they have no financial relationships relevant to this article to disclose.
FUNDING: The conference on which the articles in this supplement were based was funded by the American Board of Pediatrics Foundation, the National Association of Children’s Hospitals and Related Institutions (now the Children’s Hospital Association), The James M. Anderson Center for Health Systems Excellence at Cincinnati Children’s Hospital Medical Center, and the pediatric Center for Education and Research on Therapeutics, supported by cooperative agreement number U19HS021114 from the Agency for Healthcare Research and Quality.
- Committee on Quality Health Care in America
- Davis D,
- O’Brien MA,
- Freemantle N,
- Wolf FM,
- Mazmanian P,
- Taylor-Vaisey A
- ↵The Breakthrough Series. IHI’s Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series white paper. Boston, MA: Institute for Healthcare Improvement; 2003
- Øvretveit J,
- Bate P,
- Cleary P,
- et al
- ↵Children’s Oncology Group Web site. Available at: www.childrensoncologygroup.org/. Accessed on February 26, 2012
- ↵Simone JV, Lyons J. Superior cancer survival in children compared to adults: a superior system of cancer care? Available at: www.iom.edu/∼/media/Files/Activity%20Files/Disease/NCPF/Manuscript.pdf. Accessed August 17, 2012
- ↵Children’s Oncology Group. Impact of research. Available at: www.childrensoncologygroup.org/index.php/impact-of-cogs-research. Accessed on February 26, 2012
- Likosky DS,
- Nugent WC,
- Ross CS,
- Northern New England Cardiovascular Disease Study Group
- Payne NR,
- Finkelstein MJ,
- Liu M,
- Kaempf JW,
- Sharek PJ,
- Olsen S
- Slora EJ,
- Harris DL,
- Bocian AB,
- Wasserman RC
- Miller MR,
- Bundy DG
- Solberg LI
- Benkler Y
- Benkler Y
- ↵Olsen LA, Aisner D, McGinnis JM, eds; Institute of Medicine Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press; 2007
- ↵Clancy C. Session 1: Hints of a Different Way—Case Studies in Practice-Based Evidence, Opening Remarks. Presentation at the Institute of Medicine Roundtable on Evidence-Based Medicine Workshop: the Learning Health Care System. July 20–21, 2006; Washington, DC
- ↵Patient Centered Outcomes Research Institute. Methods meetings, July 3, 2012. Available at: www.pcori.org/assets/Charting-the-Course-Slides.pdf. Accessed February 26, 2012
- Langley GJ,
- Moen R,
- Nolan KM,
- Nolan TW,
- Norman CL,
- Provost LP
- ↵James M. Anderson Center for Health Systems Excellence; Cincinnati Children’s Hospital Medical Center. Learning network methods. Available at: www.cincinnatichildrens.org/service/j/anderson-center/learning-networks/method/. Accessed February 26, 2012
- Copyright © 2013 by the American Academy of Pediatrics