We are currently updating our website, and will have our new version online soon. Please check back later this fall.

Mailing List

Subscribe to the KT Canada mailing list




Once you have signed up, you will receive a confirmation email with your username and password. To activate your account, follow the instructions in the email.



Theme 4: Sustaining KT: Practical tools and services to support KT and KT research

4.1 Critical Analysis of Sustainability of KT Interventions

Straus, McKibbon, Khandwala, Quan, Stelfox, White, Grimshaw
48-60 months
Researchers, funders, decision makers
MRC Phase:
Community Partners:
Cochrane Collaboration


As KT interventions are found to be effective in studies, the need to evaluate their sustainability is becoming increasingly important. Many implementation studies focus on short-term outcomes relevant to a study situation and do not address long term outcomes in practice and policy domains. For example, in the systematic review of interventions to optimize antibiotic use, of 39 studies included, only 2 reported follow-up beyond 1 year.[1] This situation requires a framework analogous to post-marketing surveillance which is done by the pharmaceutical and medical device industries. Once KT interventions are implemented, their impact on outcomes needs to be monitored, and the intervention itself should be monitored to determine if its 'dose' or 'formulation' need to be adapted in the changing health care environment. Factors that may influence sustainability include those related to the stakeholders, the social context of care, the organizational context, and public policy[2] We have been unable to identify any systematic reviews that d address sustainability of KT interventions and propose to complete a scoping review to understand the factors that influence sustainability in health care decision making and to develop a framework for assessing sustainability.


  1. To conduct a systematic search of the literature to identify the impact on health care outcomes beyond 1 year of KT interventions targeting end-users including patients, clinicians (of all disciplines), public health officials (including medical officers of health, department chairs and program managers), health services managers, and policy makers (at regional, provincial and national levels);
  2. To complete a systematic search of the literature and key informant interviews to identify factors (such as those affecting stakeholders, context of care, organization and policy) that influence sustainability of effective KT interventions targeting end-users including patients, clinicians, public health officials, health services managers and policy makers;
  3. To complete a systematic search of the literature to identify how sustained change from KT interventions should be measured; and,
  4. To develop a framework for assessing sustainability of KT interventions.


Phase 1: Objective 1

Scoping review Search Strategy:

This scoping review, which will identify gaps in the literature and areas for systematic review, has been designed using methodology proposed by Arksey and O'Malley and others.[3] We will conduct a systematic search of the health, health management and social sciences literature, and search the internet, and grey literature to identify studies (including both quantitative and qualitative) that describe the impact of KT interventions beyond 1 year including those targeting patients/public, clinicians (from all disciplines), public health officials (including medical officers of health, department chairs and program managers), health care managers, and policy makers (including regional, provincial and federal). The search strategy will be developed in consultation with an information specialist.

Study Selection:

All study designs including qualitative and quantitative methodologies will be used where there is a description of the impact of a KT intervention after more than 1 year of implementation.

Methods of the Review:

Two investigators will independently review the retrieved abstracts to identify those that meet the inclusion criteria. Assessment of the full text articles and data extraction will be done by two investigators independently. The articles will be categorized and summarized quantitatively (using a simple numerical count) and qualitatively (drawing on the descriptive analytical method) to identify where gaps exist and where there is a need for a systematic review of the literature or for primary research.

Phase 2: Objective 2

The search generated in Phase 1 will be used to identify articles that explicitly address what factors influence sustainability of KT interventions. This Phase will be augmented by a series of semi-structured interviews with key informants from relevant industries (e.g. pharma and medical devices) and social sciences who have experience in implementation. They will be identified through the literature search and in consultation with the team of investigators. Snowball sampling will be used to identify additional participants. Transcripts will be analysed, drawing on grounded theory.

Phase 3: Objective 3

We will complete a systematic search of the literature to determine how sustainability of KT interventions should be measured. This will include impact on health care outcomes as well as the impact on the intervention. The search results from Phase 1 will be used for this Phase but articles which explicitly address strategies for measuring sustainability of KT interventions will be targeted for retrieval.

Phase 4: Objective 4

The results of the above scoping review will be used to develop a framework for assessing sustainability using a deliberative (Delphi) process amongst stakeholders. The framework will be developed after reviewing the materials from Phases 1 to 3 which will be assessed qualitatively using a thematic analysis.


We believe the results of this research will be useful to researchers, funders and those interested in implementing KT interventions including clinicians, health care managers, and policy makers amongst others. To our knowledge, there has been no previous attempt to understand the impact of sustained KT interventions or the factors that influence sustainability. The resulting framework for assessing sustainability will advance both the science and practice of KT.


  1. Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. In: The Cochrane Library, Issue 3, 2007. Oxford: John Wiley & Sons.
  2. Grol R, Grimshaw J. Evidence-based implementation of evidence-based medicine. Jt Comm J Qual Improv. 1999 Oct;25(10):503-13.
  3. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Social Res Method 2005;8:19-31.

4.2 Knowledge Tools for Knowledge Translation

Grimshaw, Straus, O'Connor, Haynes, Lavis, Sales, Zwarenstein, Légaré, Frémont
0-60 months
KT researchers, KT practitioners, patients, policy makers & funders
MRC Phase:

We will develop a series of Knowledge Tools for KT targeting the needs of different stakeholder groups. The resulting Knowledge Tools will be fed directly into the KT Research Clearinghouse funded by CIHR; these tools will enhance the usability of this clearinghouse for all stakeholder groups.

Knowledge tools for patients- decision aids (O'Connor)

Patient Decision Aids (PtDAs) explain options, clarify personal values, and guide patients in deliberation with their practitioners to support 'grey zone' decisions.137 O'Connor has been influential in developing methods, systematic reviews, standards, and training and has developed international standards for PtDAs;33 and a global inventory of evaluated PtDAs using the International standards.138;139 Since 2004, annual access to the Ottawa website is about 1 million hits, 200,000 page views, and 10,000 downloads. The website is currently supported by a CIHR group grant that expires in 2009; there is no further funding identified at present for this key global resource. With KT Canada funding we will maintain the PtDA website beyond 2009 and expand our toolkits. We will:

  1. design an online auto-tutorial on how to develop and evaluate PtDAs using a simpler template that meets international standards;
  2. design a template for 'patient preference reports' after patients use PtDAs which can be used in patients' electronic records;
  3. develop training modules for clinicians to interpret the preference reports and close the loop on decision making with patients; and
  4. re-design our ongoing systematic review database so that researchers can conduct reviews on sub-sets of data or meta-regressions to identify the most active ingredients of PtDAs.

Knowledge tools for health care professionals (Haynes)

The McMaster Premium LiteratUre Service (McMaster PLUS)32 continuously evaluates new studies from over 130 clinical journals, using explicit criteria for study quality, and a panel of over 4000 physicians assesses the relevance and interest. PLUS has been evaluated in a CIHR-funded cluster randomized trial in Northern Ontario and increases the use of evidence-based resources by family physicians, internists and subspecialists of internal medicine140. This service is currently available via the commercialized services, BMJUpdates and Medscape Best Evidence Alerts to over 60,000 physicians in Canada and globally. With KT Canada funding we plan to develop and evaluate derivative and new services that will provide evidence-based information to key professional groups:

  1. KT PLUS - an alerting service focusing on original studies of quality improvement and continuing education interventions for quality improvement professionals, continuing education providers, professional organizations and licensing bodies;
  2. REHAB Plus - a new service for rehabilitation professionals;
  3. MDUG PLUS - aimed at medical undergraduates;
  4. PCP Plus - adding PLUS alerts to P-PROMPT, an electronic prompting system for primary care preventive services deployed in 80 practices in Ontario; and
  5. NURSING PLUS - Funds will be used to establish prototypical services for each professional group and to undertake mixed method evaluations of the impact of these services.

Following testing locally and across the KT Canada network, we will seek sponsorship to support on-going production.

Knowledge tools for managers and policy makers (Lavis)

We currently have CIHR funding to make systematic reviews more accessible to managers and policymakers (to Dec 2009). We seek KT Canada funding from 2010 onwards to develop, maintain and evaluate a web-based tool kit for policymakers that will include: a searchable database of quality-appraised, relevance-assessed systematic reviews that address the governance, financial and delivery arrangements within which health programs, services and interventions are provided; checklists that health system managers and policymakers can use to inform discussions about the quality and local applicability of systematic reviews and the equity and scaling up considerations raised by the reviews; and a resource listing for health system managers and policymakers who want to find policy relevant primary research and systematic reviews on high priority managerial and policy issues.

Knowledge tools for knowledge translation professionals (Grimshaw)

The Cochrane EPOC group has undertaken a series of overviews of systematic reviews of professional behaviour change strategies that have been widely cited.11;141;142 We have undertaken preliminary searches to update the overview and have identified over 150 reviews covering professional behaviour change strategies. We have undertaken an overview of 50 systematic reviews relevant to prescribing behaviour for the COMPUS program of CADTH.143 The significant increase in the number of systematic reviews over the last decade highlights the challenges that KT professionals face in keeping abreast with knowledge syntheses of KT interventions and the problems of intermittent updates of the overview of reviews. With KT Canada funding we will develop an ongoing process for identifying and summarizing new systematic reviews of professional behaviour change strategies and make a database of appraised systematic reviews available for KT professionals.

Knowledge tools for KT researchers: core measures (Sales)

To advance the science of KT and encourage standardization, we need to develop a core set of measures to assess factors at the individual, organization, and policy levels that impact on KT. With KT Canada funding, we plan to: assess and collect available tools designed to measure key aspects that have an impact on KT; develop an online database of identified tools and provide information about key aspects (developers, details of reliability/validity as available, and contact information); and maintain this database through the life of the KT Canada grant. This will be informed by projects 2.1 and 2.2.

Knowledge tools for KT researchers: evaluation methods (Zwarenstein)

The Trial Protocol Tool (TPT) is an electronic guide and template for protocol writing for pragmatic trials that aims to increase the number of pragmatic trials of complex interventions.101 This web-based tool is free-of-charge for non profit use and has formed the basis of over 20 randomized trial training courses worldwide. KT Canada will revise the TPT for use in KT trials. We will: identify exemplary protocols of trials of KT interventions; develop algorithms for using the MRC framework; create a database of intracluster correlation coefficients, an essential information item for sample size estimation in cluster trials. The KT-TPT will be used as part of the training stream for KT Canada.

Knowledge tools for francophones (Légaré/Frémont)

To ensure adequate bidirectional and bilingual communication about KT, we will develop a French language platform based in the "département de médecine familiale" (DMF) of the Faculté de médecine at the Université Laval. The DMF has been a global leader in the French speaking world for evidence-based medicine and has considerable experience in bilingual translation of KT reports and development of teaching programs and on-line modules to promote KT. The French platform will make available: 1) existing materials for the proposed KT initiatives and 2) integrate newly developed materials.


Suite of web based tools to support the needs of different stakeholders, 12-18 peer reviewed publications.