Healthcare is facing increasing quality and resource challenges and telemedicine is often regarded as part of the solution. Numerous pilots are underway, but will these telemedicine services really make healthcare more efficient? Will they increase access, quality, safety of care? And what added value can satellite communication bring to such services? These are questions the ASSIST SATCOM-APPS study aims to address by developing comprehensive methods and easy to use toolkits for telemedicine assessment, which help pilot projects to objectively and rigorously evaluate their solution from an evidence-based multi-stakeholder perspective.
ASSIST is funded through the ESA’s General Studies Programme (GSP).
The ASSIST documentation and the associated toolkits can be downloaded form the ASSIST web site.
The holistic goal of this SATCOM-APPS study is the development and validation of a comprehensive methodology to evaluate outcomes and impacts generated by ESA-sponsored projects related to the provision of sustainable telemedicine services. The ultimate goal is to provide clear, relevant and impartial information to be used for evidence-based decision making concerning the migration of pilot services into real, operational undertakings.
The evaluation of health information services has to deal with a multitude of heterogeneous variables, such as actors (the people), artefacts (the technology), and the environment in which they are implemented as well as with their interactions and most importantly, their outputs and outcomes. Telemedicine has medical, technical, psycho-social, organisational, business and societal aspects. Assessing telemedicine means to incorporate all these aspects. Research shows that there is a huge bandwidth of evaluation methods, but many - if not most - of them are not very suitable to deal with a comprehensive, holistic evaluation of a telemedicine service. It should be emphasised that a telemedicine system is only part of this solution, whereas the telemedicine service also contains the whole delivery chain and all stakeholders. In addition, it is different from project evaluation which is measuresing the achievement of project objectives, whereas ASSIST is focusing on the impact achieved or achievable.
Selection criteria and critical factors to be considered in this context are the extent to which methods allow for:
- Benefits and costs estimation
- Provisions for testing rigour
- Dealing with objectivity issues and bias in measurements and estimations
- Sustainability analysis: ex-ante and formative assessment, business modelling, affordability considerations, risk assessment
- Multiple stakeholders to be considered
- Health system and operational environment variables to be taken into account
- Time dimensions (from planning to routine services)
- Validity, reliability and comparability of measures and options
- Conciseness of model
- Applicability to telemedicine / space technology
None of the methods available as of today appears to be readily suitable for telemedicine assessment. All lack some elements, sub-methods or techniques required for the purpose at hand. The starting point for the tools developed in the ASSIST project is the EHR Impact cost-benefit analysis approach and method, which provides the most advanced, and differentiated tools and techniques for data gathering and analysis. ASSIST will further develop and extend it with additional techniques to fit the project goals. ASSIST tools will allow a telemedicine project to self-evaluate their project impacts in a multi-dimensional way. It will support the evaluator in finding the data needed and analysing them. A small number of outcome measures allow to easily understanding the relation of benefits and cost. Pie charts allow assessing the distribution of cost and benefits over different stakeholder groups, identifying benefits shifts and the distribution between financial and non-financial benefits. Project leads will receive projections of results which allow them to sketch out scenarios of future use.
The project team will always take the perspective of stakeholders and decision makers, and look at the added value they may receive.
The study plan includes the following parts:
- A critical review of the existing, consolidated and comprehensive methodologies established so far for evaluating telemedicine projects.
- The elaboration of a set of adequate methods that can be adopted to carry out a systematic evaluation of the results of such projects within a multi-stakeholder perspective.
- The development of an evaluation model for the results of such projects, based on the selected assessment framework and its methods, and composed of a set of guidelines, procedures and an associated supporting toolset.
- The application of the evaluation model to a number of space-based telemedicine projects for validating the model, with a subsequent critical revision of the model.
- The delivery, presentation and promotion of the final evaluation model, and dissemination of the associated results.
ASSIST has made substantial progress during the last twelve months: The knowledge gained from an in-depth review of 16 assessment frameworks from the areas of telemedicine, eHealth, general public sector investments and others has been consolidated in the ASSIST telemedicine assessment methodology. This assessment methodology is the founding stone of a self-assessment tool-kit that guides users through the assessment process and models their telemedicine application’s impact. Decision makers will be enabled to make evidenced based decisions for their future business. The ASSIST tool is currently being applied to four different types of telemedicine projects to prove it fits its purpose. Based on the results of this validation the consortium plans to fine tune the ASSIST tool before it is finally released.