2017 ICT4D Conference

This workbook represents the collective learning from five years of engagement with development agencies working to introduce digital health projects, develop robust evaluations, and scale-up activities nationally and regionally. Among the many challenges identified on the mHealth landscape – those of program monitoring and impact evaluation have remained a continued area of confusion. Digital health projects are often very dynamic, evolving through several stages of maturity during which questions being asked of the project are also changing rapidly. Projects typically begin with exploring basic technical functionality and feasibility to user satisfaction, then moving towards trying to define the effectiveness, attributable impact and, ultimately, “value for money” of the intervention. This progression requires a combination of methods, from qualitative to quantitative, often mixing both, as appropriate, to answer the questions being asked. This workbook attempts to guide the reader through a journey which begins at the earliest stage of defining the basic technical requirements and early implementation testing through to the evaluation and reporting of program impact. The workbook makes a distinction between activities which aim to monitor program activities – that is, assure fidelity, quality and coverage of the intervention’s delivery in a population – and activities which seek to evaluate program activities – that is, attribute some output or outcome to the intervention. Although these streams are often closely intertwined in implementation, conceptually it may be simpler to disentangle them in the planning stage. This allows program managers to focus separately on establishing systems which measure how consistently a program is meeting its goals, understanding that this feeds into a broader agenda of understanding the impact of the program. This toolkit should be useful for members of the mHealth / digital health community engaged in planning or deploying a new mHealth or digital health program, as well as program managers, implementers and policy makers.