Spotlight On: Monitoring

Abt Associates offers its clients around the world deep expertise in monitoring. It’s a key component of our rigorous approach to performance management. It’s how we achieve the best and most equitable results, time and again. Whether we’re creating a theory of change, measuring program performance or designing a monitoring, evaluation and learning (MEL) system for complex data management, we ensure that high quality data is generated to answer important questions around progress toward intended results. Abt’s approach to monitoring enables decision makers to reflect, learn and adapt over time.

Methods

Our MEL experts employ a wide range of qualitative and quantitative methods to collect and manage actionable information on performance monitoring, context monitoring and complexity awareness monitoring. We proactively monitor equity in outcomes to identify and resolve gaps or unintended consequences. Using a range of approaches—including, but not limited to, routine monitoring methods, rapid feedback studies, client and community surveys, outcome harvesting and data analytics—we design monitoring efforts so our clients can learn and apply adaptive management. We collect, interpret and use data related to program implementation, reach, relevance, effectiveness, efficiency, equity and sustainability, all of which enables us to enhance our impact.

Relevant Expertise 
 

Investing in Women Initiative 
Client: Department of Foreign Affairs and Trade (DFAT), Australia 

The Investing in Women Initiative is an Australian Government flagship program that seeks to ensure women across Southeast Asia are empowered to make their own economic decisions. As the project manager, Abt has developed a rigorous monitoring, evaluation and learning (MEL) system for adaptive management. Our approach is designed to:

• Assess whether the project’s results are meeting expectations
• Improve project performance by providing contextual insights about the opportunities for—and barriers to—women’s economic empowerment
• Build the evidence base to gain private sector and government acceptance of the importance of women’s economic empowerment.

Our current MEL framework tracks a range of complex and transformative change processes, spanning social awareness and attitudes, policy reforms, formal and informal institutional change within private sector organizations, market responses to the program’s work and improvements in women’s access to resources. We use a variety of methods in addition to routine monitoring tools, such as comparative organizational case studies, rapid feedback studies, media monitoring and secondary context analysis. Abt also uses process tracing and qualitative comparative analyses to determine what factors contribute to higher-level outcomes.

PMI Evolve  
Client: US Agency for International Development (USAID)  

The U.S. President's Malaria Initiative Evolve Project supports PMI with the planning, implementing, and monitoring of malaria vector control programs. These include entomological monitoring, indoor residual spraying, insecticide-treated mosquito nets, and larval source management.

As a data driven project, Abt implements rigorous monitoring, evaluation and learning methods. Our measurement approaches inform management and operational decisions and ensure equitable coverage of critical malaria interventions. We designed and oversee a global MEL system to manage and analyze vector control and entomological data across more than 25 countries.

Our MEL approaches ensure that actionable data is available to reliably measure coverage of life-saving vector control interventions, inform malaria control decisions, and optimize impact.

Implementing the End-Stage Renal Disease (ESRD) Treatment Choices Model   
Client: Centers for Medicare and Medicaid Services (CMS)  

The Centers for Medicare & Medicaid Services (CMS) developed the ESRD Treatment Choices (ETC) Model to encourage increased use of home dialysis and kidney transplants for Medicare ESRD beneficiarieswhile reducing Medicare expenditures and improving quality of care. The ETC Model also aims to reduce health and socioeconomic disparities among Medicare ESRD beneficiaries.

In support of leading partner American Institutes for Research (AIR), Abt provides technical and operational support to CMS for the ETC Model. Abt developments rigorous and transparent computational and benchmarking methodologies. creates and maintains quality measures, and performs validation testing. Abt also collects and tracks public comments to support rulemaking.

Home Health Quality Measurement, Value-Based Purchasing Model, and Hospice Assessment Instruments Development and Maintenance 
Client: Centers for Medicare & Medicaid Services (CMS) 

Abt supports the Hospice Quality Reporting Program (QRP), the Home Health QRP, and the expanded Home Health Value-based Purchasing (HHVBP) Model. This is part of Abt’s portfolio of work supporting the CMS payment systems. The main goals of this work are:

• The development, maintenance, implementation, and re-evaluation of outcome, structural, process, and composite measures in these programs which are reflective of quality care provided, and
• The development, maintenance, and implementation of the data collection vehicles and their corresponding data elements, e.g., the Outcome and Assessment Information Set (OASIS) and the Hospice Item Set (HIS) or other such assessment/data collection instruments.

Nepal Portfolio Monitoring, Evaluation and Learning (PMEL) Program 
Client: U.K. Foreign, Commonwealth and Development Office 

The UK Foreign, Commonwealth and Development Office (FCDO) has a portfolio of programs in Nepal aligned with five strategic goals: Open Societies, Stability and Security, Inclusive Growth, Climate and Resilience, and Support to Women and Girls. The portfolio of the relies on a data-driven management approach to optimize effectiveness.

The Abt-led Portfolio Monitoring, Evaluation and Learning program (PMEL) supports FCDO in Kathmandu, making evidence-based, timely, and strategic management decisions by gathering evidence and lessons from across portfolios and providing big-picture views. PMEL supports efforts to improve coherence and make judgments about program synergies, transferability, and cumulative effects. PMEL also strengthens information use through shared indicators for portfolio-level tracking.

PMEL’s work across the portfolios continues to grow, and includes:

• Testing the alignment of the portfolio and its various approaches against BEK's vision to support Nepal's development,
• Convening stakeholders through various learning spaces, conversations, and workshops to distill valuable insights from the portfolio
• Maintaining FCDO’s internal Data Visualization System (DVS), which includes disaggregated information at various administrative levels and incorporates national data across key sectors.

In addition, Abt’s analysis of technical assistance through our Systems Change Diagnostic tool sparked debate about the ways projects are delivered. The tool assessed the spread and intent of technical assistance across the Embassy’s portfolio and determined whether it creates system change with the Government of Nepal.