Improving Monitoring, Evaluation, and Learning Approaches to Market Systems

This post was written by John Spilsbury and is the fourth post in the Feed the Future Enabling Environment for Food Security project's market systems blog series. To read earlier installments, see the first, second, and third posts. 

As USAID programs evolve to adopt a market systems approach, implementing partners must adapt their monitoring and evaluation approaches to better capture dynamics of complex systems change. Standard Feed the Future USAID results monitoring emphasizes measuring performance. On its own, measuring performance is of limited use for learning and adaptive management when applying market systems approaches. Two USAID Feed the Future programs — one in Cambodia and one in Mozambique — are exploring different approaches to better capture data, learn, and adapt their programs through a systems approach.

From 2017 to 2018, the Feed the Future Harvest II in Cambodia and the Feed the Future Mozambique Agricultural Innovations Activity (INOVA) increased their use of the market systems development approach. In doing so, they reviewed their monitoring, evaluation, and learning (MEL) processes to better understand systemic change in their target markets.

Originally designed as a traditional value chain project, the Harvest II activity has adhered more closely to its original value chain focus (fruit and vegetables), while INOVA has adopted a more holistic approach that allows it to invest opportunistically in market systems. Harvest II uses a results framework based on a predictable causal logic that links to its indicators, which are measured regularly. INOVA’s MEL framework is not based on a causal pathway, as it sees the market systems where it invests as complex and inherently unpredictable. Instead, it intends to collect data in a way that probes market systems to learn about business opportunities and what works, invest in them, and then monitor what changes. In this way, the framework offers the opportunity to learn about causality as it moves forward.

Harvest II learning has identified strength in its quantitative indicators. Review has called for wider data collection and more qualitative information to provide a better understanding of systemic change. The activity has identified data gaps in areas such as product competitiveness and understanding relationships between buyers and market actors, such as banks, microfinance institutions, and packaging companies. To address the need for more information, Harvest II has explored: (1) use of a buyer-focused participatory outcome mapping tool and (2) increasing indicators arranged according to the Adopt-Adapt-Expand-Respond framework to strengthen its market systems monitoring and to assess change.

INOVA uses a probe-sense-respond MEL approach. By the end of 2018, INOVA had reduced its performance indicators and increased use of a wide range of innovative, adaptable, tailored indicators and tools as part of its MEL. This included the use of custom indicators called probes. Probes are part of INOVA’s adaptive business partnerships that quickly gather evidence to inform rapid management decision-making in fast-changing environments. Examples of probes include supplier referral rates, customer satisfaction levels, intentions to adopt, and levels of crowding in/copying by other businesses. Activity logs, observation diaries, stakeholder feedback forms, and partner-customer centricity scorecards serve as data collection tools. By combining probes with other indicators, INOVA intends to develop the desired wider, more complete picture of systemic change in the markets in which it intervenes.

Both Harvest II and INOVA are increasingly using market systems thinking in their programming. Accompanying MEL system adjustment is taking place, and experience is growing. It is anticipated that learning will significantly increase with the use of these evolving and innovative approaches.