Prove and Improve: Adapting Monitoring, Evaluation, and Learning (MEL) to Better Support Business Growth and Project Learning in Market System Development Projects

This post was authored by Luca Crudeli, Meghan Bolden, Matt Ripley, and Jenesil Benito.

How can market systems development (MSD) projects assess whether business innovations are improving market system inclusiveness for smallholder farmers (SHF)?

Gathering performance data—on customer satisfaction, supplier performance, and commercial outcomes—is a key task for any successful business. However, in under-developed markets, companies often lack the necessary financial and management skills to do this, limiting their ability to engage deeply with customers and suppliers.

Projects have traditionally overcome the challenge by incorporating data-gathering into partnership agreements. The justification is that projects need data to report impact; therefore, market players either need to provide them data, or if not, projects will collect it on their behalf. Such a course of action, however, inhibits relationships that are facilitative in nature.

Are there other strategic approaches to data gathering within the context of a market systems approach? What leverage points, offers, and tools can be identified and developed to use monitoring and evaluation (M&E) as a lever for system change?

The USAID-funded Feed the Future Mozambique Agricultural Innovations Activity (FTF Inova) aims to stimulate a more inclusive agricultural market for SHFs. To address these questions, FTF Inova has developed and introduced a new approach. It is based on the fundamental belief that MEL is not meant just to focus on project performance, but also to help businesses successfully innovate, expand their customer/supplier base of SHFs, and effectively involve women and youth. This way, MEL can become an intervention in itself—supporting evidence-based decision-making that nudges partners towards a pathway of self-improvement and developing data-driven services in the market system. The shifts in behaviors towards evidence-based decision-making reinforces and amplifies positive systemic changes.

FTF Inova monitoring is designed to reflect the dynamic and complex environment in which it operates. In "thin" markets devoid of key support functions in Mozambique, systemic change is neither linear nor predictable in advance. FTF Inova’s MEL innovation is therefore inspired by USAID Guidance on Complexity-Aware Monitoring and The Lean Start Up, which provides a framework for introducing market innovations into situations of extreme uncertainty.

FTF Inova uses a probe-measure-learn cycle that emphasizes probing and learning based on real-world action, which allows for adaptive partnerships through fast-cycle testing using lean and rapid methods to gather evidence and use it for decision-making. FTF Inova developed a number of tools to help the project team and market players navigate the probe-measure-learn cycle. “Stakeholder Feedback Surveys” and the “Partner Customer-Centricity Scorecard” shift data gathering from a transactional, top-down mindset to a transformative, bottom-up one that aligns the M&E function with a facilitation approach[1].

Stakeholder feedback surveys aim to help FTF Inova and its partners listen directly to customers, suppliers, and employees not as passive "end-beneficiaries," but as active economic and social agents. By so doing, FTF Inova aims to build the capacity of companies to be more responsive to smallholder wants and needs. Three types of feedback surveys are run:

1. The “Value Proposition” survey, which investigates the reasons for purchasing, the perceived value for money, and product quality perceptions;
2. the “Customer Satisfaction” survey, which assesses how likely customers are to recommend a product/service, suggestions for improvement, and customer experiences and satisfaction; and
3. The “Profile Check,” which discovers who is being reached using simple proxies such as the Poverty Probability Index[2].

While stakeholder feedback gathers data, customer-centricity describes the processes and people skills that the private sector must have in place to use data effectively. That is, it describes how embedded a customer focus is within company decision-making structures. This is essential for companies to develop a culture of market segmentation and responsiveness to SHF needs. As such, the partner customer-centricity scorecard aims to help partners self-assess their own decision-making process and help FTF Inova decide when to build the capacity of partners to gather evidence and use it for decision-making.

The tool itself is a checklist comprising two components:

1. Reported fact—completed by company management to reflect on and discuss which customer-centric attributes are appropriate and priority to develop; and
2. Reported perception—anonymous responses provided by employees that show how well customer-centricity is being deployed in practice.

FTF Inova is testing these tools on a live intervention and expects positive results, such as better-informed decisions, greater stakeholder satisfaction, and two-way communication—all improving the chances of an intervention’s success. FTF Inova’s Chief of Party, Luca Crudeli, along with MEL experts from Acumen, EcoVentures International, and MarketShare Associates, will delve deeper into this topic at the 2018 SEEP Annual Conference held October 1-3 in Arlington, VA.

[1] Stakeholder Feedback Surveys are inspired by the Lean Data Field Guide, while the Customer Centricity Scorecard is adapted from and based on the Millward Brown Vermeer PulseCheck tool.

[2] A poverty Score Card for Mozambique is available. It was developed by Swiss Agency for Development and Cooperation (SDC) under the InovAgro Project. See http://www.simplepovertyscorecard.com/MOZ_2014_ENG.pdf (Pg. 2-3).