Did a Facilitation Approach Pass the Test of Time? Insights from an “Ex-Post” Assessment in Cambodia

As the LEO project comes speeding towards the finish line, hold onto your hats! Case studies, literature reviews and frameworks galore are about to flood the knowledge market. LEO’s broad goals of increasing the capacity of donors and implementers to design, implement and evaluate market systems projects have led to some landmark publications. 

One of the research streams I’ve been involved with focuses on effective ways for scaling impact. Over the last three years, we’ve reviewed 50 projects to synthesize lessons on scaling smallholder farmers’ access to input markets and output markets, with a particular focus on direct versus facilitation-based implementation models and how that influences scale.  

The promise of market systems facilitation could be articulated as scale and sustainability with benefits to the poor. This is built on an expectation that new patterns of behavior will survive past the end of a project. Sadly, there is a troubling lack of post-project evaluations—“ex-post,” in the jargon of evaluators—to back this up. 

This brings us to a new report from LEO: Scaling Impact: the Cambodia MSME Ex-Post Assessment, which highlights insights from our assessment of the MSME project’s interventions in the swine sector and, in particular, input supply. Embracing a facilitation-based approach that avoided handouts, the project was known locally as the "banana and water" project, because that is all farmers received from the project. MSME sought to address low productivity among small-scale swine raisers by improving information flow and product quality in the input sector. The project worked with wholesalers to invest in embedded training on how to use vaccines, other medicines and feed. It anticipated that this would increase investment by farmers in their farms, so that they could reduce mortality, increase weight gain, sell more high quality pigs and ultimately improve their economic situation. At the time of the project, embedded training was mainly offered by just one wholesaler and on a small scale. 

The project ended in 2012, and we conducted this evaluation in 2015-2016. We interviewed people at three key levels—wholesalers, input suppliers and swine raisers—to try to understand the degree to which behavior change had been sustained.

The first challenge we faced was finding beneficiaries and partners. For example, we learned that many swine farmers had stopped engaging in agriculture—some because of Cambodia’s flourishing off-farm opportunities and others due to low prices at the time of the assessment (driven by cheaper pork imports from Vietnam and Thailand). This is an important insight for all of us working in agriculture. It is very easy to develop tunnel vision and to believe that the only pathway out of poverty for a farmer is on-farm growth. But this is simply not true, as LEO’s extensive research into labor-based pathways out of poverty in agrarian economies substantiates. We need to look at how projects contribute to the broad economic status of a household, not simply the value-chain specific benefits! 

For those businesses we did track down, some had trouble remembering what happened those many years ago, whether because of staff turnover, the time elapsed or the "light touch" approach of the project. Facilitation is great for sustainability (actors thinking, "This was our idea all along") but brutal for attribution ("You didn’t have anything to do with this"). This forced us to redouble our efforts to connect with a sufficient number of sources to conduct the assessment and triangulate various data sources to estimate attribution. 

So here are two of the key findings from the study: 

Finding #1: The embedded training model is alive and well and has been adapted and expanded by both existing and new wholesalers in the market.

We found that nearly every wholesaler in the sector was now offering embedded training, including new entrants since MSME’s end. Some factors that supported the model’s continuation are obvious ones: farmers are spending enough on swine inputs to make it worthwhile for wholesalers to keep investing because they see profitability. Other less apparent factors include the strong information flows driven and modeled by international firms who visit Cambodia multiple times per year to give their wholesalers seminars. Finally, the growth in the number of wholesalers keeps everyone on their toes to add value and differentiate.

One interesting example of adaptation is the rise in on-farm troubleshooting by numerous wholesalers, which is a much more resource-intensive way of providing customer service to farmers. This indicates that firms grasped the underlying principle, and were not simply ‘copying’ the activities. An important thing to note is that the tailored assistance is mostly being offered to larger farmers who are big enough customers to warrant the investment.

Finding #2: Many farmers have continued to access embedded training, though broader systemic struggles have constrained income growth.

When we interviewed farmers who had been counted as beneficiaries, we found that they were still accessing embedded training, although not as frequently as when the project was in operation. Those still in the business were continuing to invest in quality inputs, but their sales and profits were less than at the end of the project. This is clearly linked to the broader market dynamics, as cheap imported pork from neighbouring countries decreased the local price. We noticed that many farmers with smaller wine operations left the industry, while farms with bigger economies of scale were expanding. 

In terms of "end game" benefits to farmers, none of the farmers we interviewed were under the national poverty line. Also, we saw a pattern whereby "treatment" farmers who had benefitted from embedded training had higher income than "control" farmers who had not. We weren’t sure how much that difference could be attributed to the intervention as compared to a selection bias by MSME, which focused on the farmers who had the capacity and incentive to invest in inputs in the first place.

What can we all learn from this ex-post assessment?

In essence:

• The clear evidence of sustained and normalized behavior changes by swine wholesalers, years after the project closed, is encouraging news for facilitation projects.
• The mixed results regarding the extent to which these industry changes are "reaching down to the poorest," however, reminds facilitation advocates of the challenge before us—and evaluators—of the difficulty in asking the "right question" when it comes to understanding impact on the poor. (e.g., is it really a bad thing that many small-scale farmers are leaving the swine industry for other opportunities? Is the indirect impact on the poor any less powerful?)
• We should be doing more ex-post assessments! Given the grandeur of our rhetoric about the sustainability of using a market systems approach, it’s about time we backed it up with more evidence and used this learning to improve and evolve.