Show Me the “Real” Data

Does the existing data vindicate donors’ reliance on small and medium enterprises (SMEs) as effective vehicles of economic growth? Yes. Is it reliable and valid? Probably not. During May’s Microlinks seminar, Caio Piza and Joao Montalvao, both World Bank economists, called for better impact evaluations to generate quality data and advised caution while interpreting the current figures.

Kicking off the proceedings, Caio emphasized the importance of the topic: public and multilateral organizations spend billions of dollars every year on SMEs in Low and Medium Income Countries (LMICs). The assumption behind these direct and indirect interventions is that SMEs, if cultivated, can generate significant economic impact for a given country. To test the validity of this hypothesis, Caio and his team looked at a sample of 40 quantitative studies (reduced from around 9,475) from the past 12 years in 18 different countries. Some of the highlights are given below:

• Matching grants were the most favored intervention (43%)
• Average firm had 58 employees (with outliers on both sides)
• Chile and Brazil featured the most (with six studies each)

Firm performance, labor productivity, and employment generation all showed a positive relationship with donor-funded SME activities. However, the tricky part is that these studies span a heterogeneous sample taken from different countries based on different assumptions. After controlling for possible biases, some of the results no longer retain their statistical power. Add the tendency to overstate findings in order to get published (read: publication bias), cautioned Caio, the results become even more questionable. 

Joao, on the other hand, conducted an analysis of a favored means of supporting SMEs – matching grants. His presentation discussed one specific study in Mozambique as well as the resultant policy implications. This topic is important because almost $2 billion have been spent on matching grant schemes in the last 20 years: around 40% of the World Bank-funded private sector development projects have included one. Plus, there are other lessons to be learned about the execution of programs and understanding of SMEs as a tool of growth. 

Joao’s team looked at the dataset from the MESE (Subsidy Program for Small Businesses) in Mozambique. They gathered the characteristics and profiles of about 300 participants and 700 non-participants (firms and owners). The results establish that awareness of matching grant scheme is one of the strongest determinants of participation. This means that younger, socially-connected, and highly-skilled owners are more likely to get support from the program. This also potentially means, according to Joao, that the program ends up targeting gazelles or high-performers by screening out lesser connected firms which may not be doing that well. The proclivity for “cream-skimming,” Joao highlighted, generates the following question: If the gazelles are already performing well, should the support be directed towards other firms?

On the whole, he echoed Caio’s sentiments regarding the need for better evaluation components in all projects. This data would help in identifying the real causal impact of the programs and help make better policy decisions regarding these projects in the future, Joao explained.