Seeking Indicators of System Health

Recently I co-authored a paper entitled Reconsidering the Concept of Scale in Market Systems Development, produced under the Leveraging Economic Opportunities project. The paper argues that there are important challenges in the way scale has been interpreted and in how that interpretation is applied in practice in market systems development projects.

In my previous post, I explained some of the unintended consequences of thinking of scale in terms of reporting high numbers for symptom indicators such as sales, uptake of specific practices/technologies, income, jobs, etc. Without context, such indicators provide very little understanding of the underlying causes of system performance, and little insight into whether a project has had any success at facilitating positive systemic change. Said in another way, from a systems perspective, these indicators are not very useful at indicating anything useful. 

But if these typical indicators are not appropriate for a systems approach, what indicators would be appropriate? This is a question I have been thinking about and discussing through various networks for some time now. What I think is evolving as an important area of research, and a much better way to frame activities that apply systems thinking, is defining patterns of behaviors or characteristics of more inclusive and healthy systems. The project I am currently running in Bangladesh, USAID’s Agricultural Value Chains project, is attempting to test some system health indicators.  Our initial assessment of types of indicators that could be more insightful identified the following:

• Churn rates of customers and suppliers: From a systems perspective, a more inclusive market system should push firms to establish longer-term and more trusted commercial relationships. Churn rates would measure the changes in repeat clients/suppliers as a percentage of all clients/suppliers. A change that indicates a growing percent of repeat or more durable relationships could indicate an improvement in business relationship patterns, especially when tracking a good representation of the firms in the market system.

• Investment flow patterns: Less inclusive markets tend to see firms fall into patterns of finance flowing out of businesses and into assets like buildings or to cover expenses of larger friends and family networks. Little is invested in clients, suppliers, information, human resources, etc. By tracking the investment patterns of numerous firms in a system, we hope to assess if any of these patterns change in favor of higher rates of investment in firm growth. 

• Circulation: Tracking the flow of goods and services from firm to firm and function to function reveal patterns over time related to how markets grow and expand, and how inclusive they are (i.e., having new actors, more repeat transactions, diversified structures, etc.). For example, it would be expected that as a market becomes more inclusive, internal incentives lead to the crowding in of new geographies and population segments, including ones that are typically excluded in more extractive markets such as the poor, vulnerable, and geographically remote. For example, Zambia’s PROFIT project effected change in the agricultural input system, driving suppliers to establish access points in more rural and poorer communities. 

• Social/cultural behavior patterns: This is an area where there are lots of options, but we are most interested in tracking three things: (i) disputes patterns, to assess changed perceptions of the fairness of transactions; (ii) trust levels in commercial relationships, to identify how commercial networks form, grow, and manage stress; and (iii) patterns of risk management, to ascertain whether there is a change towards being more proactive.  

It is still early days for this effort, and we have yet to determine whether the project is suitable for testing some of these indicators of system inclusiveness. But it is important that more efforts are initiated that rethink how projects should be evaluated in light of systems thinking.