Cost-effectiveness of USAID Interventions
By: Nathan Martinez from the Economics Team in the Center for Economics and Market Development. This is the fifth in a series of blogs around the six principles of USAID’s Economic Growth Policy.
My father-in-law is the sole proprietor of a business selling decorative ceiling products. Think tin ceilings in a classy bar. Margins are thin. Each investment decision centers on its cost-effectiveness - how much will it cost to achieve a desired impact? He is currently considering if the incremental sales from a new online sales platform justify the costs. Households use similar techniques when making investments in education, transport, and property, among others.
Public investments are no different. Governments have limited resources to address complex challenges. This means they have to weigh the tradeoffs of investing in some activities but not others. USAID recognizes these tradeoffs in the 4th Principle of the Economic Growth Policy, arguing the Agency should demonstrate the value of its programs relative to the cost. How does USAID measure cost-effectiveness? How widespread is its use?
Tools for you to consider: Cost-Effectiveness Options
Several cost-effectiveness techniques can guide decisions on how best to use scarce resources. We’ll start with some well-known techniques. Cost-benefit analysis (CBA) is the comparison between the costs and benefits of an activity over a certain time horizon. If the expected gains “with the intervention” are greater than the expected gains “without the intervention” (i.e. counterfactual), then the activity is feasible. When it is difficult to monetize a benefit, such as improved care for the sick and elderly, cost-effectiveness analysis (CEA) can be used to identify the least-cost way of achieving results by comparing the costs of alternative interventions with identical benefits (e.g., immunization).
The Inter-American Development Bank, World Bank, Millennium Challenge Corporation, and other organizations apply CBA and CEA into their designs and evaluations. Prior to the early 1980s, CBA and CEA were widely used at USAID, but the practice declined in the 1980s. In 2010, the precursor office to USAID’s Center for Economics and Market Development (EMD) started recruiting senior CBA experts, training staff, and promoting CBA/CEA use. Since 2010, USAID has helped build capacity to conduct CBAs, training over 850 USG officials, implementing partners, and host country government officials who are now conducting or managing CBAs and CEAs in partner countries. For example, a Synthesis Report of CBAs conducted in 11 Feed the Future (FTF) countries demonstrated how FTF projects created a ratio of benefits to project costs of approximately 2.01, implying that our partner countries received on average $2.00 (USD) in economic benefit for each $1.00 invested. However, the Agency’s nearly 100 publicly available CBAs over this period is small compared to the thousands of USAID projects and activities.
Cash benchmarking is a cost-effectiveness alternative that compares the impact per dollar of a development activity to the impact per dollar of a household grant. The grant serves as a “benchmark” - if money given to households is expected to produce a greater impact than the activity, then the activity should not be pursued. This approach is based on evidence showing that household grants can have significant positive impacts on many development outcomes. USAID has partnered with UC Berkeley's Center for Effective Global Action to conduct five research projects focussed on cash benchmarking. Findings will help to develop new methods for comparing development programming to a benchmark of cost-equivalent digital cash transfers. Furthermore, there have been benchmarking studies of USAID WASH, and employment and workforce readiness activities in Rwanda with differing results (some interventions were over the cash benchmarking threshold and some were not).
Another cost-effectiveness tool mentioned in the EG Policy is an impact evaluation, which measures the change in a development outcome that is attributable to an intervention. USAID requires the use of impact evaluations to determine if a new, untested approach can be catalytic. To measure the change, impact evaluations assess the differences between the treatment group (those receiving the intervention) and comparison (those not receiving the intervention). From fiscal year (FY) 2012 to FY 2019, there were 133 USAID-funded reports with impact evaluation in the title, yet only 74 USAID-funded evaluations met USAID’s criteria for an impact evaluation. To increase the use of cost-effectiveness in these studies, USAID now requires that all impact evaluations include a cost analysis.
Where to go from here?
The number of cost-effectiveness studies pales in comparison to the number of USAID activities. Why is this? One impediment is that it is often difficult to isolate the programmatic costs and benefits for each project. For example, a project may have 15 interventions used in various combinations, targeting many types of beneficiaries in different regions. Another issue is tied to the real and perceived administrative burden of performing the analysis. In addition to designing and implementing projects, there are annual reporting exercises, mandatory analyses and sector-specific analyses. Since cost-effectiveness analysis is not required, there are limited incentives to conduct them. Another issue is the actual and perceived cost of performing these studies. For small projects, some may feel that the costs are unwarranted.
As previously stated, governments have a responsibility to make the best use of limited resources - this should be the main incentive for USAID's adoption of cost-effectiveness - and the following actions can be taken to incorporate cost-effectiveness measures into project design and implementation. First, cost-effectiveness analysis should be required where possible in USAID acquisition and assistance agreements so the costs of achieving desired outputs and outcomes can be captured. USAID’s Center for Education has produced excellent resources with Cost Reporting Guidance (2018) and Cost Analysis Guidance (2020). Second, analysts should use this cost-effectiveness data to conduct CBAs and other related studies throughout the program lifecycle to inform existing and planned activities. Third, USAID and partners should build on their evidence, especially for impact evaluations and cost benchmarking, so that practitioners have the information they need to determine which interventions create the greatest value for money. Finally, policy makers and technical specialists should do a better job of communicating why cost effectiveness matters and what the administrative and cost implications might be.