In 2011, when WaterSHED introduced the Hands-Off market-based sanitation program, the rural sanitation market was negligible. To jumpstart the market and ensure sustainable change, we knew we would have to build out and fortify the market system at multiple levels. We sought not to just design a program, but rather to support the growth of a sustainable system that would exist long after WaterSHED’s exit. With every step, we needed metrics that would inform us about the sustainability of program impact at each level. Therefore, our monitoring metrics evolved with the market system. We chose an adaptive monitoring approach that was intentionally responsive to learning through in-depth research, field experience, and monitoring.
In the early years (2012-2014) of the Hands-Off market-based sanitation program, WaterSHED tracked development of missing elements in the system (e.g., presence and capacity of toilet producers, sales agent recruitment). As the market progressed and these elements matured, we identified new indicators of market sustainability. In 2016, we began tracking the proportion of latrine purchases facilitated by WaterSHED staff. By 2017, nearly all purchases were made independently of NGO facilitation – a key signal of sustainable market activity.
In 2016, WaterSHED also shifted to government-led data collection mechanisms in which village chiefs collect household-level sanitation data on a monthly basis and submit it to commune councilors to be aggregated. This exemplifies the ethos of our approach – we favor locally adapted and locally led mechanisms and sustainable solutions – not just for programming but for measurement as well. The government-led data collection mechanism was able to capture market activity outside of WaterSHED’s network (i.e. suppliers receiving technical skills training and business professionalization training). Much of the additional market activity is likely the result of copycat businesses, which have sprung up in WaterSHED target areas — another exciting indication of a healthy market.
However, increases in market activity are not the result of copycat businesses alone. We have learned that local leaders play a critical role in the system and have, in some areas, facilitated significant growth in market activity. In 2017, districts with government officials who had undergone leadership development training as part of WaterSHED’s Civic Champions program had, on average, 116 more latrine purchases than districts with untrained leaders. While it doesn’t provide the most nuanced view of market activity, this metric gives us a strong indication that our interventions aimed at the system’s enabling environment are having real, tangible impacts on service delivery.
This adaptive monitoring approach* has enabled us to be responsive to lessons about and shifts in the very system we attempt to build and monitor.
Download the slide presentation for more information.
*The metrics and methodologies described here were developed within and for the rural Cambodian context and may not be appropriate for programs that require metrics for comparison across contexts.