The fourth step is to decide on the practical implementation details such as who, how, the scale, budgeting, and so on. This is the heart of designing NSM. Key questions include the following: Who will collect the data and from whom? Will it be done jointly with government? How will it be reported? Who will be using it? How and when will it be used? Stakeholder analysis and other research might be necessary to answer these questions. This note provides only a set of issues to consider and some practical advice. There is no fixed set of steps to follow, but most NSM has to address the issues discussed in this section. Based on experience, one key principle is simplicity
Gathering data and information. NSM is more than a tool. What matters is how it is used. The primary issue is collecting the kind of data that can be processed, organized, and interpreted as meaningful information. Questions need to be phrased in a way that people can understand and provide reliable responses. Key questions to be considered before developing questionnaires and collecting data include:
- What kind of information is required?
- What kinds of methods will be used to obtain it?
- How will the indicators be determined and by whom?
- What are the key indicators that NSM should focus on?
Information could be about services communities are supposed to receive from the project, positive and negative impacts of the project, and ways of improving project implementation and its impacts. The desired information goes far beyond indicators, which have a limited ability to capture what matters to many stakeholder groups and are often inadequate at explaining causes. Therefore, the focus should be more on the needed information than the indicators. One method is to ask project stakeholders to list their most pressing concerns as a means to identify relevant indicators and issues. This also helps develop a shared understanding of indicators among various actors.
Reporting. An important planning step is deciding on how to report the information collected through NSM. There are numerous cases where the data collected has been conveyed by the monitoring actor in long descriptive reports, lacking key conclusions and analysis. One way to minimize this risk is to agree on the reporting formats as part of detailed terms of reference9 and provide clear guidance and reporting templates to the monitoring actor.
Scale and representativeness. Another choice is how broadly monitoring will be applied. NSM can be at the national, regional, district, local, or community level. It can be piloted in a few areas or used project-wide. Resource limitations usually require a trade-off between broad sampling and intensive NSM in a few locations. Whatever choice is made, the data needs to be sufficiently representative to derive reliable conclusions about project performance and needed actions.
Timing. NSM is preferably an element of the project from its design, so that it becomes a part of the project’s DNA and is properly resourced. It is possible to integrate it later on (good opportunities are during midterm reviews or restructuring), but experience has shown that a later start might leave less room for maneuvering. NSM can also take place toward the end of the project for assessing beneficiary satisfaction and as part of project evaluation. Another timing issue is the frequency of NSM: Will it be a one-time exercise, periodic, or ongoing throughout the project? Timing and frequency issues depend on the purpose of NSM, the methods used, and the available resources.
Selecting the monitoring organization. This is clearly one of the most important decisions. The capacity and skills of the monitoring organization must be sufficient to implement the NSM. Participatory monitoring requires the capacity to mobilize NSM participants. Capacity requirements will narrow down the list of potential organizations. It is highly desirable that the organization not only has experience on the issue or methodology, but also an in-depth country- or region-specific knowledge. This is many times overlooked in NSM of contracts or projects. The NSM organization should have a reputation for integrity, competence, and credibility so that the findings will be accepted. Good reputation also plays a role when this organization is “paid for” by the project. The organization’s relationship with the government, project, beneficiaries, and communities has to be good enough that it is respected, trusted, and accepted. However, for third party monitoring, the relationship should not include full dependence on the government or project that might jeopardize the third party’s ability or incentive to provide an independent perspective.
Capacity building. Often, capacity building is needed for one or more NSM actors. For example, CSO capacity is often concentrated in the capital city. CSOs in the project area may need training. Determine which actors need capacity building and in which areas. This could begin with identification of core information needs.
Budgeting. The budget for NSM itself may be self-evident, but there are several associated costs. Government or external staff time for providing support for NSM tends to be underestimated, especially in low-capacity and fragile environments. This is in addition to the budget required for training and capacity building. Quite often, NSM requires facilitation, feedback strategy, and tracking the performance of the monitoring organization. The costs of scaling up and institutionalization, if that is planned, need to be factored in.
Integrating NSM into the project. NSM is not likely to be very useful if it is not well integrated into the project. The findings may not be used and it will be a wasted effort, even if NSM is done well. The key is to make NSM integral to the project and its success. One way is to make NSM findings relevant to project decision making and performance. Link NSM to regular internal monitoring in the project and the project results framework (see Box 14). If NSM was not part of the original project design, making this link can take time. If NSM is focused on government performance, it needs to be integrated into government monitoring systems or it will likely be ignored. Ongoing dialogue with and active participation by government officials in NSM activities improves the likelihood that project implementers and decision makers will value and utilize the findings from NSM.