Topic outline

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
  • Introduction

    This chapter provides an introduction to the concepts surrounding community monitoring, based on the "Contracting Roadmap" developed by the World Bank Institute and Participatory and Third Party Monitoring in World Bank–Financed Projects: What Can Non-state Actors Do?

    It is appropriate for CSOs who are exploring whether they can monitor procurements but applies to any entity interested in contract and project monitoring.It explains the different types of monitoring and different approaches, how to access data and general rules in what to do with it afterwards in the analysis.

    Learners from CSOs can use the "Cheat Sheets" in the nav drawer to the right for quick reference material. While this is useful for a quick overview, it will not go into the detail that the chapter does. The three Cheat Sheets can be downloaded here: 

    For the purposes of this chapter, contract monitoring is defined broadly to include all stages of a procurement process, as such budgetary and delivery of goods, services and works are included. 


  • Getting started with Contract Monitoring

    GETTING STARTED WITH CONTRACT MONITORING CAN SEEM INTIMIDATING.

    THERE ARE AN INFINITE NUMBER OF IMPORTANT ISSUES THAT YOU COULD ANALYZE. WHAT YOU WANT TO MONITOR DEPENDS ON THE PURPOSE OF YOUR MONITORING EFFORTS.

  • Accessing Information

    HOW CAN I ACCESS THE LEGAL FRAMEWORK FOR A CONTRACT?

    THE LEGAL FRAMEWORK FOR A CONTRACT, ALONG WITH DETAILS IN THE CONTRACT ITSELF, OUGHT TO PROVIDE INFORMATION ON WHAT SHOULD BE DISCLOSED OR IN THE PUBLIC DOMAIN


    The first step to contract monitoring is to identify the laws that apply to the project and what obligations are created on the part of the government and the supplier by these laws. Some countries’ constitutions may be part of the regulatory framework or have other provisions that might be implicated in the contract.  Policies, laws and regulations are usually available on the website of the relevant ministries.

  • Dealing with Lack of Information

    1. What if contracts are not available?

    Focusing only on “the contract” will limit monitoring efforts because many significant obligations may not be found in the law or in the contract.  Those include, in some cases, environmental management plans, closure plans, and social impact assessments. 

    While contract transparency is important, it is neither necessary nor comprehensive for monitoring efforts.  Even countries that rely heavily on contracts will define some or company obligations in laws and regulations. Those obligations will be publicly available.  Other countries will rely mainly on the legal framework to manage the sector. While it may not be possible to identify and monitor a company’s full set of obligations, it will still be able to conduct meaningful monitoring based on those obligations that are known.


    2. Advocate for transparency

    An often daunting challenge to contract monitoring is getting the necessary information. Many countries have no history of collecting monitoring information for the government, let alone making it available to the public.

    In some cases, even before monitoring work can begin, a campaign for transparency of information and data will be necessary. The legislative branch can be a partner in this, as legislatures are a better access point than the executive branch for civil society forces seeking for information. Strengthening the legislative branch’s own access to data and analytic capacity can strengthen civil society as well.

    International financial institutions (IFIs) such as the World Bank and the International Monetary Fund (IMF) can also be allies in the struggle for transparency. The Publish What You Pay Campaign, the Open Contracting Partnership (OCP), and civil society groups are focusing on these companies to make their dealings with governments more transparent.

    Mythbusting Confidentiality in Public Contracting by OCP is an excellent publication that can help you argue in favor of contract transparency. It is often argued that investment contracts typically deal with sensitive commercial issues, which justifies protecting the confidentiality of the negotiation, of key project information and of the contract itself. Investors may be concerned that access to sensitive information may give competitors a commercial advantage. Host governments may be concerned that future investors might invoke favorable treatment granted in earlier contracts in order to extract better terms than the government may be prepared to offer in the present. OCP argues that these arguments do not stand up to closer scrutiny. Public disclosure of contracts may promote better contractual terms. Transparency may increase pressure for more balanced contracts because the parties may be held more easily accountable for the deals they sign up to. A pool of publicly available contracts can also be a powerful way of strengthening the negotiating capacity of host governments, for instance by highlighting the diverse contractual options that might be available in different circumstances. 

    Greater transparency is also a public good in itself. Citizens have a right to know how their government is managing the natural resources it owns on behalf of the nation. Access to information and public participation in decision-making are key pillars in the concept of sustainable development (under principle 10 of the 1992 Rio Declaration on Environment and Development) and international human rights law (Article 1 of the Stockholm Declaration 1972, Article 24 of the African Charter on Human and Peoples’ Rights).



  • Who Monitors What?

    What is the role of the government?

    Government roles and responsibilities for monitoring and enforcement must be clearly defined in laws or in contracts.  If no one is assigned to monitor the implementation of certain obligations, no-one will monitor them. This challenge is especially relevant for contracts that define a range of company obligations with respect to a project.  Proper coordination among the various government agencies involved in the project and responsible for monitoring different parts of the contract is important. 

    Internationally accepted reporting standards, such as the Global Reporting Initiative’s G3.1 Guidelines can guide companies and governments in their reporting efforts.  The G3.1 Guidelines provide reporting principles and performance indicators for economic, social, and environmental indicators.


    What is the role of Supreme Audit Institutions?

    Supreme audit institutions (SAIs) conduct independent audits of governmental bodies. SAIs usually investigate the use of public funds, which involves analyzing the efficiency of government agencies and the quality of their work. This includes the management of the natural resources sector.  SAIs often make their reports publicly available.  SAI investigations can improve the management by independently assessing government agencies in this area, and recommending points for improvement. 

    The process of an audit conducted by an SAI is as follows: once expenditures have been completed and all transactions have been recorded, the executive prepares an annual report showing the total expenditures incurred by agencies during the year. The SAI then verifies these expenditures for accuracy. The SAI usually submits its audit reports to the national legislature, typically to a committee mandated to review audit findings, such as the Public Accounts Committee. The committee then reviews the information, and it may hold public hearings, where executive agency officials testify regarding any significant audit findings. The committee then prepares a report laying out specific recommendations for government agencies to implement. The entire legislature then debates and adopts the report.


    What can a civil society organization monitor?

    Some civil society organizations (CSOs) have experience in contract monitoring. Some CSOs have experience in monitoring the environmental aspects of a contract, the social aspects of a contract, or the financial aspects of a contract. Sometimes CSOs work together with communities to monitor the impact of the oil, gas and mining sector on communities. In most of these cases, the CSOs are the focal point for complaints and/or data collection from the communities, and will take the collected data to inform policy makers and companies on what is going on in communities. Some CSOs will take legal measures on behalf of the communities in case the data shows non-compliance with existing standards, rules and regulations.


    Multi stakeholder and participatory monitoring

    Participatory monitoring seeks to collaboratively identify and solve problems through a process of data collection, analysis and communication between the company, community and sometimes also the government. Thus, the monitoring is done through a multi-stakeholder partnership. Traditionally, companies and agencies initiate and undertake monitoring, and in turn learn and benefit from the results. Participatory monitoring requires a change in dynamic so that a wider range of stakeholders initiates, undertakes and learns.


  • Non-State Monitoring

    IF YOU CANNOT GATHER ENOUGH INFORMATION OR YOU DO NOT TRUST WHAT IS PROVIDED YOU CAN ALSO START COLLECTING/GENERATING YOUR OWN DATA ON CONTRACTS.

    THIS IS NOT AN EASY PROCESS AND NOT SUITABLE FOR EACH AND EVERY TOPIC. ANY ORGANIZATION CAN START ITS OWN MONITORING PROGRAM.

    If you cannot gather enough information or you do not trust what is provided you can also start collecting/generating your own data on contracts. This is not an easy process and not suitable for each and every topic. Any organization can start its own monitoring program. 

    Monitoring partnerships across government, industry and civil society seek to use the overlapping interests of parties to ensure checks and balances at every level.  Forming partnerships for monitoring can help to address capacity, transparency, and incentive concerns. Communities can partner with companies and governments to monitor issues of particular concern to the community and to report back to the government on compliance with related obligations. This requires companies to share information with communities. It ensures that those with the greatest incentive to see an obligation enforced are the ones monitoring its implementation.

    Non-state monitoring (NSM) is defined as a process where parties other than the state and donors track the implementation of development projects or programs and obtain beneficiary feedback to increase accountability to the beneficiaries.3 NSAs, according to this definition, include beneficiaries, communities, CSOs, think tanks, research institutions, academia, media, for-profit firms, labor and business associations, voluntary associations such as school boards, and other groups.

    The next section describes the two main forms of non-state monitoring (NSM): third party monitoring (TPM) and participatory monitoring. NSM can be seen as an umbrella concept to represent a continuum from participatory monitoring to third party monitoring. NSM often combines elements of both because they complement each other.4

  • THE DESIGN OF MONITORING BY NON-STATE ACTORS



    This section presents a (slightly adapted) step-by-step approach to the design of NSM as written by the World Bank in ‘HOW-TO NOTES Participatory and Third Party Monitoring in World Bank–Financed Projects: What Can Non-State Actors Do?’, 2013. 

    The four-step process is a guide to help project teams determine the applicability of NSM to their project needs and to incorporate it into their project.

  • Implementation of non state monitoring

    This section helps you choose which tool or system is most appropriate for your environment. The. IFC (2008) shows that, broadly speaking, there are four different types of monitoring systems which are outlined below. These systems have incorporated many tools some of which are displayed in the blue box on the right. Once you have read about the four main systems, you can carry out the activity beneath to ascertain which tool is the best for you.

    four main monitoring systems:



    In this system, community members are trained to be observers of their environments. Citizens collect data requiring limited technical resources and which is based on field observations rather than laboratory analyses. 

    Potential benefits are that the process is simple, inclusive, low-tech, and low-cost. It requires minimal input from external technical experts. Potential challenges are that training can be difficult. Data quality is not sufficient for decision-making. Also, It can be difficult to maintain a program that relies on volunteers. A community watch program is most effective when the purpose is education and general awareness of project impact, and the project is not controversial.

    This system is particularly effective in: The monitoring of the implementation stage of a goods based contract; essentially checking whether the goods are delivered. This includes things like school textbook delivery; Construction of schools; Construction and quality of roads and; Contractor work in community-driven development projects. 

    Tools & methods often used in this system: Collecting data via mobile phones and tablets, geo-tagging the data

    Risks: Relatively limited in what can be observed and understanding may be low compared to experts




    In this type of system, community members accompany representatives of the project sponsor, company, or the government. This approach allows community members to see the site, learn about the monitoring protocols, and observe the company doing its monitoring work. The approach relies on the efforts of careful community observers who will accurately communicate what they see to the public. Potential benefits are that limited training is required, and the process gives participants a chance to see the project area, which can change perceptions of citizens when they actually see the site and find out what goes on inside the project.

    Potential challenges can be that the process is controlled by the project sponsor, company, or government and thus lacks independence. A network of observers is most appropriate when the purpose is education and general awareness, development of a baseline, or capacity building, issues are highly technical, and the project is not controversial. 


    This system is particularly effective in:
    The monitoring of the implementation stage of a works based contract; essentially checking whether infrastructure is on track and is safe. It can also be used to see if consultants have been mobilised in a service contract.

    Tools & methods often used in this system: Regular site visits, trainings, periodic meetings with project monitoring units, separate fact sheets and monthly summaries 

    Risks: Quite intensive in terms of mobilizing and training the right people. lack of understanding of the technical elements of the project may be limiting.




    A cooperative effort that integrates existing monitoring programs conducted by the company, civil society, and government. It relies on joint fact finding. Participants and technical staff collect some data at strategic locations and analyze these data with as much rigor as the participating institutions. The approach creates an integrated database; supports an integrated approach to interpretation; provides quality assurance; and ensures that results, conclusions, and recommendations are communicated to the public at large and to affected communities.

    Volunteer participants observe data collection in the field. A technical working group works with technical staff to analyze data, identify issues of concern, engage in problem solving with the company to address problems, and communicate to the public. The approach relies on the good will of the company to make changes. Potential benefits are that the process has high credibility, transparency, and independence. It may decrease the chance of competing data and interpretation among organizations that engage in monitoring. Builds public trust in data collected by the company and public agencies. It provides a balance of internal and external expertise. Potential challenges are that the process is complex to implement and requires dedicated technical experts. There is potential for conflict with participating institutions over data interpretation. Monitoring committees are most appropriate when the purpose is to investigate a potential problem, address public uncertainties, or evaluate the effectiveness of improvements, when good technical capacity is in place, and when the issue is controversial or conflict-prone.

    This system is particularly effective in: Monitoring budget transparency (planning) as well as quality of service provision (implementation).

    Tools & methods often used in this system: focus group discussions, community scorecards, citizen report cards, public hearings, simplified budget sheets, citizen budgets used during meetings, inputs into draft budget documents during budget preparation, social audits

    Risks: May be dominated by elite few and lack representation




    Independent experts are contracted to conduct monitoring. The team of experts makes regular field visits. It meets with company, community, and other stakeholders as part of each field visit: at the onset, to hear complaints; and at the end of the monitoring visit, to brief interested parties.

    Technical experts collect original data and draft a report, which is made public. The technical experts intentionally do not engage with the company in negotiating compliance; rather they guard public interest by making findings transparent and use moral authority to compel the company to make needed changes. The community may be involved in developing the terms of reference and selection process. The community may or may not accompany experts in the field. Potential benefits of this system are that a highly profession panel or organization of technical experts can monitor geographically diverse and extensive areas, such as the terrain and communities along a pipeline route. It can provide a high degree of technical credibility. It may be perceived as more independent than other approaches. It may be able to stay above the political fray. It can consult with civil society without having to create a dialogue table of all stakeholders. It may be particularly effective when people are unable to talk together for geographical reasons or because of intractable conflict. 

    Potential challenges are that experts may not be familiar with or sensitive to project specifics such as the history, constraints, local cultures and conditions, and the personalities involved. Difficulties include the cost, the effort and time involved in gaining legitimacy and credibility in the eyes of civil society, the challenge of gaining compliance with recommendations, and the overarching question of to whom the experts are accountable. A system involving independent technical experts is most appropriate when the purpose is to investigate a potential problem, address public uncertainties, or evaluate the effectiveness of improvements. The issues are so technically complex and difficult to understand that no amount of capacity building and experience will prepare the community to address them. There is a high degree of controversy and a low degree of trust among the participants. Parties are politically or ethnically divided, unable to build an effective working relationship, and cannot agree on an approach. Also appropriate when the spatial extent of the project is great and encompasses multiple ecological zones and communities that have little cohesiveness (such as with a pipeline). 

    A smaller-scale, less expensive version of this approach is appropriate when the situation is not so highly polarized, the risk is minimal, and the community is most interested in engaging a competent technical expert to periodically monitor the performance of the sponsor and report back to the public on a quarterly, semi-annual, or yearly basis

    This system is particularly effective in: Monitoring implementation of contracts that are highly technical or complex.

    Tools & methods often used in this system: In-depth evaluation and/or assessment

    Risks: Expensive to operate on a wide scale



    World Bank Institute. ‘Contracting Monitoring Roadmap’, 2013.
    World Bank. ‘HOW-TO NOTES Participatory and Third Party Monitoring in World Bank–Financed Projects: What Can Non-State Actors Do?’, 2013.


     

    Community forums
    Gatherings of people to give and/or receive information about a particular issue or subject. These usually require a clear and public agenda, and careful choice of location, timing and language to ensure that local preferences and responsibilities are accommodated. {mlang{ {mlang ne} समुदाय फाेरमहरु कुनै समस्या वा विषयका बारेमा जानकारी दिन वा लिनका लागि मानिसकाे जमघट । यसमा सामान्यतः स्पष्ट सार्वजनिक एजेण्डा हुनुपर्दछ, हाेसियारीपूर्वक स्थान, समय र भाषाकाे छनाेट भएकाे हुनुपर्दछ जसले गर्दा स्थानिय प्राथमिकता एवं जिम्मेवारीहरुकाे सम्बाेधन हुने कुरा सुनिश्चित गर्न सकिन्छ ।

    Community suggestion
    Boxes Boxes placed in an easily accessible public location. Members of a community may submit anonymous complaints, suggestions or questions. Box is opened publicly at pre-determined times (such as weekly) and a response provided to each suggestion.  

    Good neighbor agreements
    Agreements that are co-produced between companies and communities, usually based on traditional relationships among community members, to reach joint agreement among multiple stakeholders – company, communities and others -- on how issue(s) of mutual interest or concern will be addressed. Agreements are enforced through social commitments rather than legal ones.

    Participatory budgeting
    A process by which citizen delegates decide on or contribute to decisions regarding the allocation and monitoring of expenditures of all or a portion of a company’s social investment resources. {mlanf ne} सहभागितामूलक बजेट निर्माण एउटा यस्ताे प्रकृया जसमानागरिक प्रतिनिधिहरुले कुनै कम्पनीकाे सामाजिक लगानीका सम्पूर्ण वा आंशिक श्राेतहरुकाे बाँडफाँड एवं खर्चकाे अनुगमनसम्बन्धी निर्णय गर्न वा साेमा याेगदान गर्नका लागि निर्णय गर्दछन् । {mlang}

    Community Scorecards
    Participatory processes by which community groups can monitor and/or evaluate a service offered for their benefit. Beneficiary groups and service providers identify key indicators of success or progress, and rate effectiveness on a simple scale.

    Citizen Report Cards
    The Citizen Report Card (CRC) is a simple but powerful tool to provide public agencies with systematic feedback from users of public services. This includes approaches like social / performance audits and satisfaction surveys  




  • Choosing Which Tools and Systems


    This section elaborates a four-step process to help project teams think through NSM implementation in their projects


    Introducing NSM requires a period of adjustment to work out the details. Usually it is a new experience and people are not sure what to expect. It is best not to rush the process. Everyone needs to understand their roles. The first round of NSM is a testing period to see how it works and if it is ready for full-scale use. This is when a lot of the assumptions during NSM design will be put to the test. Even at this early stage it is vital to obtain feedback from the main users and implementers of the NSM initiative before full rollout. Pilot testing can identify any additional needed capacity building. This is the time to aim for some early successes to strengthen commitment to the process. NSM should not be seen as a one-off event, but as an evolving process.



    Once the NSM is under full implementation, the data needs periodic interpretation and analysis. The monitoring organization can do the first cut of data analysis, but then it helps to bring other stakeholders into the analysis. Which stakeholders are involved depends on the project, but they include the government, beneficiaries, communities, project-affected people, and others with a stake in the findings. When possible, Bank staff should also participate, including financial management, safeguards, and relevant sector specialists. Stakeholder participation is needed in interpretation, processing, and analysis of the data, especially for participatory monitoring. Stakeholders engage in critical reflection on project performance, problems and constraints. The audience determines how the data should be presented in a format they can easily understand. Documentation should be simple, clear, brief, timely, and accessible. Then the stakeholders need to discuss how the findings will be used. Sometimes this includes an action plan to address any issues that NSM revealed. Data analysis and action planning should not be seen as threatening, antagonistic, or fault-finding, but rather as constructive engagement to find solutions.



    NSM itself needs to be monitored to identify implementation problems. Supervising NSM often takes more time than supervising regular monitoring because the monitoring is not done by the project team or government agency. The extra time required needs to be planned for. The project implementation unit and Bank team need to actively guide the monitoring organization. This includes refining the methodology (and advising on what information would be useful), methods of analysis, and reporting formats. Although NSM has been used to monitor projects that have numerous, widely dispersed, or conflict-affected sites, implementation can reveal limitations that require adjustments. For example, in the ARTF, the number of proposed sites for monitoring was overly ambitious and had to be scaled back. An important part of the process is ensuring communication flows among stakeholders so that the findings are used. Monitoring by itself does not improve project performance unless there is clarity and continuity in the monitoring, and in the working relationship between the monitoring agency, the government, and project staff. Therefore, it is critical to develop feedback channels from government to communities, project beneficiaries, and others, and to ensure that feedback is used to change project processes. The Bank can serve a bridging function and promote NSM and using the findings. Some ways to help ensure that NSM findings are utilized include: 

    • Regular meetings between the government and the non-state actor to discuss monitoring findings 
    • Discussing necessary actions and possible remedies to address the problems across the project
    • Joint missions so everyone learns from each other on what to look for during site visits
    • Training on project monitoring, report writing, and how information from project monitoring should feed into decision making and inform possible changes to project design NSM also needs to document information about the NSM process itself and to assess its impacts to determine the value added of NSM. One needs to be realistic about how much NSM can achieve in the short run, especially in terms of fostering accountability. This requires a long-term agenda, since it involves measuring changes in attitudes, values, behaviors, and incentives.


    The activities covered under non-state monitoring should be designed to institutionalize relationships between the government and citizens as much as possible over the long term, rather than “projectize” relationships. Some ways this can be done include:

    • Promote close links between government officials, media, CSOs, and communities 
    • Support ways of increasing the comfort level and recognition of monitoring done by nonstate actors in ministries and the project management units 
    • Support networking, experience-sharing, and peer support among non-state actors 
    • Tailor manuals and other materials to leave behind beyond the pilot monitoring initiative
    • Share results with the public in a way that is meaningful to their everyday realities and that engages them in their own spaces

    Task managers are encouraged to attach NSM reports to supervision mission aide memoires and implementation supervision reports (ISRs). Sharing findings with a broader audience— for example, through a dissemination workshop and a website to make reports accessible online—can increase NSM impact and build support for sustaining it. The pilot NSM can be passed on to high-capacity CSOs. Finally, many NSM initiatives are not intended to be scaled up, but if the pilot is successful enough and there are sufficient reasons for expanding it, then the team may consider several factors. Is the existing initiative appropriate for scaling up? Are any changes necessary for scaling up? That is why documentation of NSM is necessary, to understand what is working and why, so any shortcomings can be corrected before scaling up. What works in one location might not work in other locations with other stakeholders. The original initiative may need adaptation and other adjustments. What additional burdens will scaling up place on NSM actors? They may need capacity building. New actors may also be needed.



    Monitoring data is collected by field engineers using smart phones and data collection applications developed by International Relief and Development (IRD). The data is used to produce monthly inspection reports and quarterly analytical reports identifying issues that need addressing. The data and findings are discussed in monthly meetings between IRD, the Bank, and government ministries. Quarterly meetings with ARTF management examine cross-cutting issues. Joint missions go to the field and a supervisory agent participates in Bank supervision missions. These reports, meetings, and joint activities facilitate data analysis, validation, and utilization.

    1. Working with non-state actors: 

    • Be realistic about what non-state actors can achieve. 
    • Allow sufficient time for consultation and training. 
    • Be prepared for capacity building and support. 
    2. Working with government: 

    • Present NSM as troubleshooting project risks and problems rather than as a watchdog. 
    • Use NSM to improve government performance rather than criticize it. 
    • Demonstrate the value of NSM with practical examples. • Identify and work with champions. 
    • Align incentives to convince service providers about the benefits of NSM.
    • Consider working with local governments that may be more open to NSM.
    3. Getting government and non-state actors to work together:

    • Focus on building the relationship between government and non-state actors.
    • Capacity building on NSM should include government officials and non-state actors.
    • Understand the organizational culture of partner organizations and their experience with NSM.
     4. Working with communities:

    • Community committees should include men, women,youth, and marginalized and vulnerable groups. 
    • Use local institutions and NGOs to support monitoring committees
    • Train community members in project monitoring, reporting, and communication skills.
    • Start slowly so people can become comfortable in articulating their views.
    • Adjust to community members’ schedules, for example by conducting meetings on weekends or in the evening.
    • Hire high-quality community facilitators. 5. Improving communication: 
    • Build public understanding for NSM.
    • Conduct information sessions and disclose project information to raise awareness.
    • Keep the communities informed about findings and on how their feedback is being used.
    • Work with the media and other communication channels to increase the outreach of NSM.
    • Consider what information should be shared with which stakeholders.
  • Analyzing the Data

    Monitoring itself is not the end goal. The information collected through monitoring efforts should be used to hold companies and governments accountable. But information and data alone do not have value until they are interpreted. Only once the information collected through monitoring is interpreted and contextualized, can it be used to generate dialogue. Therefore, this section focuses on analyzing and interpreting data.

    Data interpretation involves asking questions that lead to findings and conclusions. Findings represent what the data show. Conclusions explain why the data look the way they do. It is important that the data support the conclusions, so find expert advice when necessary. Reporting on findings might take the form of community presentations, score cards, or any method that is culturally appropriate and tailored to the unique situation of the community.

    To report Local Content is more than simply the reporting of the figures themselves. It is about understanding, and above all explaining, precisely what it is that the metric is measuring, and what is happening in the wider economic or business environment that might be influencing the figures and trends.