Setting indicator baselines and gathering data for government monitoring and evaluation systems

THE government of the Republic of Zambia is currently leading the process of formulating the Eighth National Development Plan (8NDP 2022-2026). According to the 2014 National Planning and Budgeting Policy as well as the 2020 National Planning and Budgeting Act, Government is obliged to produce a national development plan (NDP) before the current one 7NDP expires. Thus, since the 7NDP (2017-2021) is expiring at the end of this year, the new UPND Government led by President Hakainde Hichilema has a legislative mandate to develop and launch the country’s development blueprint, the 8NDP within the first quarter of 2022. In every NDP, one of the critical success elements include the careful articulation and final selection of performance indicators. Indicators are the only factors that help in tracking and measuring changes from government interventions (projects, programmes & policies). For instance, we cannot tell if positive or negative results are being achieved from any intervention in the absence of indicator information. Today, I am sharing on this crucial subject matter of setting indicator baselines and gathering data for government-wide M&E systems. Simply put, a baseline is the first [reference] measurement of any indicator. Without baseline information, you implement a project like you are blindfolded—you cannot tell how near or far you are to the desired results!

After working through the process of selecting key performance indicators to monitor development outcomes, you establish baseline data, that is, establishing where we are at present relative to the outcome we are trying to achieve. One cannot project performance into the future (set targets) without first establishing a baseline. The baseline is the first measurement of an indicator. It sets the current condition against which future change can be tracked. For instance, it helps to inform decision makers about current circumstances before embarking on projecting targets for a given programme, policy, or project. In this way, the baseline is used to learn about current or recent levels and patterns of performance. Importantly, baselines provide the evidence by which decision makers are able to measure subsequent policy, programme, or project performance.

Establishing Baseline Data on Indicators:

Baselines are derived from outcomes and indicators. We would note that establishing baselines is not an exotic idea. We gauge our personal performance against our own baseline data in our own lives. For example, we check our blood pressure against what we have had at one time in the past, track our capacity to exercise against our performance when we first began to exercise, and keep an eye on our weight against an earlier weight. A performance baseline is information—qualitative or quantitative—that provides data at the beginning of, or just prior to, the monitoring period. The baseline is used as a starting point, or guide, by which to monitor future performance. Baselines are the first critical measurement of the indicators. The challenge is to obtain adequate baseline information on each of the performance indicators for each outcome. This can quickly become a complex process. It is important to be judicious in the number of indicators chosen, because each indicator will need data collection, analysis, and reporting systems behind it.

Building Baseline Information:

There are eight key questions that should be asked in building baseline information for every indicator. What are the sources of data? What are the data collection methods? Who will collect the data? How often will the data be collected? What is the cost and difficulty to collect the data? Who will analyse the data? Who will report the data? Who will use the data? The statistical systems in developed countries frequently can deliver precise information for all three stages of traditional implementation monitoring—inputs, activities, and outputs. However, developing countries generally have less sophisticated systems. The data systems may not be available and may vary with respect to precision. Some countries will know with reasonable precision how many rural children are in school, while others will have only rough estimates. Other developing countries may know the utilisation rates of hospital beds, and some may not. The selected performance indicators, and the data collection strategies used to track those indicators, need to be grounded in the realities of what data systems are in place, what data can presently be produced, and what capacity exists to expand the breadth and depth of data collection and analysis.

Identifying Data Sources for Indicators:

Every indicator constitutes its own miniature M&E system, so the first consideration in starting to build the information system for that indicator is what sources of information potentially can supply the relevant data. A number of issues need to be considered when identifying data sources. Can the data source be accessed in a practical fashion? Can the data source provide quality data? Can the data source be accessed on a regular and timely basis? Is primary data collection from the information source feasible and cost effective? It is important to collect only the data that is intended to be used. After all, performance information should be a management tool—and there is no need to collect information that managers are not going to use. “As a rule of thumb, only collect baseline information that relates directly to the performance questions and indicators that you have identified. Do not spend time collecting other information”.

Data sources for indicators can be primary or secondary. Primary data are collected directly by the organisation concerned, and may include administrative, budget, or personnel data; surveys; interviews; and direct observation. Secondary data have been collected by other outside organisations, and are gathered for purposes other than those of the organisation concerned. Examples of secondary data include survey data collected by another agency (Zamstats, UN, World Bank, etc). For instance, financial market data, or demographic health survey data, living conditions survey data. There are pros and cons associated with the use of secondary data to establish performance trends on indicators. On the positive side, secondary data can be more cost efficient. Secondary data may also be used in instances when it is not practical or possible to collect primary data frequently, as in the case of large scale and expensive household surveys.

However, for a variety of reasons, secondary data must be used with caution. Secondary data will have been gathered with other organisation goals or agendas in mind. Other questions arise in using secondary data as well: Are the data valid? Are they reliable? How often are the data collection instruments validated? Furthermore, using secondary data means using someone else’s data to report progress and success in moving toward your own desired outcomes. Are you as a manager comfortable with this arrangement, given all the advantages and disadvantages of doing so? Examples of sources of actual data may include administrative records (written or computerized) from government and nongovernment organisations; interviews and surveys with client target groups, programme officials, and service providers; reports from trained observers; and mechanical measurements and tests.

An increasing understanding of the need for streams of information, not discrete studies that are episodic and spaced out over time, is emerging in public sector organisations throughout the world. Managers are looking for information—whether on policy strategies, utilisation of health clinics, farming methods, or migration patterns—that they can trust and use in real time. Waiting for months or even a year or more for studies to be completed is not helpful. The new approach to building results-based M&E systems is increasingly toward building those systems that provide more or less continuous information streams. Therefore, it is my hope that the UPND Government will produce the 8NDP with sound performance indicators accompanied with baseline information for each one. Aluta continua for a stronger Government-wide M&E system and measurable development process informed by indicators with baseline information.

Dr. Vincent Kanyamuna holds a Doctor of Philosophy in Monitoring and Evaluation and is lecturer and researcher at the University of Zambia, Department of Development Studies. For comments and views, email: vkanyamuna@unza.zm

Leave a Reply

Your email address will not be published. Required fields are marked *