ODS-3

Methodological Note on the Monitoring Dashboard for SDG 3 Targets, Indicators, and Inequalities

The Monitoring Dashboard for SDG 3 Targets, Indicators, and Inequalities is a tool for visualizing and tracking the evolution over time of the health indicators for Sustainable Development Goal 3 (SDG 3), assessing achievement of the targets set for 2030, and exploring the magnitude of social inequality gaps and trends, explicitly upholding the promise to “leave no one behind”.

For each selected geographical area (“Place”), each SDG 3 health “Indicator”, and each “Equity Stratifier”, the dashboard shows four basic monitoring elements: 1) general trend of the health indicator and its projection for 2030; 2) trend in absolute and relative social inequality gaps; 3) percent change in the period, average annual percent change in the general average, and absolute and relative inequality gaps between two specific years; and 4) distributive inequality of the selected health indicator according to social quintiles determined by the equity stratifier selected in the two calendar years selected by the user.

Trends in projected values. The values projected for 2030 correspond to extrapolation of the exponential model that adjusts the annualized series of observed values, according to the following equation:

where y_i is the predicted value of the health indicator at the time t_i, α is the intercept, β is the slope or intrinsic growth rate, and y is the Naperian base that determines the exponential function.

Trends in absolute gap. The absolute gap is a simple summary metric of health inequality; it corresponds to the arithmetic difference in the value of the health indicator between two socially determined population groups, usually at opposite ends of the social spectrum. On the dashboard, these metrics are calculated according to the following expression:

where AG is the absolute gap; HI is the health indicator; q₁, the most disadvantaged social quintile, and q₅ is the most advantaged quintile. The absolute gap is expressed in the same unit of measure as the health indicator; a zero (0) value for the absolute gap denotes absence of inequality.

Trends in relative gap. The relative gap is a simple summary metric of health inequality; it corresponds to the arithmetic ratio of the value of the health indicator between two socially determined population groups, usually at opposite ends of the social spectrum. In the dashboard these metrics are calculated according to the following expression:

where RG is the relative gap; HI is the health indicator; q₁ is the most disadvantaged quintile, and q₅ is the most advantaged quintile. The relative gap is expressed without units of measure (that is, its value represents the number of times that the numerator fits into the denominator). A relative gap value of one (1) denotes absence of inequality.

Percent change in the period. The percent change is a general progress metric over time. In the dashboard, the percent change (PC) in the magnitude of a selected health indicator (HI) in the period between year 1 (t₁) and year 2 (t₂) is calculated according to the following equation:

and is expressed in percentage points (compared to year 1).

Average annual percent change. The average annual percent change is a summary metric of the trend during a specified fixed interval and expresses the average growth rate (or decrease). In the dashboard, the average annual percent change (AAPC) in the trend for a selected health indicator (HI) during the period between year 1 (t₁) and year 2 (t₂) is calculated according to the following equation:

and is expressed in percentage points (by unit of time, that is, per year). In general, the value of AAPC reflects the speed of change of a health indicator over time. If the health indicator has negative polarity (i.e., when a lower value for the health indicator over time indicates a more favorable situation, for example, the death rate), the AAPC reflects the average annual percent decrease. If, on the contrary, the health indicator has positive polarity (this is, when a higher value of the health indicator over time indicates a more favorable situation, for example, coverage of care), the AAPC reflects the average annual percent increase.

Equiplot. An equiplot is a specific way to visualize the distribution of a health indicator in social quintiles, defined according to an equity stratifier. The dashboard has a weighted equiplot, representing the quintile distribution of the health indicator in the two years selected for comparison, with the size of the spheres proportional to the size of the population in each social quintile. The color expresses the social position gradient, from the most socially disadvantaged quintile (red) up to the most advantaged (blue). The location of each sphere (quintile) on the horizontal scale corresponds to the value of the health indicator. The spread between the spheres and, specifically the difference between the top and bottom quintiles (at the extremes of the social spectrum) reflects the magnitude of social inequality for the health indicator and its respective absolute inequality gap. For health indicators with negative polarity (such as death rates), progress over time is reflected in a shift of the distribution toward the left (toward zero on the horizontal scale). For health indicators with positive polarity (such as care coverage), progress over time is reflected in a shift of the distribution toward the right (toward universal coverage).

Analysis tools

Methodology for monitoring indicators

Methodology for target setting

Instruments for the measurement, analysis and monitoring in health

SUSTAINABLE DEVELOPMENT GOALS 3:

Methodology for Regional and National Monitoring of SDG 3 Targets, Indicators, and Inequalities

SUSTAINABLE DEVELOPMENT GOALS 3:

The methodology proposed below complies with the general and specific recommendations of the World Health Organization (WHO) set forth in the following three policy documents:

Monitoring, evaluation, and review of national health strategies: a country-led platform for information and accountability (2011)
Handbook on health inequality monitoring: with a special focus on low- and middle-income countries (2013)
National Health Inequality Monitoring: a step-by-step manual (2017)

The methodology consists of the following two main components:

Monitoring of the indicators for the targets in Sustainable Development Goal 3 (SDG 3)
Monitoring of inequalities in the indicators for the targets of SDG 3

Furthermore, it covers two major areas:

Regional
Nacional

Figure 1. Basic flow chart for monitoring inequality in health

Step 1. Scope

Select the list of indicators for the SDG 3 targets that will be validated, collected, and monitored by each country, out of the total of 27 indicators identified. For the selected indicators, establish 1) their definitions and technical specifications, and 2) the calculation methods that will be used.
For monitoring the SDG 3 indicators themselves, it is recommended to make a minimum selection that covers each of the following six categories of indicators:
1. 1. Maternal and child health: 1.1, 3.2.1, 3.2.2
  2. Communicable diseases (GBD.1[1]): 3.1, 3.3.2, 3.3.3., 3.3.4, 3.3.5
  3. Noncommunicable diseases (GBD.2): 4.1
  4. Injuries (GBD.3): 4.2, 3.6.1, 3.9.3
  5. Health hazards: 5.2, 3.7.2, 3.9.1, 3.9.2, 3.a.1
  6. Access to health services and health service coverage: 1.2, 3.5.1, 3.7.1, 3.8.1, 3.8.2, 3.b.1, 3.b.2, 3.b.3, 3.c.1, 3.d.1
To monitor inequality in health, agree on a subset of basic tracer indicators of health inequalities (BTIHIs) from the list of 27 indicators for the SDG 3 targets. This subset of BTIHIs should be common to all countries and the agreed selection criteria must:
- consider the four BTIHIs already selected to monitor the impact level 1 indicator in the Strategic Plan of Pan American Health Organization 2020-2025: “reduction of within country health inequalities.”
- cover the four basic stages of the life course (early childhood, adolescence and youth, adulthood, and old age).
- cover the two basic dimensions of health (health outcomes and contact with the health services system).
- cover, as much as possible, the six categories of indicators indicated above.
The following basic list of BTIHIs is proposed:
1. Neonatal mortality rate (3.2.2)
2. Adolescent birth rate (aged 10-14 years; aged 15-19 years) per 1,000 women in that age group (3.7.2)
3. Tuberculosis incidence per 100,000 population (3.3.2)
4. Mortality attributed to cardiovascular disease, cancer, diabetes, or chronic respiratory disease (3.4.1)
The list can be expanded with the following BTIHIs:
1. Death rate due to road traffic injuries (3.6.1)
2. Coverage of essential health services (3.8.1)[1]
3. Health worker density and distribution (3.c.1)
4. Proportion of births attended by skilled health personnel (3.1.2)
5. Proportion of women of reproductive age (aged 15-49 years) who have their need for family planning satisfied with modern methods (3.7.1)
6. Proportion of the target population covered by all vaccines included in their national program (3.b.1)
7. Maternal mortality ratio (3.1.1)

Reach a consensus on the possible magnitude of inequality (i.e., stratify equity). In principle, it should include the stratifiers included target 11.1 of the Sustainable Health Agenda for the Americas (SHAA 2030), namely: place of residence (urban/rural); race, ethnic group; occupation; gender, sex; age; education, or socioeconomic status. It is recommended that at minimum, a consensus be reached on an equity stratifier that represents at least one of the three dimensions of sustainable development (social, economic, and environmental), or a population group recognized to be in a situation of greater social vulnerability, together with the equity reference population group (i.e., the comparison group).

[1] GBD: global burden of disease. [2] Once this indicator is devised and enjoys a global consensus (in which case, any redundant indicators would be eliminated).

Step 2. Data

Monitoring relies on the periodic availability of comparable data (health, population, and social) from recognized official institutional sources. Accordingly, it is necessary to critically examine all sources of information (both within and outside the sector) in terms of their usefulness for institutionalizing the proposed monitoring in the country (i.e., mapping of data sources), including an objective assessment of their strengths and limitations.
To monitor inequalities in SDG 3, the BTIHIs should be disaggregated subnationally or by geopolitical criterion (at least the first level of geopolitical disaggregation; ideally, at the level of municipality or district), or else categorized based on individual microdata from surveys.
The equity stratifying indicators should be disaggregated at the same level as the BTIHIs, or by geopolitical criterion, or else categorized based on microdata (that is, at the individual or household level) from surveys.
It is recommended, in principle, to give special importance to the use of socio-geographically disaggregated data,[3] according to their availability and the level of development of national initiatives on core health data. It is also recommended to favor the use of survey data (for example, demographic and health surveys, multiple cluster indicator surveys, and reproductive health surveys), when available and as a means of verifying internal consistency over time of the magnitude, trends, and patterns of distributive inequality in the BTIHIs, depending on how representative they are..

Establishment of quantitative national targets

This proposal suggests the establishment of quantitative national targets as a methodological step for monitoring population health in general, and SDG 3 and its inequalities in particular, since the assessment of progress toward a desired goal gains objectivity when the milestone or expectations are explicitly defined, which in turn strengthens accountability. Indeed, implementation of the 2030 Agenda for Sustainable Development implies the adoption of national targets for each SDG, quantitatively adapting the global targets to local conditions. In order to establish the national targets for the SDG 3 indicators, this proposal applies a simple method based on the average annual percent change in time series. Alternatively, it also considers the use of a complex method based on multivariable regression models. In order to establish the targets for the basic tracer indicators of health inequalities, an algorithm is proposed that measures progressivity between social strata consonant with the established national target. Alternatively, the use of historical patterns of inequality can be considered, as well as multivariable regression models in specific or targeted subnational social groups.

[3] Data that are geographically disaggregated and ordered hierarchically in accordance with one social variable: the equity stratifier. This is an ecosocial approach to monitoring health inequality.

Step 3. Analysis

Step 4. Communication

Methodology to Establish SDG 3 Targets and Reduce Inequalities

SUSTAINABLE DEVELOPMENT GOAL 3:

The methodology proposed below presents a target achievement algorithm designed to simultaneously establish quantitative targets for the national average of an SDG 3 health indicator and to bridge its geographical inequality gaps. This methodology requires disaggregated geographically relevant data from the subnational level (for example, by department, state, province, municipality, canton, district, etc.). The algorithm is based on a criterion of proportional progressive improvement in the speed of change of the indicator in question and generates average targets and inequality gaps that are consistent with each other, which facilitates monitoring and accountability with respect to the commitment to “leave no one behind.”

Summary of Methodology

If information on a health indicator is available at two established times (t₀ and t₁), it is possible to calculate its average annual percent change (AAPC) by applying the following expression:

In general, the AAPC value obtained with expression (1) reflects the speed of change of a health indicator over time. If the indicator has negative polarity (that is, when a lower value of the health indicator over time indicates a more favorable situation, for example, mortality), the AAPC reflects the average annual percent decrease. If, on the contrary, the indicator has positive polarity (that is, when a higher value of the health indicator over time indicates a more favorable situation, for example, care coverage), the AAPC reflects the average annual percent increase.

Presuming that a health indicator’s AAPC is known as is a value of the indicator at an initial time (t_i), the value of the health indicator at a future time (t_f) can be estimated by applying the following expression:

The estimate of a health indicator at a future time can be expressed in relative terms (that is, as a percentage of the initial time value). This can be calculated using the expression for the percent change (PC) of the indicator between the initial time (t_i) and the future time (t_f), as follows:

The absolute gap (AG) and the relative gap (RG) are simple summary metrics of health inequality and correspond to the arithmetic difference and quotient, respectively, in the value of the health indicator between two groups at opposite ends of geographical stratification. Let us say there are four geographical strata for some health indicator. The geographical inequality gap metrics are calculated according to the following expressions:

where q1 is the first stratum (that corresponds to the stratum with the health indicator in a worse situation) and q4 the fourth stratum (that corresponds to the stratum with the health indicator in a better situation). The AG is expressed in the same units of measure as the health indicator. An AG equal to 0 denotes absence of inequality. The RG is expressed without units of measure (that is, its value represents the number of times that the numerator fits into the denominator). An RG of 1 denotes absence of inequality.

Target achievement algorithm

Step 1. Calculate the national AAPC of an SDG 3 indicator based on a known time series between the initial time and a reference year, applying expression (1).

Step 2. Determine the geographical strata:

Order the geographical values of the health indicator for the initial time according to its polarity (that is, from higher to lower if it has negative polarity, or in ascending order if it has positive polarity).
Identify the cut-off points that define the geographical strata, either by pre-established categories (for example, above and below an established national reference value) or by groups (for example, quintiles, quartiles, or terciles).
Calculate the (weighted) average of the health indicator for each geographical stratum thus defined.

Step 3. Apply the criterion of AAPC proportional progressive improvement to each geographical stratum defined:

If the health indicator has negative polarity, assign a proportionately higher AAPC the higher the health indicator of the stratum, guided by the proportionality factor.
If the health indicator has positive polarity, assign a proportionately higher AAPC the lower the health indicator of the stratum, guided by the proportionality
In any case, make sure that the arithmetic average of the AAPC of all the strata equals the national average AAPC at the initial time (applied in step 1).

Step 4. Determine the targets at the subnational and national level:

Calculate the value of the health indicator at a future time for each territorial unit (subnational target) with expression (2).
Calculate the weighted average of the values of the health indicator in all territorial units at a future time (national target).
These results represent the subnational and national targets for the SDG 3 indicator in absolute terms. Expression (3) produces the targets in relative terms.

Step 5. Determine the geographical inequality gap reduction targets:

Calculate the AG and the RG applying expressions (4) and (5) at the initial and future times.
Obtain the percentage changes of the AG and RG for the period by applying expression (3).
These results represent the geographical inequality gap reduction targets for the SDG 3 indicator (distributional targets) in absolute and relative terms, respectively.

Application of the target achievement algorithm

The data presented in this application were used to draft the inequity profile in women's, children’s, and adolescent health prepared by Guatemala.[1] The example here shows the maternal mortality ratio (MMR), which expresses the number of maternal deaths per 100,000 live births in a given year. In accordance with step 1 of the target achievement algorithm, the national AAPC of the MMR was calculated based on MMR data for 2009 (140 maternal deaths per 100,000 live births) and 2014 (113 maternal deaths per 100,000 live births), which corresponds to the national average for those years, replacing these values in expression (1):

The first numerical column of Table 1 shows the values of the MMR in Guatemala in 2014 by department, ordered from higher to lower. In accordance with step 2, based on this ordered geographical distribution of the initial MMR, groups have to be identified in departments or geographical strata to establish the contribution of the subnational levels to achieving the MMR target for 2030.

Table 1. Initial subnational values of the maternal mortality ratio in 2014 in Guatemala and departmental targets for 2030

Notes: The dotted lines separate the four groups of departments, going from highest to lowest maternal mortality ratio in 2014.

MMR: maternal mortality ratio.

In this application, four department groups were established normatively according to MMR in 2014. These four geographical strata remained well-defined on the basis of two criteria: first, a national MMR reference value for 2030 calculated from the national AAPC (‒4.3%) and the initial national MMR (113 maternal deaths per 100,000 live births) applying expression (2); and second, a maximum threshold equivalent to double the SDG 3 global target for this indicator (140 maternal deaths per 100,000 live births). In this way, the four strata group the departments as follows: above the maximum threshold (group 1); below the top stratum but more than double the national reference value (group 2); less than double the national reference value but above the reference value (group 3); and below the national reference value (group 4).

Based on step 3 of the algorithm, each of these four geographical strata were assigned different AAPC intensity, following the proportional progressive improvement criterion: it is believed that the higher the MMR in the department, the faster the reduction (that is, the AAPC) should and could be; while the lower the MMR, the lower the AAPC should be (this is due to the so-called principle of diminishing returns). Table 2 presents the geographical strata determined by the criteria established in this application, as well as the values of AAPC assigned to each of these four groups. Note that the average group AAPC equals the national average AAPC obtained in step 1.

Table 2. Department strata and levels of average annual percent change according to 2014 maternal mortality ratio values

Notes: *The average of these values equals ‒4.3, which is the national AAPC calculated in step 1.

MMR: maternal mortality ratio; AAPC: average annual percent change.

By extension, if we consider the initial MMR levels (2014), the AAPC values for each geographical stratum (Table 2), and apply expression (2), it is possible to obtain the MMR target that each department of the country should reach by 2030, as presented in the second numerical column of Table 1.

Using step 4 of the algorithm, based on the new departmental distribution of MMR for 2030, the national MMR value was calculated, expressed as the average weighted by the population size of live births in every department (projected for 2030). This weighted average represents the national target in absolute terms, which equals 53 maternal deaths per 100,000 live births by 2030. In relative terms, the national target is expressed as a 53% reduction of the national MMR from 2014 to 2030.

Finally, using step 5 of the algorithm, with the MMR values for the initial year and the target year corresponding to strata 1 and 4 (i.e., the extreme strata of the abbreviated distribution), the respective inequality gaps (AG and RG) were calculated using expressions (4) and (5).

Table 3 summarizes the initial values, as well as the joint targets for reducing the national average and the geographical inequality gaps (AG and RG) in the maternal mortality ratio for the country by 2030:

in absolute terms, a national target of 53 maternal deaths per 100,000 live births by 2030 or, in relative terms —and according to expression (3)—, a 53.0% reduction in the MMR between 2014 and 2030;
a target of reducing the MMR absolute gap by 72.0% between 2014 and 2030, and
a target of reducing the MMR relative gap by 47.0% between 2014 and 2030.

Table 3. Maternal mortality ratio in Guatemala: national average and absolute and relative inequality gaps in 2014 and targets for 2030

RMM: maternal mortality ratio; PC: percentage change.

[1] Every Woman, Every Child. Situation of social determinants of health in Guatemala. Panama: APR LAC; 2015. Available from: https://www.everywomaneverychild-lac.org/paises/.

Monitoring and Analysis

Monitoring and Analysis

Monitoring of targets and their inequalities

Analyses of indicators and scenarios

Equity monitoring and analysis initiatives

Monitoring & analysis toolkit

A. Monitoring of targets and their inequalities

Methodological Note on the Monitoring Dashboard for SDG 3 Targets, Indicators, and Inequalities

B. Analyses of indicators and scenarios

Country profiles

Analytical reports and executive syntheses

Production of scientific articles

Scenarios for Achieving the SDG 3 Targets in the Region

C. Equity Monitoring and Analysis Initiatives

Equity Monitoring and Analysis Initiatives

D. Monitoring & Analysis Toolkit

Instruments for the measurement, analysis and monitoring in health

Methodology for Regional and National Monitoring of SDG 3 Targets, Indicators, and Inequalities

Methodology to Establish SDG 3 Targets and Reduce Inequalities