International Training Centre of the ILO 2007 A

© International Training Centre of the ILO 2007 A 1 -02570 Trade Union Training on Occupational Health and Safety and HIV/AIDS (Inter-regional Course: Africa and Caribbean) (Turin, 27 September to 8 October 2010) Monitoring and Evaluation www. itcilo. org International Training Centre of the ILO 1

Objective of the Session To provide information that will enable participants to monitor and evaluate HIV/AIDS programmes in the workplace. l To identify indicators that will help participants in monitoring and evaluating HIV/AIDS programmes in the workplace. © International Training Centre of the ILO 2007 l www. itcilo. org International Training Centre of the ILO 2

l Monitoring is the systematic and continuous assessment of a programme over a period of time. l On-going monitoring allows us to see how and where the programme must change. l Monitoring entails routine tracking of the key elements of programme/project performance, usually inputs and outputs, through record-keeping, regular reporting and surveillance systems as well as health facility observation and client surveys. www. itcilo. org International Training Centre of the ILO 3 © International Training Centre of the ILO 2007 What is Monitoring?

l Evaluation is the episodic assessment of the change in targeted results that can be attributed to the programme or project/project intervention. l Evaluation attempts to link a particular output or outcome directly to an intervention after a period of time has passed. l Evaluation entails measuring the impact of an intervention programme to see if it has an effect or not. l Evaluation is usually conducted after the programme has been in existence for some time. www. itcilo. org International Training Centre of the ILO 4 © International Training Centre of the ILO 2007 What is Evaluation?

l Ascertain the appropriateness and effectiveness of the programme. l Assess how well the programme is running. l Assess the effect of the programme on the workplace. l Understand the effectiveness of the programme in changing people’s behaviour with regards to HIV/AIDS and STIs. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 Why are monitoring and evaluation important? 5

Two aspects to be monitored and evaluated l The impact of HIV/AIDS in the workplace. l The effectiveness and impact of the HIV/AIDS and STD programme. Monitor Baseline Information Evaluate Report www. itcilo. org © International Training Centre of the ILO 2007 HIV/AIDS Programme Feedback International Training Centre of the ILO 6

l Has the programme changed behaviour, reduced the prevalence of HIV/AIDS and STIs and minimized or eradicated discrimination against people living with HIV/AIDS? l What is the prevalence of HIV infection among workers? l How appropriate and effective is the HIV/AIDS workplace programme? l What is the efficiency and progress of the HIV/AIDS workplace programme? www. itcilo. org International Training Centre of the ILO 7 © International Training Centre of the ILO 2007 Questions to address in monitoring and evaluation

What is an Indicator? l An indicator is a direct or indirect measure of change. l Behaviour change with regard to safer sex practices is the change that is of ultimate interest in an HIV/AIDS and STI programme. l © International Training Centre of the ILO 2007 Indirect indicators: l Prevalence of STIs and the number of condoms distributed: indicate behaviour change with regard to safer sex practices. Number of employees volunteering to become peer educators: indicate levels of awareness and motivation around HIV/AIDS and STIs in the workplace and is an indirect indicator of behaviour change. www. itcilo. org International Training Centre of the ILO 8

What is an Indicator? l Number of condoms distributed: indicate the effectiveness of an education programme on the use of condoms. l Number of employees attending and participating in HIV/AIDS and STI education programme: indicate whether the programme is reaching those targeted. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 Direct indicator 9

l Establishment of HIV/AIDS working committee and/or structures. l Adoption of an HIV/AIDS workplace policy. l Number of workers who are aware of the policy. l Number of trained peer educator/counsellors. l Number of workshops conducted by peer educators. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 Examples on Indicators 10

l Number of workshops conducted by peer educators. l Number of support groups in the workplace. l Number of workers participating in support groups. l Number of HIV/AIDS-related deaths and burials. www. itcilo. org International Training Centre of the ILO 11 © International Training Centre of the ILO 2007 Examples on Indicators

l Absenteeism due to HIV/AIDS. l People’s attitudes/perceptions/behaviours. l Number of claims for occupational exposure to HIV. l Number of medical claims for HIV-related illnesses. l Number of times the Management talks about HIV/AIDS as a workplace issue. www. itcilo. org International Training Centre of the ILO 12 © International Training Centre of the ILO 2007 Examples on Indicators

Examples on Indicators Number of collective bargaining agreements with progressive clauses that protect the rights of people living with HIV. l Changes in the budget for HIV/AIDS. l What else? © International Training Centre of the ILO 2007 l www. itcilo. org International Training Centre of the ILO 13

Key aspects to identify indicators Valid – they should measure the condition or event they are intended to measure. l Reliable – they should produce the same results when used more than once to measure the same condition or event. l Specific – they should measure only the condition or event they are intended to measure. l Sensitive – they should reflect changes in the state of the condition or event under observation. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 l 14

l Operational – it should be possible to measure or quantify them with developed and tested definitions and reference standards. l Affordable – the costs of measuring the indicators also should be reasonable. l Feasible – it should be possible to carry out the proposed data collection. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 Key aspects to identify indicators 15

Monitoring the impact of HIV/AIDS in the workplace Maintaining records on sickness or absence and looking at the causes of the absence. l How often employees take special or compassionate leave. l Staff turn-over by reason for departure. l Death by cause. l Decide which indicators you will measure (how will behaviour change be measured? ). www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 l 16

l Decide how the indicators will be measured e. g. in-depth interviews or use of questionnaires. l Calculate the costs of the monitoring process. l How often will monitoring take place to give the maximum amount of information with the least disruption and costs? www. itcilo. org International Training Centre of the ILO 17 © International Training Centre of the ILO 2007 Monitoring the impact of HIV/AIDS in the workplace

l Identify programme goals and objectives. l Examine existing data and past evaluation studies (baseline). l Identify internal and external evaluation resources and capacity. l Determine evaluation questions, their feasibility and appropriate designs and indicators. l Plan for disseminating and using evaluation findings. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 Developing an integrated and comprehensive evaluation plan 18

Examples of process measures Participants: Number, demographic information, frequency of attendance, turn-over rate, etc… l Services that are provided: Classes, programmes, workshops, publications, other services, communications (number, type, content and frequency). © International Training Centre of the ILO 2007 l www. itcilo. org International Training Centre of the ILO 19

l Changes in policies – new or modified policies. l Changes in collective bargaining agreements. l Changes in practices – new or modified practice. l Changes in programmes – new or modified service programmes. l Changes in behaviours – new or modified behaviours. www. itcilo. org International Training Centre of the ILO © International Training Centre of the ILO 2007 Examples of outcome measures 20

Main Outcomes Expected Accountability Build Capacities Revise programmes as needed © International Training Centre of the ILO 2007 Learning from experiences Make more informed decisions www. itcilo. org International Training Centre of the ILO 21

l Impact of the HIV/AIDS and STI epidemic in the workplace must be monitored. l Effectiveness and impact of the HIV/AIDS and STI programme on the epidemic must be monitored and evaluated. l Baseline information on key indicators must be collected during the need analysis component of the policy and programme development process. l Subsequent indicator information can then be compared with the baseline information to assess the impact of the epidemic and the prevention programme. www. itcilo. org International Training Centre of the ILO 22 © International Training Centre of the ILO 2007 Summary