Publications : ILO Publications : Placement of Job-Seekers with Disabilities :

Placement of Job-Seekers with Disabilities
4. Monitoring and evaluation

A comprehensive employment service for people with disabilities involves many different levels and component parts, as illustrated in Diagram 1 (p. 41). Effective management of this service requires a monitoring and evaluation system to help ensure that:

• the service is delivered in line with objectives
• planned service outcomes become actual outcomes.

Monitoring involves the systematic collection of information about the service to check its operation and observe its progress over time.

Evaluation involves making judgements about the service, based on comprehensive information on current performance. Comparisons may be made with past performance or with targets set.

Monitoring and evaluation are increasingly part of management practice in publicly funded activities. They are useful in:

• assessing effectiveness (Is the service working?)
• identifying strengths and weaknesses (What needs to be improved, what needs to be changed?)
• assessing efficiency (Is it 'value for money'?).

Information obtained in this way is a valuable tool for managers and policy-makers in the placement service, enabling them to have a regular overview of service performance, to identify any problem areas which need attention, and to take timely corrective action.

Monitoring and evaluation are particularly important in managing a service which includes so many different component parts, since the performance of one section may have an impact on the performance of others. The information generated can make this link apparent, identify the source of the problem, and provide the basis for a solution.

The goals and objectives of the employment service provide the framework for the monitoring and evaluation system. In developing this system, several steps are involved:

• The stated objectives of the service are translated into measurable terms.
• Indicators of performance which fully reflect service delivery and outcomes are identified and agreed with relevant stakeholders.
• Information is gathered systematically, often over a period of time, from service providers and users.
• The information is analyzed.
• Comparisons are made.
• Judgements are formed.
• The findings are presented.
• Recommendations for change are made, if necessary.

Measureable objectives

Some employment service objectives are stated in terms which are easy to measure. Examples are objectives

• to place disabled job-seekers to employment
• to assist employers in filling vacancies.

Other objectives are less amenable to measurement - for example, where a service objective is:

• to promote awareness among employers of the working ability and capacity of disabled people
• to help job-seekers to find jobs which best suit their abilities, by providing vocational guidance and assessment
• to enable job-seekers to find jobs by providing training in jobseeking skills and access to related facilities.

Such broad, intangible objectives need to be translated into terms which will allow their achievement to be measured, either through statistics (quantitatively) or description (qualitatively), before the monitoring and evaluation can start.

Performance indicators

Identifying indicators which will mirror how well the employment service meets its objectives is a crucial step in developing the monitoring and evaluation system.

Where the objectives are easy to measure quantitatively, it may be relatively easy to identify and agree the relevant indicators. Examples are:

• the number of job-seekers with different disabilities registered with the service
• the number of employers contacted
• the number of job vacancies notified
• the number of placements obtained
• cost per placement.

In developing these indicators, care should be taken to ensure that they reflect the work involved. For example, if a person has a severe disability, more time and effort may be required to secure a placement than in other cases. If statistics are based on a simple 'head count', this extra work will not be reflected. But if the statistics are weighted to reflect the severity of disability or the amount of service time required, a more accurate picture of service performance will be obtained.

Where the objectives are less tangible, a more creative approach is required to develop performance indicators. In addition to statistics, it may be decided to:

• create an indicator of client satisfaction by carrying out regular surveys
• make a video, illustrating aspects of the service
• prepare a photographic exhibition.

Once the performance indicators have been identified, it is important that all relevant decision-makers and managers in the employment service agree that they provide a comprehensive picture of the work carried out and services provided.

Gathering information

The method of gathering the information required for monitoring and evaluation should:

• be as simple as possible
• take as little time as possible
• not interfere with service delivery.

Training should be provided for placement officers to ensure that there is clarity on the information required and the approach to be used.

Making judgements

The monitoring and evaluation information should be regularly compiled and analyzed to form the basis of judgements about service performance over time or in comparison with targets which have been set. It is important to do this regularly so that the findings and judgements do not come as a surprise.

Communicating the findings

Generally, evaluation results are presented as a report. If this is lengthy, it is useful to prepare a shorter summary for wider circulation.

In addition to the written report, evaluation findings may be presented through:

• seminars and workshops targeted at policy-makers
• a photographic exhibition
• video.

An external assessor may be appointed to conduct an evaluation. This may be the case when a funder wishes to verify that the service is being provided according to the agreed format.

Alternatively, an outsider may be called in if the managers or staff do not have the requisite skills.

But increasingly, the tasks of monitoring and evaluating are part of the ongoing work of staff and managers of the service or activity. This approach is often preferred, as it ensures that the service providers are actively involved, and are not apprehensive about the evaluation.

Outcomes of the monitoring and evaluation are useful in:

• establishing whether the service is having the desired impact
• informing decisions to expand or cut back
• identifying ways the service can be improved
• justifying funding requests
• informing policy-makers of any changes which may be required in policy, to promote service effectiveness
• identifying the need for changes in existing law or for the introduction of new laws.

While the outcomes may be used to justify the continuation of the service, their main value lies in helping to improve service delivery and outcomes.

4. Monitoring and evaluation
Placement of Job-Seekers with Disabilities

top of the page