Consultant for Final Independent Evaluation - Industries and Workplaces for sustainable and inclusive growth through Tripartite Dialogue (InSIGHT), sharing good practices of Greener Business Asia (GBA), Occupational Safety (OSH) and Industrial Relations (IR) project

Terms of Reference

Final Independent Evaluation

Industries and Workplaces for sustainable and inclusive growth through Tripartite Dialogue (InSIGHT), sharing good practices of Greener Business Asia (GBA), Occupational Safety (OSH) and Industrial Relations (IR) project.

Project Code: RAS/14/05/JPN

Administrative unit: Regional Office for Asia and the Pacific (ROAP)

Geographical coverage: Indonesia and ASEAN Countries

Technical Unit: Green Jobs

Timing and Type of evaluation: Final Independent

Closing Date of Application 19 July 2018

1. The project is funded by the Ministry of Labour, Health and Welfare of Japan with the total budget of US$ 1,760,511. It is aimed to strengthen policy dialogue and institutional capacities to promote sustainable and inclusive growth in industries and workplaces in Asian region. It is also aimed to enhance local tripartite structure and support systems to foster sustainability, competitiveness and socially responsible business practices at sector and workplace levels.

2. The project focuses on regional level interventions and full-fledged intervention in Indonesia. The project duration is from April 2015 to June2018. In 2017, the project conducted a self-evaluation.

3. The final evaluation is due as per the ILO evaluation policy guidelines for result-based evaluation which states that all projects over US$ 1 million and/or lasting more than 30 months must undergo at least one independent evaluations. The evaluation will be managed by evaluation manager who has no prior involvement in the project and will be conducted by an independent evaluation consultant. The independent final evaluation will be guided by the ILO Evaluation Policy (2017) and will adhere to UNEG Norms and Standard (2016).

3. The introduction of the Sustainable Development Goals (SDG) combined with the anticipated growth and consolidation of the ASEAN region, means that engaging the ILO’s tripartite partners in relevant regional discourse, agreements and actions, is key. The project is expected to foster a broad-based understanding of the implications and challenges created by economic growth, the environment and the need for social inclusion, in the context of decent work.

4. The project work towards initiating the process of adapting jobs and workplace practices to greener, resource productive, socially responsible & ‘future-oriented’ ways of doing business, while mainstreaming social dialogue and harmonious Industrial Relations. This approach provides mutual benefits to workers and employers, and aligns with industry development efforts to stay globally competitive.

5. The project also fosters a broad-based understanding of the implications and challenges created by economic growth, the environment and the need for social inclusion, in the context of decent work. The Project works at both regional and national level, with activities that encompass the ASEAN region, initiating national-level interventions in Indonesia, as it also prepares to replicate experiences in other countries. Starting up in Indonesia is a strategic way to contribute to impacts relating to ASEAN, considering that Indonesia accounts for about 50% of the labor force in ASEAN region.

6. The project has taken forward from the lessons learnt and good practices from the previous ILO Japan funded projects, Greener Business Asia phase I and II (in Thailand and the Philippines), and also the ASEAN-ILO/Japan Project on Industrial Relations (AIJPIR), and the ILO/Japan Occupational Safety and Health in Hazardous Work in Southeast Asia (OSH) Project. The project development objective is to contribute to promoting industries and workplaces that drive sustainable and inclusive growth in the ASEAN region. The project has 2 immediate objectives.

Immediate objective 1: To strengthen policy dialogue and institutional capacities for promoting workplaces and industries that contribute to sustainable and inclusive growth. This covers conducting forums and technical workshops for government and social partners around the promotion of sustainable and responsible business practices, green jobs, sustainable enterprises, addressing concerns relevant to ASEAN region.

Immediate objective 2: Local tripartite structures and support systems form fostering sustainability and competitiveness of sectors and workplaces are strengthened

This objective is to demonstrate how principles of decent work, green jobs, skills and sound industrial relations can translate into concrete industry and workplace level actions, the project supports capacity building approaches and action-learning initiatives for the tripartite partners and key stakeholders.

7. At the regional level, the project contributes to Regional outcome RAS176 - Increased Member States' capacity to develop coherent policies and programmes on green employment in climate and environment challenged societies, and to RAS126 - Increased knowledge and capacity in the region to promote coherent policies in support of decent work for all women and men. The project also contributes to RAS 101 – Mainstreaming Decent Work in Regional Integration and Partnerships in Asia and the Pacific.

8. The project also contributes ILO Programme and Budget (P&B) Outcome 3 (Sustainable enterprises create productive and decent jobs) and Outcome 12 (Tripartism and strengthened labour market governance contribute to effective social dialogue and sound industrial relations). In relation to Indonesia’s Decent Work Country Programme and Country Programme Outcomes (CPO), the project contributes to the following CPOs:

‘Employment creation for inclusive and sustainable growth’. It will support IDN129 ‘improved policies and programmes on entrepreneurship, business and cooperative development for job creation including financial inclusion;

‘Sound industrial relations in the context of effective employment governance’. It will support IDN152 ’tripartite constituents effectively engage in social dialogue to apply labour regulations and international labour standards’; and

‘Strong and representatives employers’ and workers’ organisations influencing economic, social and governance policies’. It will support: IDN801 ‘Strengthened institutional capacity of employers' organisations; IDN802 ‘Strengthened institutional capacity of workers’ organisations;

IDN803 ‘Increased capacities of employers’ and workers’ organisations to participate effectively in the development of social and labour policies

9. Further, it contributes to engaging the ILO’s tripartite partners in relevant global discussions and initiatives and in the region, which includes the commitments to the Sustainable Development Goals (SDG), the ASEAN Economic Community (AEC) integration, and understanding the context of the ‘future of work’, Industry 4.0 and its impacts on jobs, skills and employment relations.

10. The project Chief Technical Adviser who is based in Bangkok, oversee the overall coordination of the project. A project technical Officer who is based in Jakarta is responsible for providing technical and coordinating project activities in Indonesia. Technical support is provided by ILO Decent Work Technical support team in Bangkok.

11. Main partners are ASEAN, Ministry of Manpower (MoM) and social partners (K-SPSI: confederation of All Indonesian Trade Union, and APINDO:The Employers’Association of Indonesia) in Indonesia, Ministry of Environment, Global Green Growth Institute- Indonesia, BAPPENAS, Indonesia Business Council for Sustainable Develoment (IBCSD), Kamar Dagand Dan Industri Indonesia (KADIN), Ministry of National Development Planning (BAPPENAS)

12. In 2017, a self-evaluation was conducted and the main findings are

The thematic area and approach/strategy is relevant for the current situation in Indonesia. Industrial Relations is a key issue that the tripartite constituents will always regard as important. However, it had to likewise develop and update its content so that it will be aligned to the current times.

The ‘Green’ issue is a national agenda for Indonesia, especially for the current administration. However, the Ministry of Manpower has yet to acknowledge its importance in their mandate. With regards to anticipating labor market & industry impacts, which will then need solutions, which could include skills dimension, could be the link. With regards to Trade Unions, ‘green’ is not on their priority now, as they have yet to realize that it will have implications to the core advocacies that they are carrying forward like fair wages, standardizes employment contracts, etc.

In order to gain more traction in its work, the project should continue to expand their partners, and in doing so, enhances the exposure and capacity of the tripartite partners to contribute meaningfully.

With regards to opportunities, InSIGHT should maximize its engagement with ASEAN and using it as a platform not only for knowledge sharing, but likewise for pursuing concrete collaboration with the respective countries. With regards to working in Indonesia, while manufacturing sector is very crucial to the growth of the economy, the emerging sectors which can also provide substantial impact when provided support includes services related sectors like tourism, transportation & logistics, and the digital/knowledge-based sectors.

13. To date, most of the project outputs have been delivered (Technical progress report Dec 2017).

14. The main purpose of this final evaluation are two folds. First it is aimed to promoted accountability to the tripartite constituents, the donor and key stakeholders. Secondly, it is aimed to promote learning within ILO and among key stakeholders. The evaluation findings will be used to inform similar interventions in the future.

15. The specific objectives of the evaluation are to:

Assess the project implementation effectiveness including the progress in achieving its intermediate objectives and results (including intended and unintended, positive and negative results), and effectiveness of management arrangements;

Assess the project implementation efficiency;

Assess the impact and sustainability and Identify factors (including challenges, opportunities)that enable the sustainability, particularly of the national stakeholders in Indonesia;

Provide recommendations and describe lessons learned and good practices that could be replicated.

16. Scope: the evaluation will encompass all activities and components of the project during the whole duration from April 2015 to June 2018. The evaluation will focus at the country level in Indonesia and the work at the regional level. The evaluation will address overall issues of relevance and strategic fit of the programme, the achievement and effectiveness, adequacy and efficiency in the use of resource, as well as effectiveness of the programme management arrangements, impact and sustainability. The evaluation will also examine the programme’s performance in relation to the ILO’s cross-cutting issues on gender, non-discrimination, social dialogue, and environment.

17. Client: the primary clients are the ILO constituents at the regional and in Indonesia, ILO ROAP, the donor (Japan), ILO Decent Work Technical team in Bangkok, and ILO Jakarta. The secondary clients are ILO technical unit at HQ and other key stakeholders.

18. The evaluation should address the following ILO evaluation criteria: intervention progress and effectiveness; efficiency of resource use; effectiveness of management arrangements; and impact orientation and sustainability of the intervention as defined in the ILO Policy Guidelines for results-based evaluation, 2017 (Annex 1).

19. The core ILO cross-cutting priorities, such as gender equality and non-discrimination, promotion of international labour standards, tripartite processes, and constituent capacity development should be considered in this evaluation. In particular, gender dimension will be considered as a cross-cutting concern throughout the methodology, deliverables and final report of the evaluation. To the extent possible, data collection and analysis should be disaggregated by sex as described in the ILO Evaluation Policy Guidelines and relevant Guidance Notes (Annex 1).

20. It is expected that the evaluation address all of the questions detailed below to the extent possible. The evaluator may adapt the evaluation criteria and questions, but any fundamental changes should be agreed upon between the ILO evaluation manager and the evaluator.

21. Suggested evaluation criteria and evaluation questions are summarised below: -

1) Relevance and strategic fit

The extent to which the project has addressed the need of the tripartite constituents in ASEAN and in Indonesia on the industrial relations, social dialogue and green growth.

Assess the constituents’ perception of the ILO and the support it provideds vis-à-vis other development partners, including the reasons they chose to work in certain thematic areas with the ILO over other areas (?).

What is the current areas of interest of the constituents vis-à-vis project original theme – whether there have deviated from the original design. What could have contributed to changes, if any? To what extent the project has adapted to those changes?

2) Effectiveness (including effectiveness of management arrangements)

To what extent has the project achieved its planned results (including intended and unintended, positive and negative)? and If not, why?

i. How effective were the ILO-ASEAN policy component on IR, skills on green jobs regional seminar/forums? To what extent it contribute to institutional capacity of tripartite constituents

ii. How effective was the supporting activities at the enterprise level?

Assess/identify what appropriate methods for capacity development that work well with different constituents in Indonesia (methods preferred by the Employers and Trade Unions)

How effectively has the project interventions integrating gender?

To what extent do project management capacities and arrangements put in place support the achievement of the planned results?

To what extent have stakeholders, particularly workers’ and employers’ organizations been involved in project implementation?

3) Efficiency of resource use

To what extent has the project delivered value for money? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically and efficiently to achieve expected results? Could they have been allocated more effectively and if so, how?

4) Impact orientation and Sustainability

What strategies has the project put in place to ensure continuation of actions initiated after the project support ends?

To what extent the project outputs/benefits will be sustained

22. The evaluation will comply with evaluation norms, standards and follow ethical safeguards, as specified in the ILO’s evaluation procedures. The ILO adheres to the United Nations system of evaluation norms and standards as well as to the OECD/DAC Evaluation Quality Standards.

23. Both qualitative and quantitative evaluation approaches should be considered for this evaluation. The evaluation fieldwork will be qualitative and participatory in nature. Qualitative information will be obtained through field visits, interviews and focus groups as appropriate. Opinions coming from stakeholders will improve and clarify the quantitative data obtained from project documents. The participatory nature of the evaluation will contribute to the sense of ownership among stakeholders. Quantitative data will be drawn from project documents including the Technical Progress Reports (TPRs). Sound and appropriate data analysis methods should be developed for each evaluation question. Different evaluation questions may be combined in one tool/method for specific targeted groups as appropriate. Attempts should be made to collect data from different sources by different methods for each evaluation question and findings be triangulated to draw valid and reliable conclusions. Data shall be disaggregated by sex where possible and appropriate.

24. A detailed methodology will be elaborated by an independent evaluator on the basis of this ToR. The detailed methodology should include key and sub-question(s), detailed methods, data collection instruments and data analysis plans to be presented as a key element in the inception report.

25. The methodology for collection of evidences should be implemented in three phases (1) an inception phase based on a review of existing documents to produce inception report; (2) a fieldwork phase to collect and analyze primary data; and (3) a data analysis and reporting phase to produce the final evaluation report.

26. The gender dimension should be considered as a cross-cutting concern throughout the methodology, deliverables and final report of the evaluation.

27. The evaluator will provide the following deliverables and tasks:

Deliverable 1: Inception report. The inception report will include among other elements the evaluation questions and data collection methodologies and techniques, and the evaluation tools (interview guides, questionnaires, etc.). The instrument needs to make provision for the triangulation of data where possible. The evaluators will prepare an inception report as per the ILO Checklist 3: Writing the inception report (Annex 1).

Deliverable 2: Stakeholder workshop. The evaluators will conduct a stakeholder workshop in Jakarta to validate information and data collected through various methods and to share the preliminary findings with the ILO and local stakeholders at the end of evaluation mission. The stakeholder workshop will be organized by the project team. Evaluation findings should be based on facts, evidence and data. This precludes relying exclusively upon anecdotes, hearsay and unverified opinions. Findings should be specific, concise and supported by triangulation of quantitative and qualitative information derived from various sources to ensure reliability, validity and generalizability.

Deliverable 3: First draft evaluation report. Evaluation report should include action-oriented, practical and specific recommendations assigning or designating audiences/implementers/users. The draft evaluation report should be prepared as per the ILO Checklist 5: Preparing the Evaluation Report which will be provided to the evaluator. The first draft evaluation report will be improved by incorporating evaluation manager’s comments and inputs.

Deliverable 4: Revised and Final evaluation report with evaluation summary (standard ILO format). The evaluator will incorporate comments received from ILO and other key stakeholders into the final report. The report should be finalized as per the ILO Checklist 5: Preparing the Evaluation Report which will be provided to the evaluator. The quality of the report and evaluation summary will be assessed against the ILO Checklists 5, 6, 7, and 8 which will be provided to the evaluator.

28. The reports and all other outputs of the evaluation must be produced in English. All draft and final reports including other supporting documents, analytical reports and raw data should be provided in electronic version compatible with WORD for windows. Ownership of the data from the evaluation rests jointly between ILO and ILO consultant. The copy rights of the evaluation report rests exclusively with the ILO. Key stakeholders can make appropriate use of the evaluation report in line with the original purpose and with appropriate acknowledgement.

29. Ms. Pamornrat Pringsulaka, Regional M&E Officer, is designated to manage this independent evaluation with oversight provided by the ILO Evaluation Office. A qualified independent consultant will be commissioned to conduct this evaluation. The evaluation will be funded from the project budget. A list of tasks of the evaluation manager is following:

Draft and finalize the evaluation ToR upon receiving inputs from key stakeholders;

Review CVs and proposals of the proposed evaluators;

Liaise with the project team and provide project background documents to the evaluator;

Coordinate with the project team on the field visit agenda of the evaluator;

Brief the evaluation consultant on ILO evaluation procedures;

Circulate the report to all concerned for their comments;

Review and provide comments of the draft evaluation report; and

Consolidate comments and send them back to the evaluators.

30. The project team will handle administrative contractual arrangements with the evaluator and provide any logistical and other assistance as required. They will be responsible for the following tasks:

Provide project background materials to the evaluator;

Prepare a list of recommended interviewees;

Schedule meetings for field visit and coordinate in-country logistical arrangements;

Be interviewed and provide inputs as requested by the evaluator during the evaluation process;

Review and provide comments on the draft evaluation reports;

Organize and participate in the stakeholder workshops; and

Provide logistical and administrative support to the evaluator, including travel arrangements and all materials needed to provide all deliverables.

31. The evaluator reports to the evaluation manager. The evaluator will be selected through a competitive process from qualified independent consultants. The consultant will lead the evaluation and will be responsible for delivering the above evaluation deliverables using a combination of methods as mentioned above.

32. Indicative time frame and responsibilities

No.	Task	Responsible person	Time frame
1	Preparation, sharing and finalization of the TOR, and ToR approved	Evaluation Manager/Regional M&E Officer	By 12 July 2018
2	Issuance of EOI, advertisement of consultant, and selection of consultant	Evaluation Manager/ Regional M&E Officer	By 12 July 2018
3	Issuance of contracts	Project	By 30 July 2018
4	Draft mission itinerary for the evaluator and the list of key stakeholders to be interviewed	Project	BY 27 July 2018
5	Brief evaluators on ILO evaluation policy and the project (Evaluator to start desk study, Skype calls with project manager, ILO Japan CTA)	Evaluation Manager and Project Manager	By 1 August 2018
7	Document review and development of the inception report submitted to Evaluation Manager	Evaluator	3 August 2018
8	Approve Inception report	Evaluation Manager	7 August 2018
9	Conduct Evaluation Mission and Stakeholder Workshop	Evaluator	8-16 August 2018
10	Draft report submitted to Evaluation Manager	Evaluator	31 August 2018
11	Share the draft report with all concerned stakeholders for comments for one week	Evaluation Manager	3-14 September 2018
12	Consolidate comments into the draft report and send to the evaluator	Evaluation Manager	17 September 2018
13	Revised report Submit to Evaluation Manager	Evaluator	20 September 2018
14	Submission of the evaluation report to ILO Evaluation Office for approval¹	Evaluation manager and ILO Evaluation Office	24 September 2018

33. An independent consultant with the following relevant, skills, experience and qualifications are being sought

No previous involvement in the delivery of the InSIGHT project;

University Degree with minimum eight years' experience at the national level and/or five years’ experience at the international level in industrial relations, and/or social dialogue, green business/enterprise development;

Minimum five years of experience in conducting programme or project evaluations;

Knowledge of, and experience in applying, qualitative and quantitative research methodologies;

Knowledge of ILO’s roles and mandate and its tripartite structure as well as UN evaluation norms and its programming;

Excellent analytical skills and communication skills;

Demonstrated excellent report writing skills in English;

Working experience in Indonesia and/or ASEAN

34. It is foreseen that the duration of this evaluation will between 27 July to 30 September 2018. The field mission in Indonesia is expected during 8-16 August 2018. Below are indicative working days required and tasks to be completed. Numbers of days foreseen for the consultant in one task can be reallocated to another task where justified and in consultation with the evaluation manager.

Tasks	Number of working days
Desk review of project related documents; Skype briefing with evaluation manager, project manager, donor, DWT-Bangkok, and ILO HQ; Prepare inception report.	5 days
Conduct Field visits and interviews relevant project staff, stakeholders, and beneficiaries; conduct a national stakeholder workshop.	7 days
Analysis of data based on desk review, field visit, interviews/questionnaires with stakeholders; draft report	7 days
Revise and Finalize the report including explanations on why comments were not included.	2 days
Total	21 days

35. The evaluation will comply with UN Norms and Standards. The ToR is accompanied by the code of conduct for carrying out the evaluation. UNEG ethical guidelines will be followed. It is important that the evaluator has no links to project management or any other conflict of interest that would interfere with the independence of evaluation².

ILO Policy Guidelines for results-based evaluation, 2012 /eval/Evaluationguidance/WCMS_176814/lang--en/index.htm

Code of conduct form (To be signed by the evaluators)

/eval/Evaluationguidance/WCMS_206205/lang--en/index.htm

Checklist No. 3 Writing the inception report

/eval/Evaluationguidance/WCMS_165972/lang--en/index.htm

Checklist 5 preparing the evaluation report

/eval/Evaluationguidance/WCMS_165967/lang--en/index.htm

Checklist 6 rating the quality of evaluation report

/eval/Evaluationguidance/WCMS_165968/lang--en/index.htm

Template for lessons learnt and Emerging Good Practices

/eval/Evaluationguidance/WCMS_206158/lang--en/index.htm

/eval/Evaluationguidance/WCMS_206159/lang--en/index.htm