The impact of Training Policies in Latin America and the Caribbean: 
The case of "Programa Joven"*

Cristián Aedo and Sergio Nuñez**
Graduate Program in Economics
ILADES/Georgetown University
May 2001

 

Complete document: pdf format

Abstract

This research evaluates the "Programa Joven", a training program conducted by the Ministerio del Trabajo of Argentina. We adapt and apply a non-experimental evaluation methodology to answer the following questions: Does "Programa Joven" increase the labor income of the trainees? Does "Programa Joven" increase the probability of being employed? And (3) what is the rate of return to dollars spent on the "Programa Joven"? We used Propensity Scores Matching Estimators as our basic methodology to obtain a measure of the impact of the training program. Our choice of this methodological approach was based upon both the theoretical developments in the area of Program Evaluation and the availability of relevant information. We used three different set of data to estimate the Propensity Scores which allowed us to analyze the question on how sensitive Program impact estimates are to different propensity score specifications? This question has not been addressed by the previous literature.

Our results indicate first, that Program impact on earnings were statistically significant for young males and adult females. This result was not sensitive to the number of nearest neighbors. Second, the estimated Program impact on employment was statistically significant for adult females only. Again the result was not sensitive to the number of nearest neighbors. Third, impact estimates on earnings and employment for the groups with statistically significant results were not sensitive to the different sources of information used to estimate the propensity scores. This was a surprising result as we expected to observe greater variability in the impact results across different propensity score specifications. Fourth, the cost-benefit exercise conducted suggest that we required at least 9 years of duration of the earnings impact for the Program to have a positive net present value for the groups with statistically significant results.

Index

1. Introduction

2. Description of "Programa Joven"

3. The Data: Description of the Different Information Available for the Evaluation

4. Program Participation:

4.1 Determinants

4.2 Strategic Behavior

5. Estimation of Program Participation (Propensity Scores):>

5.1 Universe (PSTOT)

5.2 Universe and Sample (PSUN)

5.3 Sample (PSMU)

5.4 A comparison of the propensity score estimates

5.5 Common Support

6. Impact Estimates: labor earnings and employment

6.1 Labor Earnings

6.2 Employment

7. Cost-benefit analysis:

7.1 Cost Data Description

7.2 Simulations

7.3 Results

8. Conclusions

Appendix

A1. Variables Description

A2. Common Support

A6. MATLAB Codes.

a. Average Nearest Neighbor Estimators.

b. Bootstrapping

 

1. Introduction

Latin-American countries invest a significant amount of resources in training programs. An evaluation of these experiences is needed in order to learn from them and to design more effective programs. However, Program evaluation faces many difficulties (Heckman, LaLonde and Smith, 1999): first, due to the heterogeneity of impacts that Programs produce there are many parameters of interest in their evaluation. Second, there is no unique way of conducting Program evaluations. The choice of an appropriate Program impact estimate depends upon the question to be answered and data availability. Third, to produce "good" evaluations it is needed to have "good data". Usually econometric methods need to be used to correct for data problems. Fourth, to obtain Program impact estimates it is necessary to compare comparable individuals, which increases the complexities of Program evaluation. Thus, it is necessary to reduce the biases by comparing comparable individuals, by administering similar questionnaires to participant and non-participants, by using similar time frameworks, and by drawing the samples of participants and non-participants from similar labor markets. Fifth, nonexperimental Program impact estimates solve the selection problem under different assumptions, which generates variability in their results. An experimental evaluation provides an important reference framework to analyze the performance of alternative non-experimental evaluation methodologies. Sixth, social Programs at the national or regional levels have an impact on both participants and non-participants. The usual approach to deal with this Program "contamination" is to assume that the impact on nonparticipants is not significant.

This research evaluates the "Programa Joven", a training program conducted by the Ministerio del Trabajo of Argentina. We adapt and apply a non-experimental evaluation methodology to answer the following questions: (1) Does "Programa Joven" increase the labor income of the trainees? (2) Does "Programa Joven" increase the probability of being employed? And (3) what is the rate of return to dollars spent on the "Programa Joven"?

To answer these questions we used the Matching Estimators approach as our basic methodology. This choice was based upon both the theoretical developments in the area of Program Evaluation (Heckman et al., 1995, 1997, 1998 & 1998) and the availability and quality of relevant information. As described in Tood (1999) the application of this methodology requires two steps: first, the estimation of a model of program participation (Propensity Scores) and second, and conditional on the estimated propensity scores, the usage of matching estimators to obtain the impact of the Program.

To estimate the propensity scores we used three sources of information: first, the data for all the individuals who registered and qualified to take training programs in the period March 1996 to December 1997 (approximately 140,000 individuals). Second, the information contained in a sample of beneficiaries and controls used by the Ministerio del Trabajo of Argentina to evaluate the Program (3,340 individuals in total).1 And third, the information contained in the first database but restricted to the 3,340 individuals contained in the second database. The access to these different sources of information allowed us to analyze the additional question on how sensitive Program impact estimates are to different propensity score specifications? This question has not been addressed by the previous literature and we face it here. Our hypothesis is that impact estimates are sensitive to different propensity score specifications.

We report and compare the Propensity Scores estimated from each one of these data sources and we estimate the program impact on earnings and employment based upon these propensity scores.

Finally, based upon cost information and the program impact estimates (benefits), we applied a cost benefit analysis of the Programa Joven. This analysis was conducted under different scenarios with regard to benefit duration, discount rate and the ratio of indirect to direct cost.

 

  ___________________________

* Financial support from the Interamerican Development Bank is gratefully acknowledged. This paper has been presented at the IADB research network seminars in Washington D.C. and Rio de Janeiro. I thank participants for helpful comments. I am particularly grateful to Jeffrey Smith, Petra Todd, James Heckman, and Gustavo Marquez for many helpful comments and suggestions. All remaining errors and omissions are my own. We acknowledged the cooperation of the Unidad de Estadísticas y Evaluación de Impacto, in providing the information used in this research. In particular, we are grateful to Mónica Muscolino, consultant of the Unidad de Estadística, and Sergio Diba, Coordinator of the Programa Joven.

** Correspondence to Cristian Aedo, Professor of Economics, Graduate Program in Economics ILADES/Georgetown University, Universidad Alberto Hurtado. Email: caedo@uahurtado.cl

The Inter-American Centre for Knowledge Development in Vocational Training (ILO/Cinterfor)
Avda. Uruguay 1238 - Montevideo - Uruguay - Tel: (5982) 908 6023 - 902 0557 - 908 0545 - Fax: (5982) 902 1305
webmaster@cinterfor.org.uy

Copyright © 1996-2008 International Labour Organisation (ILO) - Disclaimer