Functional Literacy Programme - Indonesia

Directorate General of Out-of-school Education, Youth and Sports

Department of National Education
Indonesia 1999

Source: Department of National Education

1. Background
2. Goal to Build Local Capacity
3. Implementation
4. Current Status
5. Training of Trainers Design
6. Materials Development of Action Learning Guides
7. Action Learning Guides
8. Evaluation
9. Conclusion

Annex 1.: Methodology for the Evaluation Study

A. Background

Since 1979, Directorate of Community Education had developed literacy programme related to Community social condition, with focus on functional in order to strengthen the programme. In 1995, the Directorate of Community Education (Dikmas) continued to develop functional literacy programme. Consultants was hired and a team of counterparts and master trainers embarked on a three year action research project to develop a new model that would address the current needs of literacy learners in the diverse regions of Indonesia. The new model, based on a participatory action learning approach is a major change from the book-based, centrally-managed Packet A programme which was previously used by Dikmas. Therefore, a comprehensive and systematic training programme was needed to prepare learning groups to take on more responsibility for their own learning, and to prepare each district to design and implement locally relevant versions of the programme.

The new functional literacy programme was designed to cover Indonesia’s need for both a literacy and post-literacy programme. Current educational statistics indicate that there are still approximately 6.9 million illiterates between the ages of ten and forty-four. In addition, there are many citizens with minimal education who do not have the functional literacy competencies, they need for solving problems in daily life. Approximately 75% of the learners in current learning groups fall into this category. The other 25% are pure illiterates.

In order to meet the needs of multilevel groups, the functional literacy programme distinguishes three levels of literacy development: Basic Skills (Pemberantasan), Guided Learning (Pembinaan), and Self- Learning (Pelestarian). Basic Skills activities focus on the needs of those individuals who do not have basic reading, writing and math skills. Guided Learning activities provide an opportunity for learners to develop functional competencies for using their literacy skills in daily life. Self-Learning activities focus on helping learners develop the capacity to meet their own learning needs through finding their own reading materials, joining community development programmes, planning their own income-generating activities, and so forth.

The New functional literacy programme is developed through a bottom-up strategy based on the following principles:

1. Local Context. The definition of “functional” depends on what kind of reading and writing skills are commonly needed in the learners’ community. People living in cities and working in factories and offices clearly need an information and literacy skills than people living in remote villages. Therefore, each group needs their own definition of functional literacy.
2. Local Design. Each learning group makes it own learning plan based on the learners’ needs and interests. The tutors receive training on how to assess the learners’ literacy skills, knowledge, needs and interests; how to design their own learning activities and curriculum; how to make their own learning materials; and how to network with local organization in order to find learning materials and resources;
3. Participatory Processes. The learners are involved in all aspects of the learning group. They develop their functional capabilities by making decisions and plans for their own learning activities. They also participate in finding and making their own learning materials.
4. Action Outcomes. Results are measured in term of the learners’ ability to use their reading, writing and math skills for practical purposes in daily life, for learning by doing, is the focus of every reading, writing and math activity from the very beginning of the programme.

B. Goal to Build Local Capacity

The goal of the functional literacy programme is to develop the ability of learners making use of literacy skills to solve problems and meet learning needs in daily life. The programme is learner-centered rather than book centered. This means that the learning activities focuses on helping learners reaching/obtaining goals and skills that are relevant to their personal needs and interests. The programme identifies three stages for literacy learning and recommends that learners participate for at least three years to achieve the programme goal. The first stage is for building basic skills (Pemberantasan or “eradication of illiteracy”). The second stage is for guiding the learners in how to use their literacy skills to solve problems and find information in daily life (Pembinaan or “guided learning”). The third stage is for developing self-learning capacity (Pelestarian or “lasting or continuous learning”).

Although the goal and stages of literacy learning are logical, implementation is difficult because the learners in each group have different levels of prior education and experience. As a result, they start at different levels within the three stages and then progress at different rates. This poses a problem for the tutor who must prepare learning activities for a multi-level group. It also poses a challenge for evaluation in that pre-assessment of existing skills and abilities is critical for accurate understanding of the learners’ subsequent progress in the learning group.

The goal of the training workshops is to build local capacity for functional literacy so that every district will have people qualified to identify community needs and potential, teaching learners, training tutors, developing materials, designing curriculum, organizing groups and managing the programme. The training and development of the district-level support system is the most critical factor for long-term success of the programme because all government agencies are currently undergoing a process of decentralization. At this point in time, the exact configuration of the decentralized system is still being defined and the future roles of Dikmas personnel and District Learning Centers (SKB) are still unclear. The training materials will need to be adapted to prepare for the new situation. In the meantime, the following roles are recommended for the SKBs to provide support to the functional literacy programme.

1. Supervision and Support for Learning Groups Coordinate activities with Community Learning Centers and other organizations in the villages to find learning materials and resources to meet the on- going needs of the learning groups.
2. Tutor Training Provide the five days tutor training workshop when the tutors begin working. Co-ordinate monthly tutor association meetings to share experiences and improve their skills. Visit the learning groups to provide in-service assistance.
3. Collection and Distribution of Learning Materials Promote the collection, circulation and use of reading materials that are already available in the local area (libraries, reading corners, government agencies, book stores, etc) Produce and distribute a newsletter with local news, articles, stories, recipes, experiences and activity reports written by the literacy learners.
4. Local Materials and Curriculum Development Create locally relevant materials for the learning groups that use the language and perspective of the learners, and include guidelines to assist the tutor in planning action learning activities.

C. Implementation

Steps for implementing the functional literacy programme include:

1. effective co-operation,
2. training of trainers,
3. community support,
4. tutor training,
5. self-managed learning groups,
6. integrated evaluation , and
7. active support system.

1. Effective Co-operation

Successful implementation of a bottom-up strategy in top-down system begins in the provincial office which manages the programme budget. The programme’s success is dependent on the leaderships and ability of the provincial office to collaborate with master trainers and community educators from the BPKB (Provincials Center for Development of Education Activities) and the SKB (District Level Learning Center). The BPKB provides training expertise and knowledge on how to develop local education models. The SKBs provide district-based expertise for the development of local functional literacy models and materials, training of tutors and the support of learning groups.

2. Training of Trainers

At the provincial level, a team of master trainers from the BPKB and provincial office is trained to implement the programme throughout the district in the province. The master trainers are responsible to plan the implementation strategy and provide two types of training to the SKB in each district. a. Technical Training for the SKB to prepare community educators and field workers to train tutors and support learning groups (10 days). b. Administrative Training for Dikmas to orient administrative personnel and supervisors (5 days).

3. Support Services

The SKB Learning Centers are ideally situated to serve as resource centers for training and materials development. The community educators from the SKBs work together with the supervisors and fieldworkers who monitor and assist the groups from the sub-district level.

1. Management of Learning Groups The community educators at the SKB work with the supervisors and fieldworkers as well as community leaders and local organization to set up and supervise learning groups in the district. These various groups of people from a support system to identify learning materials and resources to meet the on-going needs of the learning groups. They also coordinate activities with the Community Learning Centers and other organizations in villages. The purpose is to integrate the learners into local community development activities by using the functional literacy groups as a bridge.
2. Training The community educators at the SKB provide a five days tutor training workshop as well as follow-up support through tutor association meeting. The fieldworkers visit the learning groups to provide in-service assistance to the tutors.
3. Collection and Distribution of Learning Materials There are already many reading materials available in the village reading corners (TBM), SKB library, Dikmas Office, other government agencies and local organization. The SKB plays a role in promoting the collection, circulation and use of these existing reading materials in the learning groups. The SKB library networks with the community learning center, village reading corners and learning groups to identify learning interest and distribute relevant learning materials to the groups. The community educators simplify the materials for beginning readers.
4. Local Materials and Curriculum Development The community educators at the SKB create locally relevant materials for the learning groups. These materials need to use the language and perspective of the learners, and include guidelines to assist the tutor in planning action learning activities. In order to create such materials, the community educators work with the learning groups to identify topics. Materials include a bulletin for learners, action learning guides for tutors and other learning materials on topics of local interest. The bulletin provide a place for announcements community educators write articles and news stories based on needs and interest of the learning groups. The learners write articles, stories, report, based on their experience, local news and activities in the learning groups.

4. Tutor Training

The tutor plays the most important role in the learning process. Creative and innovative tutors are recruited and trained to use action learning methods with the learners. The training activities are based on experiential learning methods with a minimum of lectures. Training activities include:

1. Field trips to survey the uses of literacy in the community;
2. Meeting with local organization to find resource person and learning materials;
3. Meeting with learners to practice pre-assessing ability levels, needs and interest;
4. Simulations, role-play and follow-up visit to a learning groups to practice technique to teach reading, writing and math;
5. Small group work to simplify reading materials and to learn how to make lesson plans;
6. Discussion on how to manage multi-level learning groups;
7. Discussion on how to demonstration on how to use checklists to evaluate the learners’ progress on basic skills and functional competencies.

5. Self-managed Learning Groups

1. Characteristics of Learners
   The learning group consists of learners between the ages of 10 and 44. They come from the same neighborhood and share common interest in improving their income, community, family life, etc. Most groups are at multi-level, meaning that pure illiterate learners are mixed together with learners that already have some prior education. The tutors manage multi- level learning activities by dividing the learners into small groups or partners so that they can help each other with various reading, writing and math tasks.
2. Meeting Times
   The learning group is responsible to choose and manage their own meeting times by making a contract. They are free to reduce or change times during busy seasons or holidays. However, they commit to follow their contract and meet regularly.
3. Planning Process
   Each learning group decides on their own topics and learning activities. The groups are given guidelines and forms for making their own learning plans. In addition, the tutors help the learners making plan and writing a proposal to receive a learning fund of Rp. 100.000 which they can use for any type of learning activities they choose. Groups usually choose to learn a practical skill or try out an income-generating activity.
4. Learning Activities
   The basic learning approach combine literacy skills with learning by doing or learning by one own experience. Every topic that the learner’s study is explored with the following learning activities:
   1. discussion,
   2. writing,
   3. reading,
   4. math, and
   5. action or application.

   For example, if a group wants to make and sell mats, they discuss their ideas and make a learning plan. They practice writing the names of materials and instructions for making the mats. They read the results of their writing as well as a leaflet on math-making. They use math to calculate the materials and cost for making the mats. They practice making the mats. Then they analyze the potential for marketing the mats and write a plan. Finally, they calculate sales prices and keep records of their sales and profits (or losses);

5. Learning Materials.
   The learners are involved in making many of their own learning materials by drawing and writing about their own experiences and knowledge. The most simple reading materials is a word or sentence spoken by a learner and written by a tutor. Learners also write recipes, stories, description of daily activities, directions for making traditional medicines or handicrafts, as well as maps, charts, diagrams, calendars to analyze local knowledge, experience and conditions.
   Each learning group is allocated Rp 100.000 for creating their own learning materials and practicing their writing skills. The materials include notebooks, pencils, scissors, cutters, newsprint and poster board, colored markers, eraser, blank paper, and resources for making copies. The learners can publish their writing on a wall newspaper (bulletin board), in small booklets or in the bulletin for learning groups produced by the SKB.
   To supplement their own knowledge, the groups select reading materials such as posters, forms, brochures and booklets collected from the Village Reading Corner (TBM), SKB, health post, agriculture service, post office, bank, and other agencies. They can also use literacy modules and Package A booklet produced by Dikmas. Guided-learning activities focuses on using existing materials from daily life. Identify cards, advertisements, government form, bank form, record books, etc. are easy to find their own practical learning materials. Self-learning activities encourage learner corner or other available source to find and borrow books as one of the regular activities of the learning groups. The learning groups can even make their own lending libraries with free materials they collect from agencies as well as article from used magazines and newspapers.
6. Continuity of the Groups
   The groups are funded for a minimum of two years to enable learners to complete the basic skills and guided learning activities. Groups that continue for the third year will prepare a proposal for their own self-learning activities. They can request between Rp 100.000 and Rp 1.000.000 to cover the cost of resource persons, skill development, materials business capital, etc. Groups are funded based on the quality of their proposals and their capacity to carry out the activities. The community educators and fieldworkers assist groups in planning and implementing their self-learning programmes.

6. Integrated Evaluation

Since the learners enter the group with different level of ability and progress at different rates, the tutor has to continually evaluate learners’ ability level in order to plan appropriate learning activities. Evaluation starts with an assessment of the learners’ interest, needs and skill level at the time of joining the group. This information is used to make learning plans and to establish a base-line for measuring subsequent progress. At the end of each month, the tutor uses checklist to review the learners’ progress in developing basic skills and functional competencies. This information is used by the tutor to plan learning activities during the coming month.

Each month the tutor and learners write a brief report for Dikmas about their learning activities. The tutor report on what activities they did, what learning materials and resources they used and any problems they encountered during the month. The learners each write a discuss the progress of the groups and identify what materials and resources will be needed in the coming month. At the end of six months, the learners take an achievement test which provides Dikmas with information to compare progress groups and ensure that learners are reaching the established standards for reading, writing and math ability.

D. Current Status

The functional literacy programme is currently in its second year of implementation. The following results have been accomplished:

1. Training Materials Completed
   1. Teaching Training Manual for Community Educator SKB and Dikmas Fieldworker (TLD);
   2. Administrative Training Manual for District and Sub-district level personnel;
   3. Tutor Training Manual;
   4. Tutor Handbook All training materials were field-tested and improved based on input from master trainers.
2. Personnel Trained 1997-1998

   39 Master Trainers from 9 provinces and central level

   250 Community Educators at 49 SKBs

   392 Fieldworkers from 196 sub-districts

   196 Supervisors from 196 sub-districts

   250 Dikmas staff from District Offices

   24 Head of District Learning Center (SKB)

   24 Head of Dikmas District Offices

   1,230 Tutors

   5,000 Learners

3. Achievement of Learning Groups 1997-1998
   Five hundred (500) groups completed six months of functional literacy learning activities. Based on data from a sample of the groups:
   1. 100% designed their own curriculum;
   2. 95% adopted 2 or more of the methods and techniques from the training;
   3. 82% used the action learning methods to help the learners analyze their own experiences;
   4. 59% combined practical skills with the literacy activities;
   5. 40% made bulletin boards with the learners writing;
   6. 6% networked with other organizations for resources
4. Achievement of Learners 1997-1998
   Five thousand (5000) learners were trained. Based on data from testing a sample of 1000 learners:
   1. 75% could do basic math operation (25% needed help);
   2. 54% could complete a cost estimate for making and selling a product;
   3. 72% could accurately read a simple paragraph;
   4. 36% could write a paragraph about their experience.
5. Planned Number of Learners and Tutors 1998-1999
   1. 7,300 1st Year Learners;
   2. 5,000 2nd Year Learners;
   3. 100,000 Additional Learners;
   4. 10,000 Additional Tutors.
6. Evaluating Progress of Learners
   Two comprehensive evaluation checklist have been developed base on data collected from the field test in 1997-1998. One checklist measures the development of basic reading, writing and math skills. The other checklist measures the development of functional literacy competencies.
7. Future Needs 1999-2000
   In order to expand the programme into the other provinces, they will need to be an initial investment in training master trainers and preparing SKBs and Districts:
   1. 90 Master Trainers (5 for each of the 18 provinces);
   2. 72 SKBs (4 per new province) – Technical Training;
   3. 72 District with 4 sub-district each – Administrative Training;
   4. 36 NGOs (2 per province) – Technical Training;
   5. 21,600 1st year learners in 18 new province;
   6. 7,300 2nd year learners in 9 province;
   7. 5,000 3rd year learners in 9 province

E. Training of Trainers Design

There are three levels of training integrated in a cascading training of trainers model. The first level prepares master trainers, the second level prepares the district level support system, and the third level prepares tutors. Table 1 provides an overview of the content and participants.

Table 1: Types of Training Workshops

F. Materials Development of Action Learning Guides

1. Status

The status learning guides were developed in response to last year’s evaluation data which indicated that the tutor needed assistance in developing appropriate and comprehensive functional literacy learning activities. The guides are designed to help the tutors implement a participatory curriculum that includes discussion, writing, reading, math and active implementation of new knowledge. Each guide provides instructions and ideas to involve the learners in collecting and analyzing local information in order to make the topic relevant to their living conditions. Every learning activity includes ideas for action as well as a checklist to help the tutor evaluate the learner progress and readiness to continue with the next activity.

2. Activities Undertaken

1. Regional Workshops
   Three regional training workshops were completed to teach master trainers how to develop Action Learning Guides based on topics of local interest. See attachment 3 for participant list and schedule of activities.
   1. 17 – 28 September in Palembang for participants from North Sumatra, South Sumatra and DKI Jakarta;
   2. 05 – 16 October in Surabaya for participants from East Nusa Tenggara, East Java and South Sulawesi;
   3. 26 October – 6 November in Ungaran for participants from West Java, Central Java, and South Kalimantan;
2. Provincial-Level Follow-up
   The master trainers who participated in the regional workshops have the responsibility to complete the development of the action learning guides with the three experienced SKBs in their provinces. The follow-up workshops involved the community educators from the 25 SKBs which are in the second year of the functional literacy programme. Each SKB is working with 2-5 learning groups to field test the action learning guides. The field test includes two days orientation workshops, one month try-out of the materials and a 2-day follow-up reflection workshop. Table 2 shows the training that was completed during this quarter.

Table2: Training for Developing and Field-testing Action Learning Guides

3. National Consultants
   Two national consultants have been hired to assist with the development of action learning guides and other functional literacy activities during the final six month of the project. They participated in the regional workshops and then assisted the master trainers to implement the follow-up workshops in each province.
4. Summary Workshop for Central Master Trainers
   Following the regional workshops, the consultant presented a summary of the process and results for making action learning guide to the master trainers at Directorate of Community Education and Directorate of Technical Personnel Education. This group reviewed the learning guides, corrected mistakes, added learning activities and evaluation checklist to make them more complete. The revisions will be sent to the field to assist the master trainers and community educators in improving the manuals for the try-out with the learning groups.

3. Results

Two sets of documents accompany this report:

1. Copies of the nine (9) Action Learning Guides developed at the workshops (draft);
2. Guidelines, schedule and materials for implementing workshops to develop Action Learning Guides.

4. Follow-up

During the next quarter, approximately 75 learning groups in the nine provinces will try-out the action learning guides. Each of the 25 SKBs will hold a two-day reflection workshops to collect the results of the try-out including revision to the guides and learning materials produced by the tutors, learners and community educators.

G. Action Learning Guides

1. Product

The action learning guides are an innovative new learning materials that was developed specifically for the functional literacy programme in Indonesia. Each guide provides a step-by-step outline that help the tutor work with the learners to analyze their local environment, identify their learning needs and opportunities and implement a systematic learning process to develop reading, writing, math and practical skills. Each of learning guide the learners the following things:

1. Instruction and self-learning instrument to help the learners analyze their experience and local environment;
2. Math activities to help the learners analyze and solve the problems;
3. Reading materials that provide new technical information on the selected topic;
4. Writing activities to help the learners make new learning materials based on their own experience and knowledge;
5. Action ideas to guide the learners in implementing their new knowledge and skills to improve their daily lives;
6. Evaluation form to help tutor monitor the progress of the learners on the above activities.

More than thirty (30) action learning guides were developed and field-tested with learning groups. The best guides will be printed and distributed to learning groups in the coming year. The majority of the locally produced guides focused on skills for developing small businesses. Eight were for food product, five for handicrafts or sewing, three for clean environment, two for money management. Although the topics are similar to other Dikmas and SKB publications, these materials are more locally relevant because they include instructions for action learning activities, for collecting data about the local market, and for discussing problems and possibilities. They also provide a more complete learning experience because writing, math and application activities are included with the reading materials.

2. Results of the Try-out

The action learning guides were developed in a participatory method that involved learners, tutors, pamong belajar, fieldworkers (TLD) and peniliks in the process. Two national consultant helped facilitate three regional workshops, nine provincial workshops and 25 district-level workshops to develop the guides and adapt them to local situations. The following people participated in the progress:

1. 36 Master Trainers (BPKB and Bidang Dikmas);
2. 135 Pamong Belajar (SKB);
3. 50 Tutors;
4. 36 Peniliks;
5. 36 TLDs;
6. 250 Learners

From their active involvement in the process many learners increased their self-confidence to participate actively in setting goals, learning new skills and managing learning activities to continue their own development. Many tutors increased their ability to facilitate a learning process that is learners-centered in a more systematic way. They also increased their ability to monitor the development and results of the learners so that they could plan better ways to help the learners improve their skills.

3. Problems

In spite of the positive out-comes, there are still many things that need to be improved in the action learning guides. First of all, the time for the filed test was not long enough because it was interrupted by the fasting month, and the orientation workshops for tutors and learners was too short. Secondly, the concept is so new that the people who helped developed the materials are not yet aware of all the possibilities and learning strategies that need to be incorporated into the learning guides. As a results, there are still things that need to be analyzed, learned and developed.

1. As with all functional literacy learning groups, a bottom-up approach still needs support from the system. Groups that did not receive their learning funds or that were not supported by monitoring and inputs from the Pamong Belajar, TLD or Penilik did not do as well as groups that received these supports.
2. Not all tutors were able to adapt the guides to local circumstances. They still feel obliged to follow books to the letter. In the worst case, a tutor had the learners copy the crops from the sample maps onto their local maps even though crops like tobacco weren’t even grown in the region.
3. The most successful action learning guides were the ones that focused on making and selling a small craft or food product. These are based on the same types of materials that the SKBs have always made and reveals the fact that everyone tends to view functional literacy as small business and skills development with reading and writing. Even some of the groups that field-tested learning guides on healthy families and managing the family economy ended up writing an action learning guide for a small business. While small business is clearly important to economic survival, this practice undermines that potentials to develop action learning guides to promote other ways to improve the standard of living such as gardening, managing money, cleaning the home environment, etc.
4. The most difficult action learning guides to make and field-test were the ones that required the groups to actually for a survey of the problems and potential in their local environment. Both the materials developers and the learning groups wanted to jump immediately to making and selling a small product to earn a few additional rupiah. The failures of the guides show that there is a great need to educate Directorate of Community Education and Directorate of Technical Personnel Education staff on the basic skills and theory of action research, problems posing and community development. Furthermore, there is a need for more focused research and development activities in local areas to identify a strategy for making and marketing small crafts and snacks is not sufficient to bring learners and their families out of poverty.

4. Recommendation

1. Edit the most successful action learning guides for publication and distribution to tutors when they are trained next year;
2. Improve and simplify the action learning guides that require more local analysis of the community problems and potential;
3. Develop new topics to explore a range of problems in addition to income generating;
4. Continue developing the concept of Action Learning Guide for functional literacy materials as well as for Income Generating Programme (KBU), Package B, etc.;
5. Make a plan to train every SKB in the methods for making Action Learning Guide based on local issues. The Pamong Belajar need more comprehensive training on action research and participatory needs assessment to enable them to produce more effective local action learning guides.

H. Evaluation

A comprehensive evaluation system that evaluates the learners’ ability and progress before, during and after the learning process has been developed. This system is incorporated into the training, learning and reporting materials.

1. Evaluation before the learning process begins

The preachment evaluation was designed to gather information about the learners’ interests, technical skills and literacy skills. This information was used for planning the initial group activities. The tutor helps the group produce a learning contract and learning plan. Dikmas can collect copies of the preachment evaluation forms to document the learners’ starting points. Copies of learning contracts and learning plans can enable Dikmas supervisors to keep track of what type of learning activities each group is doing.

2. Evaluation during the learning process

In contrast to traditional programme planning where needs are assessed only at the beginning, the functional literacy programme incorporates an on going needs identification, planning and evaluation process. The reason for this is that the learners don’t know all of their needs and interests from the beginning. In fact, the ability to identify learning interests and needs is a functional skill which is developed during the learning process. The tutors and learners fill out monthly reports, identify new learning needs and make new learning plans on a regular basis. Dikmas supervisors are able to use the monthly reports to monitor the progress of the groups.

Each month, the tutor is also expected to review each learner’s progress and record their achievement on progress checklists. The tutor uses two different types of checklists. The first checklist (Basic Skills) is especially designed for learners who do not yet have basic literacy skills such as knowledge of the alphabet, or the ability to sound words, or read sentences fluently. The second checklist (Functional Competencies) includes a list of functional literacy applications generally arranged in order from most simple to most difficult. The tutors use these lists to get ideas for teaching activities as well as to document when the learners have the capacity to use their literacy skill for daily life activities. The tutors are also encouraged to supplement the checklists with their qualitative observations of the learners’ progress by writing notes in a teachers’ log or diary. In addition to helping the tutors plan and evaluate, these checklists provide a valuable source of data about the learners progress and have been used extensively in the present study. More details and analysis are given below.

3. Evaluation after the learning process

At the end of each funded learning period, an achievement test is given to the learners. Currently, this test is being used to gather comparative data. However, because many learners and tutors want some sort of certification test, the results of the present study will be used to make recommendations for this purpose. Although the main purpose of the evaluation process is to provide information and feedback to the tutors and learners, tests are also designed so that Dikmas can use the same information for certification and for research and monitoring purposes.

The data from two years of field-testing is being analyzed to identify achievement standards for the three levels of functional literacy learning: basic skills, guided learning and self-learning. Furthermore, report from the nine field test provinces are being compiled to assess the capability of the SKBs to provide training and support for functional literacy at the district level.

1. Product
   1. Evaluation checklists for learners’ progress in basic literacy skills and functional competencies;
   2. Achievement test and scoring guide;
   3. Data summary form for statistical data collection and monitoring;
   4. Survey of SKB and district level support capability.
2. Problems data analysis is not complete at the time of this report due to the fact that learning groups could not be evaluated before the and of February. Data from sample of 850 learners has been entered into the computer. Report from six out of nine provinces have been received. The full evaluation report will be submitted as an accompanying document to this report in a few weeks time.
3. Recommendation qualitative and quantitative research still needs to be done to evaluated and understand the functional literacy programme. Therefore, it is recommended that additional research be done by S2 students who are also functional literacy master trainers at State University of Yogyakarta. Discussions have already begun with professors and the students about how to collaborate with Dikmas on research to evaluate learners’ progress, identify standard, document the learning process and to evaluate the ability level of tutors.

I. Conclusion

The functional literacy field-test has results in all of the materials necessary to reproduce the programme in the other 18 provinces of the country. There are training manuals to prepare master trainers for each province, manuals to prepare trainers and support staff at the district level and manuals to prepare tutors. Furthermore, there are four types of orient other administrative staff as well as out-side to the theory and methods of the functional literacy standards and progress points to enable planners and tutors to guide the learners more effectively. The functional literacy programme has been enthusiastically received by learners, tutors and fieldworkers. The final and most critical recommendation is for Dikmas to develop an action plan for training that will enable the programme to be implemented effectively in the new provinces and incorporated into the new Community Learning Activities Center (PKBM) programme that is being developed.

Annex 1.: Methodology for the Evaluation Study

Approximately 12,300 learners in nine provinces participated in learning groups during the second year of the functional literacy field test. 5,000 were continuing their second year of learning and 7,300 were starting their first year. The evaluation materials for the research consisted of two basic instruments: the progress checklists and the achievement test. The progress checklists were filled in by the tutors each month over a six month learning period based on their personal observation of the learners’ progress. These checklists were collected when the achievement test was given to the learners in February 1999.

A. Research Sample

At the time the achievement test was administered, the District Learning Centers (SKB) in each of the 49 participating districts were asked to submit a sampling of test results and progress checklists from four learning groups. With 10 learners in each group, this provided a potential sample of 1960 learners. SKBs in the second year of programme implementation provided tests and checklists from learners that were continuing for the second year. SKBs in the first year of programme implementation provided tests and checklists from new groups.

After sorting out tests with major problems (e.g. all of the learners had copied the same answers), and selecting a representative sample from each of the nine provinces, a research sample of 888 was assembled including 423 tests from first year learners and 465 tests from second year learners. Data from 66 learners in the second year group had also been included in last year’s evaluation study providing an opportunity for a longitudinal comparison of their progress. Because many groups did not submit checklists, the research sample for the checklist analysis was only 452.

B. Evaluation Instruments

1. Progress Checklists

There are two types of progress checklists, one measuring basic skills and one measuring functional competencies. The progress checklist for basic skills us divided into three sections: reading, writing, and math. It focuses on the knowledge of letters and numbers and the operational skills for making meaning out of written text, and doing basic calculations and measurements with numbers. The functional competencies checklists is divided into five sections that cover the basic activities of the functional literacy groups: discussion, reading, writing, math and action (or application). This checklists focuses on the application of literacy skills for daily life activities such as contribution, reading a recipe, writing a letter, calculating a profit, or managing an income-generating activity. Copies of the checklists may be found in Annex B.

2. Achievement Test

The achievement test Is comprised of four pages. Page one is a bio-data form, page two is a proposal plan for a small income generating activity, page three is an essay questions about the impact of the learning group, and page four is for observations of the tutor. The learner is expected to read and write the answers for the first three pages during a typical two hour class session. The tutor then fills in the information on page four. A copy of the test can be found in Annex C.

C. Data Analysis

1. Procedure for Analyzing the Progress Checklists

The purpose in analyzing the progress checklists was to gain insight into the progress of various learners a multi-level learning group. Information from the first one month on the checklist is used to validate and compare prior ability with prior education. Subsequent monthly checks are used to identify the length of learning level. This level corresponds to observations that the majority of functional literacy learners have sufficient literacy skills to participate in learning group activities and do their own reading and writing without a lot of assistance. The two lower levels of scores were classified as two stages of basic skills development: Level A = emerging skills and Level B = minimal skills. This classification attempts to differentiate between those illiterate learners who require constant help to form letters and understand the use of symbols to represent sounds and those learners who have a conception written language and a set of minimal skills, but still need regular help and encouragement to remember how to put words together to express their ideas. The two groups of higher level scores were classified as self-learning A to identify those learners whose skills are good enough to begin to take more independent responsibility for learning activities and those learners who clearly exhibit independent learning skills. Of course, the scores on a paper-pencil test must be verified with direct observation of the learners both in the classroom and in daily life to verify that the score do in fact represent the correct level of achievement. Therefore, the test scores are not particularly meaningful without the results of the progress checklists and comments from the tutor’s observations.

The evidence from the learning curve graphs clearly illustrates that if learners do not attain a sustainable level of skill, they do in fact relapse back into a lower level of functional literacy. Therefore, DIKMAS needs to ensure that learning groups are funded for adequate lengths of time and that evaluation strategies include a way to follow-up on the sustainability of skills.

The next five sections examine each of the five areas of functional literacy skills in more detail. Each section includes a discussion of the subscore from the achievement test and an analysis of the related section of the progress checklists.

C. Discussion and Thinking Abilities

One of the dominant characteristics of learners entering the functional literacy group is their shyness and lack of confidence. Therefore, one of the key activities of the group is to encourage the learners to value their own experience by talking about the things they know and care about. This encourages the learners to build confidence in their ability to think, talk and work together with a group to solve problems create opportunities. Discussion ability could not be measured by a paper and pencil test. However, the self- confidence and thinking ability of the learners was documented in the achievement test by scoring the learners ability to do the following activities.

1. Organize the steps and write the instructions for the process on page 2.
2. Choosing the right values for production units and unit costs of their products.
3. Explaining their ideas clearly in the essay questions.
4. Demonstrating by their answers that they understood questions on the test.
5. Writing their own ideas for the questions on page 2 and 3 rather than copying others.

[Top]