national evaluation of Resources for Learning lessons

I. Purpose

The primary purpose of this evaluation was to discover the effectiveness of the classroom lessons, Resources for Learning, produced by Project Food, Land & People (FLP). This study examined changes in student learning and attitudes about agriculture, the environment, and diverse cultures as a result of teachers using the FLP materials in the classroom.

II. Design

The Field Test version of Resources for Learning, with 54 lessons, was evaluated during the 1996-97 school year in three states (California, Connecticut, and Ohio), in three demographic areas (rural, suburban, and urban), with students in grades 2-3 and 4-6, and with grade 7-9 science and social studies classes. One hundred eighty-nine teachers and more than 3,600 students were involved in the evaluation.

III. Pre-test Student Assessment

A pre-test assessment of students' knowledge and attitudes indicated that before using FLP lessons, students had only a limited understanding of the issues surrounding agriculture, the environment, and diverse cultures.

Assessment of Knowledge (Cognitive Domain)
The pre-test assessment data suggest that all students--urban, suburban and rural--in grades 2-3, 4-6, and 7-9 in California, Connecticut, and Ohio lack the level of understanding about food, land, and people that 1,600 agriculture, education, conservation, and environmental leaders from across the country have identified as "basic knowledge" that every student should know about agriculture and the environment.

The combined data of 1,061 2^nd & 3^rd, 1,225 4^th to 6^th and 1,365 7^th to 9^th grade participants in the pre-test assessment of cognitive knowledge about food, land and people statements show only slight differences among states at each grade level as shown in Figure 1. Students averaged only 45.6 percent correct answers, with the 2-3 grade group scoring highest. Rural and suburban students performed slightly better than urban students (Figure 2). No comparisons were made of 2-3 grade students by area of residence.
Assessment of "Feelings" (Affective Domain)
The majority of students at all grade levels expressed positive responses to food, land and people statements posed in the pre-test assessment. There was strong agreement among responses across states. Statistical differences across states were small, and when significant were well within the range expected, given the diverse geographic regions represented.

Overall Student Response

Students were asked to indicate the strength of their convictions about the truth of a set of statements, choosing among responses from "strongly agree" to "strongly disagree." All grades responded to the following statements with "agree" or "strongly agree:"
- Water, soil, minerals and other natural resources are important for growing things.
- I care about the environment.
- I would like to learn more about people around the world.
- I would like to know more about the land.
- Plants are important to life on earth.
- It is important to eat many different kinds of foods.
- Everyone, everywhere must conserve water.
- How we wash our hands makes a difference in killing germs.
Similarly, there was general consensus on statements where students "disagreed" or "strongly disagreed." These included:
- Plant products should be the main food people eat.
- I would like to learn more about food.
- I would like to learn more about fiber.
- I would like to learn more about agriculture.
- I try to eat from each food group every day.
The differences between the statements with which students agreed and those with which they disagreed suggest that terms related to "agriculture" may appeal less to students than language related to the "earth" and the "environment".

2^nd-3^rd Grade Perceptions

Second and third graders demonstrated very positive feelings toward living things, the environment, health and well being, and conservation. Their understanding of "agriculture" was limited. They rated "I like learning about agriculture" the lowest of all 27 statements, yet they had a positive feeling toward "learning more about food or plants" and agreed that "farmers care about the environment."

They felt strongly that "people should know more about people around the world" but few agreed that "people should know more about agriculture" or that "agriculture is important to life on earth."

Farmers and government were viewed twice as positively as industry in "caring about the environment."

Second and third graders had a 66% positive response to the 27 attitudinal statements. Only eight statements received less than a 50% positive response, averaged across all students. Connecticut students had the highest positive feelings and California students the lowest.

4^th - 6^th Grade Perceptions

Fourth through sixth grade students across all states had very positive responses to 12 of 33 statements, and neutral or negative responses to only eight statements. California students had the most positive responses and Ohio students the most negative responses.

In general, 4-6 grade students strongly agreed with statements related to environment, conservation, and health and nutrition. Students rated statements about food, fiber, and agriculture neutral or negative. However, the lowest-rated statement was "I affect the environment by the way I shop for things." For the 12 most positively rated statements, only 12% of the students marked "not sure" while 42.5% of the students marked "not sure" for the 13 least positive statements.

7^th-9^th Grade Perceptions.

Seventh through ninth grade students from all three states were less positive in their responses to the "feeling" statements than were the younger students. They rated 9 of 35 statements as "agree" or "strongly agree," with a similar number of statements being "neutral" or "slightly disagree." Differences among states regarding "feeling" statements were slight, with California students exhibiting a slightly less positive feeling than students from Connecticut or Ohio.

Students were very positive about environmental, land, water, and food issues. They were less sure about agriculture-related issues such as erosion, technology, and dependence on agriculture. For the nine most positively rated statements, only 8.2% of the students marked "not sure" while 40% of the students marked "not sure" for the 14 least positive statements.

Common statements assessed across grade levels

Eight statements were posed to all students. The statements presented were:
1. Water, soil, minerals and other natural resources are important for growing things.
2. I would like to learn more about people around the world.
3. It is important to eat many different kinds of foods.
4. Ranchers/Farmers take good care of their animals.
5. Farmers care about the environment.
6. I try to use recyclable and reusable things.
7. I would like to know more about food.
8. Plant products should be the main food that people eat.
Response patterns by grade level were quite similar, with small statistical differences associated with less positive responses by 7^th-9^th grade students (Figure 3).

Five additional statements were presented to upper grade level students. The additional statements presented were:
1. It is important to save open space.
2. Agriculture is important to me.
3. I care about the environment.
4. Seed collecting is an important part of agriculture.
5. I try to eat from each food group every day.
The average response by residence for the 13 statements indicates that rural students were more positive then their suburban and urban counterparts. Data also indicates that 4-6 grade students were more positive then the 7-9 grade students (Figure 4).

Overall, responses to both the cognitive and feeling portions of the pre-test assessment showed consistency across states, residency, and grade level.

IV. Post-Test Student Assessment Comparisons

Major Finding: Statistical analysis indicate that FLP materials had a significant positive impact on student knowledge and attitudes about agriculture, the environment and diverse cultures.

Answers to the following key evaluation questions about the effects of grade level, area, state, and subject reflect the impact of FLP materials and some benefits of the FLP educational program:

Were gains similar with 2-3, 4-6 and 7-9 grade students? No. When compared to their control peers, the attitudes and knowledge gains of 2-3 grade students were greater, followed by 4-6 grade students, followed by 7-9 grade students. The FLP materials worked as intended with elementary students, second through sixth graders. Elementary students in the experimental group (whose teachers had the FLP materials) had significantly greater knowledge gains and increases in positive attitudes than students in the control groups. Junior high students (7-9) in the experimental group recorded higher cognitive gains and more positive attitudes than their control group cohorts did, but the differences were not statistically significant.
Do changes in students' FLP knowledge and attitudes differ by residence in rural, suburban or urban areas? Yes. Suburban students in the experimental grade 2-3 classes showed greater knowledge gains than their rural contemporaries. Rural 2-3 grade students recorded greater gains in positive attitudes than their suburban counterparts but not at a statistically significant level. Insufficient data were available to compare urban 2-3 grades adequately with the other residency groups.
Suburban students in the 4-6 grade experimental group showed the greatest and most positive changes in their attitudes about food, land and people statements. Suburban experimental students registered significantly higher positive attitude changes than did their cohorts in the suburban control or the urban and rural control and experimental groups.

Regardless of treatment (control or experimental), the knowledge gains of all rural 4-6 grade students between the pre- and post-assessment tests were significantly greater than gains of the urban and suburban 4-6 grade students.

No statistically significant differences were observed for 7-9 grade students on the basis of area of residence.
Do students' FLP knowledge and attitudes differ based on geography or state of residence? Yes. Statistically significant differences were found based on state comparisons at all grade levels. All experimental groups registered higher cognitive gains than their cohorts in the control groups did, but these gains were not statistically significant. The knowledge gains and attitude changes for students appear to mirror the length of time the teachers in each state had access to the FLP field test materials. California and Connecticut teachers, who had workshops held before or at the beginning of the school year and administered the pre-tests earlier, showed much larger student gains in knowledge and more positive attitude changes. The students in Ohio, where teachers had the FLP materials and were part of the field test for only four months, showed the smallest knowledge gains and attitude changes.
California's second and third graders had the largest knowledge gains and attitude changes. Knowledge gains for 2-3 grade students in Connecticut were next highest, followed by Ohio. Significant attitude differences based on state and treatment effects were observed. The attitude changes in the experimental 2-3 grade students in California and Connecticut were nearly identical and significantly higher than the attitude changes noted for the Ohio students.

At the 4-6 grade level, all students in California and Connecticut showed similar knowledge gains and nearly identical attitude changes from pre- to post-assessment. The 7-9 grade students in Connecticut had higher cognitive scores than did their cohorts in Ohio.

Inadequate data were available to permit statistical comparisons of California 7-9 graders with other states. Data for urban 7-9 grades were lacking from all states.
Are there differences between the performances of junior high school students in science versus social studies classes? Yes. Although not statistically significant, students in social studies experimental classes showed greater learning gains between pre-test and post-test than did their cohorts in the social studies control classes or any science classes. The positive gains in attitudes of students in social studies control classes were greater than those of students in the FLP social studies classes or either of the control or experimental science groups.

V. In Summary

Before using FLP materials, students demonstrated a positive attitude and knowledge about the environment, but very limited or negative attitudes and knowledge about agriculture, food, and fiber. The good news about this disparity is that the years of effort by environmental educators appear to have had an impact on our students in our nation's schools. Yet as Craig Cox, Executive Director of the International Soil and Water Conservation Society said at FLP's 1^st Annual Symposium, Building Bridges of Understanding Between Agriculture, Education and the Environment, "It's not a question of whether agriculture has an effect on the environment. The reality is, for most of the United States, agriculture is the environment." Our young people need to better understand both of these critical areas.

This evaluation indicates that FLP lessons can play a key role in students learning more about food, cultures, and the environment. Use of FLP materials produced statistically significant gains in knowledge and more positive attitudes for students at the elementary level. More studies are needed to statistically demonstrate the impact on junior high school students.

Records indicate that during the field test, experimental teachers were able to incorporate five to six FLP lessons into their grade level curriculum. Thus, if five to six different lessons were incorporated into each grade level (PreK-12) curriculum, in all schools across the country, our young people would be better prepared to meet the challenges of global resource use for the benefit of humans in the next millennium.

The national evaluation of FLP lessons demonstrates that the Resources for Learning curriculum is an effective educational tool for boosting student understanding of the interactions between the environment, agriculture and human cultures.

VI. Acknowledgements

We gratefully acknowledge the contributions of key individuals who worked on the design, operation, analysis, interpretation, and review of this field test.

Dr. Lyn Fleming, AZ--Independent Evaluator

Roxanne Brickell-Reardon, CO-- FLP Executive Director

Rich Engle, CA--State Coordinator

Bernie Kayan, CT--State Coordinator

Carol Davis, OH--State Coordinator

Dr. Vernon Cardwell, MN--Executive Summary Writer

Alison Peters, CO--Editor

A special thanks also to:

The 189 teachers, their students and their school administrators who participated in the field test. The many contributors from the three participating states who made this important effort possible.

April 2001

back