Methodology for the evaluation of the Stephanie Alexander Kitchen Garden program
L. Gibbs A E , P. K. Staiger B , M. Townsend C , S. Macfarlane B , L. Gold D , K. Block A , B. Johnson A , J. Kulas A and E. Waters A
+ Author Affiliations
- Author Affiliations
A Jack Brockhoff Child Health and Wellbeing Program, The McCaughey VicHealth Centre of Community Wellbeing, The University of Melbourne, Vic. 3010, Australia.
B School of Psychology, Deakin University, Burwood, Vic. 3125, Australia.
C School of Health and Social Development, Deakin University, Burwood, Vic. 3125, Australia.
D Deakin Health Economics, Deakin University, Burwood, Vic. 3125, Australia.
E Corresponding author. Email: lgibbs@unimelb.edu.au
Health Promotion Journal of Australia 24(1) 32-43 https://doi.org/10.1071/HE12905
Submitted: 7 February 2012 Accepted: 21 October 2012 Published: 27 March 2013
Abstract
Issues addressed: Community and school cooking and gardening programs have recently increased internationally. However, despite promising indications, there is limited evidence of their effectiveness. This paper presents the evaluation framework and methods negotiated and developed to meet the information needs of all stakeholders for the Stephanie Alexander Kitchen Garden (SAKG) program, a combined cooking and gardening program implemented in selectively funded primary schools across Australia.
Methods: The evaluation used multiple aligned theoretical frameworks and models, including a public health ecological approach, principles of effective health promotion and models of experiential learning. The evaluation is a non-randomised comparison of six schools receiving the program (intervention) and six comparison schools (all government-funded primary schools) in urban and rural areas of Victoria, Australia. A mixed-methods approach was used, relying on qualitative measures to understand changes in school cultures and the experiential impacts on children, families, teachers, parents and volunteers, and quantitative measures at baseline and 1 year follow up to provide supporting information regarding patterns of change.
Results: The evaluation study design addressed the limitations of many existing evaluation studies of cooking or garden programs. The multistrand approach to the mixed methodology maintained the rigour of the respective methods and provided an opportunity to explore complexity in the findings. Limited sensitivity of some of the quantitative measures was identified, as well as the potential for bias in the coding of the open-ended questions.
Conclusion: The SAKG evaluation methodology will address the need for appropriate evaluation approaches for school-based kitchen garden programs. It demonstrates the feasibility of a meaningful, comprehensive evaluation of school-based programs and also demonstrates the central role qualitative methods can have in a mixed-method evaluation.
So what?: This paper contributes to debate about appropriate evaluation approaches to meet the information needs of all stakeholders and will support the sharing of measures and potential comparisons between program outcomes for comparable population groups and settings.
Key words: children, economic evaluation, evaluation methods, health-promoting schools.
References
[1] Robinson-O’Brien R, Story M, Heim S (2009) Impact of garden-based youth nutrition intervention programs: a review. J Am Diet Assoc 109, 273–80.| Impact of garden-based youth nutrition intervention programs: a review.Crossref | GoogleScholarGoogle Scholar | 19167954PubMed |
[2] Lautenschlager L, Smith C (2007) Beliefs, knowledge, and values held by inner-city youth about gardening, nutrition, and cooking. Agric Human Values 24, 245–58.
| Beliefs, knowledge, and values held by inner-city youth about gardening, nutrition, and cooking.Crossref | GoogleScholarGoogle Scholar |
[3] Ozer EJ (2007) The effects of school gardens on students and schools: conceptualization and considerations for maximizing healthy development. Health Educ Behav 34, 846–63.
| The effects of school gardens on students and schools: conceptualization and considerations for maximizing healthy development.Crossref | GoogleScholarGoogle Scholar | 16861584PubMed |
[4] Phibbs EJ, Relf D (2005) Improving research on youth gardening. HortTechnology 15, 425–8.
[5] Murphy JM. Education for sustainability: findings from the evaluation study of the edible schoolyard. Berkeley: Centre for Ecoliteracy; 2003.
[6] Skelly SM, Bradley JC (2000) The importance of school gardens as perceived by Florida elementary school teachers. HortTechnology 10, 229–31.
[7] Blair D (2009) The child in the garden: an evaluative review of the benefits of school gardening. J Environ Educ 40, 15–38.
| The child in the garden: an evaluative review of the benefits of school gardening.Crossref | GoogleScholarGoogle Scholar |
[8] Newell S, Huddy AD, Adams JK, Miller M, Holden L, Dietrich UC. The Tooty Fruity Vegie project: changing knowledge and attitudes about fruits and vegetables. Lismore, NSW: Southern Cross University; 2004.
[9] Somerset S, Markwell K (2009) Impact of a school-based food garden on attitudes and identification skills regarding vegetables and fruit: a 12-month intervention trial. Public Health Nutr 12, 214–21.
| Impact of a school-based food garden on attitudes and identification skills regarding vegetables and fruit: a 12-month intervention trial.Crossref | GoogleScholarGoogle Scholar | 18647431PubMed |
[10] Heim S, Stang J, Ireland M (2009) A garden pilot project enhances fruit and vegetable consumption among children. J Am Diet Assoc 109, 1220–6.
| A garden pilot project enhances fruit and vegetable consumption among children.Crossref | GoogleScholarGoogle Scholar | 19559139PubMed |
[11] Lautenschlager L, Smith C (2007) Understanding gardening and dietary habits among youth garden program participants using the theory of planned behavior. Appetite 49, 122–30.
| Understanding gardening and dietary habits among youth garden program participants using the theory of planned behavior.Crossref | GoogleScholarGoogle Scholar | 17336424PubMed |
[12] Morris JL, Koumjian KL, Briggs M, Zidenberg-Cherr S (2002) Nutrition to grow on: a garden-enhanced nutrition education curriculum for upper-elementary schoolchildren. J Nutr Educ Behav 34, 175–6.
| Nutrition to grow on: a garden-enhanced nutrition education curriculum for upper-elementary schoolchildren.Crossref | GoogleScholarGoogle Scholar | 12047843PubMed |
[13] McAleese JD, Rankin LL (2007) Garden-based nutrition education affects fruit and vegetable consumption in sixth-grade adolescents. J Am Diet Assoc 107, 662–5.
| Garden-based nutrition education affects fruit and vegetable consumption in sixth-grade adolescents.Crossref | GoogleScholarGoogle Scholar | 1:CAS:528:DC%2BD2sXkvFOhsLk%3D&md5=24cf15dda04554282d7f750793d4c997CAS | 17383272PubMed |
[14] Morris JL, Briggs M, Zidenberg-Cherr S (2000) School-based gardens can teach kids healthier eating habits. Calif Agric 54, 40–6.
| School-based gardens can teach kids healthier eating habits.Crossref | GoogleScholarGoogle Scholar |
[15] Lineberger SE, Zajicek JM (2000) School gardens: can a hands-on teaching tool affect students’ attitudes and behaviors regarding fruit and vegetables? HortTechnology 10, 593–7.
[16] Hermann JR, Parker SP, Brown BJ, Siewe YJ, Denney BA, Walker SJ (2006) After-school gardening improves children’s reported vegetable intake and physical activity. J Nutr Educ Behav 38, 201–2.
| After-school gardening improves children’s reported vegetable intake and physical activity.Crossref | GoogleScholarGoogle Scholar | 16731458PubMed |
[17] Pothukuchi K (2004) Hortaliza: a youth ‘nutrition garden’ in southwest Detroit. Child Youth Environ 14, 124–55.
[18] Parmer SM, Salisbury-Glennon J, Shannon D, Struempler B (2009) School gardens: an experiential learning approach for a nutrition education program to increase fruit and vegetable knowledge, preference, and consumption among second-grade students. J Nutr Educ Behav 41, 212–17.
| School gardens: an experiential learning approach for a nutrition education program to increase fruit and vegetable knowledge, preference, and consumption among second-grade students.Crossref | GoogleScholarGoogle Scholar | 19411056PubMed |
[19] Morris JL, Zidenberg-Cherr S (2002) Garden-enhanced nutrition curriculum improves fourth-grade school children’s knowledge of nutrition and preferences for some vegetables. J Am Diet Assoc 102, 91–3.
| Garden-enhanced nutrition curriculum improves fourth-grade school children’s knowledge of nutrition and preferences for some vegetables.Crossref | GoogleScholarGoogle Scholar | 11794509PubMed |
[20] Morris JL, Neustadter A, Zidenberg-Cherr S (2001) First-grade gardeners more likely to taste vegetables. Calif Agric 55, 43–6.
| First-grade gardeners more likely to taste vegetables.Crossref | GoogleScholarGoogle Scholar |
[21] Somerset S, Ball R, Flett M, Rebecca G (2005) School-based community gardens: re-establishing healthy relationships with food. Journal of the HEIA 12, 25–33.
[22] Graham H, Beall DL, Lussier M, McLaughlin P, Zidenberg-Cherr S (2005) Use of school gardens in academic instruction. J Nutr Educ Behav 37, 147–51.
| Use of school gardens in academic instruction.Crossref | GoogleScholarGoogle Scholar | 15904578PubMed |
[23] Klemmer CD, Waliczek TM, Zajicek JM (2005) Growing minds: the effect of a school gardening program on the science achievement of elementary students. HortTechnology 15, 448–52.
[24] Pigg AE, Waliczek TM, Zajicek JM (2006) Effects of a gardening program on the academic progress of third, fourth, and fifth grade math and science students. HortTechnology 16, 262–4.
[25] Alexander J, North M-W, Hendren DK (1995) Master Gardener Classroom Garden project: an evaluation of the benefits to children. Children’s Environments 12, 123–33.
[26] Canaris I (1995) Growing food for growing minds: integrating gardening and nutrition education into the total curriculum. Children’s Environments 12, 134–42.
[27] Waliczek TM, Bradley JC, Zajicek JM (2001) The effect of school gardens on children’s interpersonal relationships and attitudes toward school. HortTechnology 11, 466–8.
[28] Skelly SM, Zajicek JM (1998) The effect of an interdisciplinary garden program on the environmental attitudes of elementary school students. HortTechnology 8, 579–83.
[29] Aguilar OM, Waliczek TM, Zajicek JM (2008) Growing environmental stewards: the overall effect of a school gardening program on environmental attitudes and environmental locus of control of different demographic groups of elementary school children. HortTechnology 18, 243–9.
[30] Koch S, Waliczek TM, Zajicek JM (2006) The effect of a summer garden program on the nutritional knowledge, attitudes, and behaviors of children. HortTechnology 16, 620–5.
[31] Cason KL (1999) Children are “growing healthy” in South Carolina. J Nutr Educ 31, 235–6.
| Children are “growing healthy” in South Carolina.Crossref | GoogleScholarGoogle Scholar |
[32] Liquori T, Koch PD, Contento IR, Castle J (1998) The cookshop program: outcome evaluation of a nutrition education program linking lunchroom food experiences with classroom cooking experiences. J Nutr Educ 30, 302–13.
| The cookshop program: outcome evaluation of a nutrition education program linking lunchroom food experiences with classroom cooking experiences.Crossref | GoogleScholarGoogle Scholar |
[33] Nastasi B, Hitchcock J, Brown L. An inclusive framework for conceptualizing mixed method design typologies. In Tashakkori A, Teddlie C, editors. SAGE handbook of mixed methods in social and behavioral research, 2nd edn (pp. 305–38). London: Sage; 2010.
[34] Kickbusch I (1989) Approaches to an ecological base for public health. Health Promot Int 4, 265–8.
| Approaches to an ecological base for public health.Crossref | GoogleScholarGoogle Scholar |
[35] World Health Organisation. Ottawa charter for health promotion. Ottawa: Department of Health and Welfare, World Health Organisation; 1986.
[36] Australia Health Promoting Schools Association. A national framework for health promoting schools (2000–2003). Canberra: Commonwealth Department of Health and Family Services; 2001.
[37] St Leger L (2004) What’s the place of schools in promoting health? Are we too optimistic? Health Promot Int 19, 405–8.
| What’s the place of schools in promoting health? Are we too optimistic?Crossref | GoogleScholarGoogle Scholar | 15522948PubMed |
[38] World Health Organization. Regional guidelines: development of health-promoting schools: a framework for action. Manila: World Heath Organization Regional Office for the Western Pacific; 1996.
[39] St Leger L (2005) Protocols and guidelines for health promoting schools. Promot Educ 12, 145–7.
| Protocols and guidelines for health promoting schools.Crossref | GoogleScholarGoogle Scholar | 16739504PubMed |
[40] Quinn LJ, Horacek TM, Castle J (2003) The impact of Cookshop on the dietary habits and attitudes of fifth graders. Topics Clin Nutr 18, 42–8.
[41] Ozer EJ (2007) The effects of school gardens on students and schools: conceptualization and considerations for maximizing healthy development. Health Educ Behav 34, 846–63.
| The effects of school gardens on students and schools: conceptualization and considerations for maximizing healthy development.Crossref | GoogleScholarGoogle Scholar | 16861584PubMed |
[42] Block K, Gibbs L, Staiger P, Gold L, Johnson B, Macfarlane S, Long C, Townsend M (2012) Growing community: the impact of the Stephanie Alexander Kitchen Garden program on the social and learning environment in primary schools. Health Educ Behav 39, 419–32.
| Growing community: the impact of the Stephanie Alexander Kitchen Garden program on the social and learning environment in primary schools.Crossref | GoogleScholarGoogle Scholar | 22167317PubMed |
[43] Stufflebeam DL (2001) Evaluation models. New Dir Eval 89, 7–98.
| Evaluation models.Crossref | GoogleScholarGoogle Scholar |
[44] Robinson CW, Zajicek JM (2005) Growing minds: the effects of a one-year school garden program on six constructs of life skills of elementary school children. HortTechnology 15, 453–7.
[45] Anderson AS, Bell A, Adamson A, Moynihan P (2002) A questionnaire assessment of nutrition knowledge: validity and reliability issues. Public Health Nutr 5, 497–503.
| A questionnaire assessment of nutrition knowledge: validity and reliability issues.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DC%2BD38zltFWlsw%3D%3D&md5=377196da3f354414db3138064a3eb2aaCAS | 12003663PubMed |
[46] Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold; 2000.
[47] Eldridge SM, Ashby D, Kerry S (2006) Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method. Int J Epidemiol 35, 1292–300.
| Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method.Crossref | GoogleScholarGoogle Scholar | 16943232PubMed |
[48] Goldstein H. Multilevel statistical models. London: Arnold; 2005.
[49] Hanley JA, Negassa A, Edwardes MD, Forrester JE (2003) Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol 157, 364–75.
| Statistical analysis of correlated data using generalized estimating equations: an orientation.Crossref | GoogleScholarGoogle Scholar | 12578807PubMed |
[50] StataCorp. Stata statistical software: release 7.0. College Station, TX: Stata Corporation; 2000.
[51] Greene J, Hall J. Dialectics and pragmatism. In Tashakkori A, Teddlie C, editors. Sage handbook of mixed methods in social and behavioral research, 2nd edn (pp. 119–67). London: Sage; 2010.
[52] Collins K, ed. Advanced sampling designs in mixed research: current practices and emerging trends in the social and behavioral sciences, 2nd edn. London: Sage; 2010.
[53] Block K, Gibbs L, Johnson B, Staiger P, Townsend M, Macfarlane S, Gold L, Long C, Kulas J, Okoumunne OC, Waters E. Evaluation of the Stephanie Alexander Kitchen Garden program. final report to: The Stephanie Alexander Kitchen Garden Foundation. Melbourne: McCaughey Centre. 2009. Available from: http://www.kitchengardenfoundation.org.au/uploads//02%20ABOUT%20THE%20PROGRAM/pdf/kgevaluation_key_findings.pdf [Verified 11 February 2013].
[54] Westerman LE. Gardening, cooking and enjoying food–together: does participation in a school based kitchen garden program influence children’s vegetable related behaviour and attitudes? Unpublished Honours Thesis, La Trobe University, Australia, 2005.
[55] Anderson AS, Bell A, Adamson A, Moynihan P (2002) A questionnaire assessment of nutrition knowledge: validity and reliability issues. Public Health Nutr 5, 497–503.
| A questionnaire assessment of nutrition knowledge: validity and reliability issues.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DC%2BD38zltFWlsw%3D%3D&md5=377196da3f354414db3138064a3eb2aaCAS | 12003663PubMed |
[56] Cunningham-Sabo L, Marr-Lyon L, Hood N, Walters L (2004) Classroom cooking program’s impact on children’s cooking attitudes, cooking self efficacy and food neophobia: development and 20 testing of a quantitative evaluation instrument. J Nutr Educ Behav 36, S45–69.
[57] Erhart M, Ottova V, Gaspar T, Jericek H, Schnohr C, Alikasifoglu M, Morgan A, Ravens-Sieberer U (2009) Measuring mental health and well-being of school-children in 15 European countries using the KIDSCREEN-10 Index. Int J Public Health 54, 160–6.
| Measuring mental health and well-being of school-children in 15 European countries using the KIDSCREEN-10 Index.Crossref | GoogleScholarGoogle Scholar | 19652910PubMed |
[58] Waters E, Ashbolt R, Gibbs L, Booth M, Magarey A, Gold L, Kai Lo S, Gibbons K, Green J, O’Connor T, Garrard J, Swinburn B (2008) Double disadvantage: the influence of ethnicity over socioeconomic position on childhood overweight and obesity: findings from an inner urban population of primary school children. Int J Pediatr Obes 3, 196–204.
| Double disadvantage: the influence of ethnicity over socioeconomic position on childhood overweight and obesity: findings from an inner urban population of primary school children.Crossref | GoogleScholarGoogle Scholar | 18608640PubMed |
[59] Murphy JM, Schweers E. Evaluation of a food systems-based approach to fostering ecological literacy: final report. Cambridge, MA: Massachusetts General Hospital, Harvard Medical School; 2003.
[60] Townsend CD, Carter RI (1983) The relationship of participation in FFA activities and leadership, citizenship, and cooperation. J Am Assoc Teach Educ Agric 24, 20–5.
[61] Peterson B, Gerhard G, Hunter K, Marek L, Phillips C, Titcomb A. National 4-H impact study. Washington, DC: US Department of Agriculture; 2001.