Evaluating Vision Valley through the open-ended box
The Pymble Institute is collaborating with colleagues at our outdoor campus, Vision Valley, to research a range of facets of the Year 9 outdoor education residential program. We are interested in understanding the impact this program has on participating students with a focus on girls’ educational experiences around resilience, academic buoyancy and their sense of belonging.
One of the focus points is to evaluate the efficacy of a method to collect student reflections at least one year after the end of the program. This is an important area to explore because most research into outdoor education focuses on short term impact and programs specifically for girls or in all-girls schools are not extensively researched.
To explore a way of understanding the medium-term impacts of our outdoor education residential program, an additional question was added to the existing post-program survey which was administered. This survey was recently given to the ‘pioneer’ students who attended the first two sessions of the program in 2022. The additional question asked, What have you learnt from your time at Vision Valley that has stayed with you? Concerned students may misinterpret such an open-ended question, we provided the prompt, For example, are you better at making friends or dealing with friendship issues? Do you feel more confident at solving problems or dealing with challenges? Are you less likely to get upset by setbacks at school? Do you spend more time in the outdoors? Do you feel grateful? The question then invited students to Please take a moment to reflect and to write responses in a text box.
We aimed to phrase the question in an open way to prompt deeper thinking without it being too general. Whilst most of the forty-four responses utilised the sub-question prompts, the quality and richness of the comments revealed this may have assisted in having the comments cover more than one reflective point. During analysis, it was easy to arrange the responses thematically. However, we are conscious the prompts may have guided students’ thinking.
One response is shared below:
Are you better at making friends or dealing with friendship issues? I feel more connected with my peers. Do you feel more confident at solving problems or dealing with challenges? Yes. Are you less likely to get upset by setbacks at school? I have been more relaxed and less stressed in an academic perspective. Do you spend more time in the outdoors? Yes, I have been trying to spend more time connecting with nature. Do you feel grateful? Yes, I am very grateful for this experience. It has been one of the best opportunities Pymble has offered and I am grateful to be one of the first people to experience it.
Overall, sufficient text-based data was generated to give a rich insight into students’ memories of the program, and four key themes emerged:
- Appreciation, including feeling grateful
- Relationships, including friendships
- Natural world, including outdoor environments
- Resilience, including confidence, physical and mental strength, independence
The data itself has been analysed in detail and discussed with the Vision Valley and Upper School teams as a way of informing decision making for current and future programs. We look forward to sharing insights on our website in the near future.
As researchers, our interest is also in the usefulness of the tools we use to collect the data. Considering many students gave responses to each prompt, an alternative would be to ask each sub-question separately and include an open-ended ‘Other’ box at the end. A negative of this is the additional length of the survey and intended openness may be removed. Another approach is to hold focus group or one on one interviews, but this would be more time consuming for students as well as staff. Using our current (one year on) data as a baseline, we plan to explore alternatives in data collection for the next cohort and develop a better understanding of ways of asking students about their memories of the program and its impact.
Are you interested in our research into the Vision Valley program? Click here
