Research and Evaluation
At the Exploratorium

For over 35 years, the Exploratorium has pioneered the development and definition of new learning environments, programs, and tools that help people nurture their curiosity about the world around them. Research and evaluation are integral parts of these efforts. We believe they are critical in understanding the nature of learning and how to design for learning innovation. There are altogether four PhDs on staff in research and evaluation, and senior researchers and evaluators, with backgrounds in science education, the learning sciences, science teaching, museum studies, psychology, physics, engineering, and computer science. Below is a snapshot and brief description of some past and on-going activities in research and evaluation.

Online Professional Development and Digital Library for Afterschool Educators

The Exploratorium, together with the California School-Age Consortium (CalSAC), an established statewide afterschool leader training network, is creating and offering workshops and online professional development materials for use by state and regional afterschool organizations. The activities, materials, and workshops combine and blend both hands-on science and art activities with web-based activities, making use of the National Science Digital Library. This project is surveying afterschool programs and designing, testing, and evaluating how well afterschool educators can adopt and support a designed in-person and online approach to STEM learning and professional development.

Online Audience Research

The Exploratorium website includes rich media, articles, webcasts, hands-on activities, teaching materials, and other web resources. As many as 27 million unique online visitors have come to the Exploratorium website annually, representing over 200 different countries. Why are online visitors coming to the Exploratorium website? How do online visitors find us on the web? What impact is the Exploratorium website making on science education and learning via the Web? This area of research assesses who is our online audience and investigates a range of existing and new methods used to measure the educational and instructional impact of the Exploratorium website on learning.

Evaluation of the NISE Network: Nanoscale Informal Science Education Network

A new national Nanoscale Informal Science Education Network (NISE Net) launched in the Fall of 2005 with funding from the National Science Foundation with the mission of fostering public awareness, engagement, and understanding of nanoscale science, engineering, and technology through the establishment of a national infrastructure that links science museums and other informal science education organizations with nanoscale science and engineering research organizations.

Multiple evaluation studies are being conducted with visitors, educators, and scientists to help guide in the planning of public exhibitions, programs, media, and web content for NISE Network. This includes an evaluation of the growth and capacity of the network.

Center for Informal Learning and Schools (CILS)

In 2002, the museum inaugurated CILS, a partnership with the University of California Santa Cruz and King’s College London. It offers doctoral, post-doctoral, and professional development programs for educators, scientists, and researchers in the natural and social sciences. CILS aims to develop a better articulation between schools and informal learning institutions (such as museums and zoos), and to understand what makes this articulation a success for the benefit of K-12 science learning and teaching. Several studies taking place at Exploratorium are exploring features of effective museum-based professional programs, museum schools, and teachers’ perception of informal science institutions. CILS has also conducted a national survey of informal science institutions.

Center for Informal Learning and Schools: Museum Schools

Recent years have seen a growth in a particular type of partnership between informal education and formal education that is frequently called a “museum-school.” Museum-schools integrate formal and informal education by involving the regular use of museum resources within school-based curricular instruction. This project involves in-depth case study analyses of selected museum-schools around the country in order to investigate how and why museum-schools are first established and sustained over time, how they function, and the challenges and solution strategies they encounter in their operation.

Center for Informal Learning and Schools: ISI-Based Teacher Development Study

This study investigates the extent to which informal science institution (ISI)-based professional development incorporates features of how to produce measurable effects on teachers’ instructional practice and what factors influence districts’ decisions when selecting PD programs. This study is guided by the four central questions: What are the design features of ISI-based PD? To what extent do ISI-based PD programs integrate particular features that we know through research produce measurable effects on teaching practice? What feature of professional development programs do teachers and district leaders report as being valuable? What contextual factors do teachers and district leaders report as influencing their selection of and/or participation in PD programs, specifically ISI-based PD. In the Fall of 2005, 310 informal science institutions received surveys.

GIVE: Group Visitor Inquiry Experiences
Contact: Sue Allen and Joshua Gutwill

A science museum with engaging and interactive exhibits constitutes an ideal and understudied setting for research on inquiry learning by groups. This research study, funded by the National Science Foundation’s REESE division, creates a genre of exhibit-based, group inquiry programs for family and field-trip museum visitors. The purpose of such programs is to provide visiting groups with a small set of inquiry skills that can be used at any exhibit or even beyond the museum, to support fruitful and extended investigations. Informed by the existing literature on scientific inquiry as well as formative studies, the project aims to identify suitable skills and pedagogical strategies that are effective in the social environment and on the timescale of a typical museum visit. Exploratorium researchers are currently conducting a set of rigorously controlled experiments to assess the programs’ impact on visitors’ inquiry skills. This research will yield a series of effective programs and a set of theoretical principles that account for their efficacy.

The Outdoor Exploratorium
Contact: Toni Dancu
www.exploratorium.edu/outdoor

The Outdoor Exploratorium is an exhibit development project funded by NSF to encourage and support people in noticing the world outdoors. Some of the exhibits being developed for this project will sit at the Exploratorium’s current location at the Palace of Fine Arts, but a large majority with be situated at a remote site in San Francisco. This project is also developing web-based activities as well as activities using mobile technologies to support people throughout San Francisco in noticing their outdoors. This project’s evaluation studies look at questions such as:

· What are our audiences’ outdoor habits?
· What do visitors notice about the outdoor environment?
· (How) do the exhibits we develop help visitors notice aspects of the outdoors?

Public Program Evaluation
Contact: Veronica Garcia-Luis

Public programs mission is “To inspire curiosity, appreciation, understanding and new ways of thinking about the world and our role in it through human interaction, innovative programs, unique experiences and fresh perspectives.” Public programs creates uniquely relevant programming for families, kids, adults and new audience groups from the Latino and Chinese communities. Evaluation of public programs investigates the visitor experience in the following ways:

General Audience Questions

· What kind of socially engaging interactions are the visitors experiencing?
· What programming formats are interesting to visitors?
· What kind of inquiry opportunities do visitors like to engage in?
· What are visitors coming away with after experiencing a public program?

New Audience Development Questions

· Which audiences are underserved by the Exploratorium?
· Why is it important to create relationships with new audiences?
· Why are multilingual offerings important?
· What kind of programs interest Latino and Chinese families?

Queens Borough Public Library: Science in the Stacks (SIS)
Contact: Veronica Garcia-Luis

This NSF-funded project is an exhibit development and public programs collaboration between the Queens Borough Public Library in New York and the Exploratorium. This project integrates hands-on experiences of basic science into children’s libraries. SIS aims to provide science learning opportunities for children 3-12, especially those who already use urban public libraries but are typically under-served by science experiences. This project’s evaluation studies will investigate the challenges of:

· Presenting STEM-related interactives in a library setting
· Building proper and effective use of the interactives for 3-12-year-olds
· Designing graphics that communicate to the library’s diverse patrons

Geometry Playground
Contact: Joshua Gutwill

Geometry Playground is a traveling exhibition and set of outdoor play environments where visitors explore and experiment with geometric shapes and mathematical models. Funded by the NSF, the project provides an important laboratory for research into the efficacy of immersive experiences for enhancing informal learning. Geometry Playground is a combination of interactive exhibition, artful playground, and research project in the service of creating and evaluating a strong experiential foundation for geometrical learning.

In-depth research is being conducted on (a) immersive exhibits and their linkages with other forms of exhibits, and (b) the impact of curricular packages (Math Pathways) used in schools before, during, and after a visit to the exhibition.

Learning Science in Informal Environments

The National Research Council (NRC) through the Board on Science Education (BOSE) is conducting a synthesis of research on science learning in informal environments, across a broad range of venues (museums, media outlets, state parks, science clubs, after-school programs) and audiences (individuals, schools, families, and society). Sue Allen, the Exploratorium’s Director of Visitor Research & Evaluation, has been selected to serve on this committee of 12-14 recognized experts in research and evaluation, exhibit design, program development, and education. This consensus study will draw together the disparate informal science literatures, synthesize the state of knowledge, and articulate a common framework for the next generation of research on informal science learning.

11/08