These two articles (1, 2) help move chemical education research on survey development into a new era. This new era is characterized by two things: (i) a theoretical base outside the chemical education literature, and (ii) validity and reliability measures for instruments that have a strong statistical foundation. Others in the chemical education field (3, 4) have helped contribute to this new sophistication in survey research design. No longer is it acceptable for a researcher to sit alone in an office to develop a survey to administer to students and report the results as publishable research. Surveys must be developed with a theoretical construct in mind and often this construct about learning has been developed in fields other than chemical education. Bauer and Barbera et al. (1, 2) both point to research on beliefs versus attitudes that have been established in other disciplines. Once the construct such as beliefs or attitudes is defined through the literature, arguments are presented in these articles about why this construct has some meaning within the field of chemical education. Both researchers make the distinction between attitudes or beliefs and achievement. This is a point that is often overlooked by researchers when reporting survey results. Once the theoretical construct such as attitudes or beliefs has been established, these authors take the reader through a careful discussion of decisions that were made in the design of the instrument from Bauer’s discussion of the semantic differential model to Barbera et al.’s description of how the CLASS-Phys instrument was modified in the creation of the CLASS-Chem instrument. Especially important in Barbera et al’s discussion is how the initial modifications were tested and revised. Both authors describe this validity process in terms of student interviews and initial use and analysis of the instrument with a separate student sample. Validity testing using factor analysis is an especially strong approach because it relies on data that can be analyzed for the presence of grouping or factors within the survey questions. Grouping, factors, or subscales provides the researcher with a more useful way to explain the results of a survey rather than presenting student responses as percentages on individual questions. Once a group, factor, or subscale has been identified, the survey results can be presented as measures of identifiable variables. Validity, however, is only one part of the process. The other part is reliability. Both authors calculated and correctly interpreted the classic statistical measure of reliability (Cronbach’s α). They also both took the reliability test a step further by measuring a test–retest statistic. This second statistic provided a more informed answer to the question of whether the surveys described were reliable. Without validity and reliability results that the survey developer can use to modify the instrument, the data collected from an implementation of the survey may be either in error or uninterpretable. Research on how students learn must be based upon the data provided by instruments that have been proven to measure what they say they are measuring and are shown to provide stable results. Anything less could lead to erroneous results or interpretation of data. Both of these authors have done an admirable job of describing the arduous task of conceiving, developing (or modifying) survey instruments, proving that the instruments are valid and reliable and helping the chemical education community to understand the process. Others in the field can build on this process and use it as a model to develop or modify new instruments that will help us measure the variables involved in the complex operation of learning.
Chemical Education Research; Communication / Writing; First-Year Undergraduate / General; High School / Introductory Chemistry; Upper-Division Undergraduate
Our Secondary School editors work hard to distill all the JCE materials to produce a fraction of particular interest to high school teachers. We call it CLIC.
In recent years we have worked hard to better match our advertisers with our readers. When shopping for chemistry education materials, visit our advertisers' WWW sites first.
Take JCE along on your outreach missions. Copies of the Journal, guest access to JCE Online, our publications catalog, and more are available for your participants.
For journals in other fields of chemistry visit ACS Publications.
