Reaction mechanisms are a central tool for organic chemists in making predictions and understanding how chemical processes occur. Learning to compare reactions and judge the plausibility of reaction mechanisms is a key skill for organic chemistry students to learn and has been the topic of study by the following researchers. Bode, Deng, and Flynn¹  asked students to discuss which of two nucleophilic substitution mechanisms were plausible and categorized students’ reasoning as descriptive, relational, or causal. Most students provided causal arguments for their claims but did not provide the expected level of granularity and struggled to identify relevant features of the problems. Caspari and Graulich²  developed a teaching scaffold to assist students in considering multiple reaction pathways in organic chemistry mechanisms. Using the scaffold, students were able to identify implicit features within a problem. Lieber and Graulich³  examined the reasoning of students while judging the plausibility of reaction pathways and found that providing students with scaffolded opportunities to reflect on their reasoning lead students toward more meaningful learning. In each of these studies, students were interacting with representations of chemical phenomena.

When interacting with a representation we need to understand how the representation encodes information. To do this we make connections between the represented world (e.g., a sample of butane gas) and the representing world (e.g., a line structure of butane showing the connectivity of atoms).⁴  When categorizing representations into groups, Kozma and Russel⁵  found that students focused on surface features of the representations. This focus on surface features has been demonstrated in many contexts within organic chemistry education research.^6–15  For example, students often take a connect-the-dots or decorating-with-arrows approach to proposing mechanisms and may not attribute the appropriate meaning to the symbols commonly used in organic chemistry.^16–18  When solving problems students tend to focus on the structure of a substance instead of its function^{9,10,14,17,19–21}  and they tend to focus on just one feature of the problem.²² 

How a representation is presented can influence what students do and how successful they are while problem solving.²³  Flynn & Featherstone²⁴  found that students scored higher on problems when atoms were explicitly shown than when carbons and hydrogens were implicit. DeCocq & Bhattacharyya²⁵  showed that providing students with the product—in addition to the reactants—when asked to propose a mechanism, can push students toward providing reasoning based on backward-chaining (i.e., proposing a mechanism to get to the product) over forward-chaining (i.e., proposing a mechanism based on properties and interactions of the starting materials). These students preferred line structures over other types of representations because they perceived line structures to have the most relevant and least distracting information. Successful and unsuccessful students may interact with representations in different ways. While solving mechanism problems and given the option between a ball-and-stick representation and an electrostatic potential map, students spent more time viewing the ball-and-stick representation.²⁶  However, when asked to identify the location on a molecule with the most partial positive charge, students who were more successful at the task spent more time viewing the electrostatic potential map. Stieff et al.²⁷  showed that when given multiple representations to solve a problem, students visually focused on just two of the three representations available. In an eye tracking study on students interpreting infrared spectra, Cullipher and Sevian²⁸  found that students with lower conceptual sophistication tended to focus more on the atoms present. Students with more advanced conceptual sophistication had patterns of viewing the spectra that indicated they were relating the structure of the substances to the spectra. Another eye tracking study on students’ interpretation of nuclear magnetic resonance (NMR) spectra showed that novice students tend to evenly distribute their attention across the features of the problem, whereas experts tend to focus their attention on a small number of areas of interest (AOIs).²⁹  Rodemer et al.³⁰  conducted an exploratory eye tracking study to gauge visual scan patterns in students while working organic chemistry problems. They also found that the more advanced students took less time overall to solve the problems. Additionally, they found that both beginner and advanced students gave more attention to reactants than products. Overall, the more advanced students made more transitions between AOIs than beginners did, leading to a lower fixation-to-transition ratio.

This past eye tracking research has provided information on the types of representations students focus on in organic chemistry and how students transition between different parts of representations but it has not characterized what students visually focus on while evaluating the plausibility of reaction mechanisms and how that relates to the types of features they discuss in their reasoning. The relationships between students’ eye gaze and their reasoning strategies in organic chemistry is an area in need of investigation.³⁰  Additionally, curved arrow notation is central for chemists in communicating how a chemical reaction happens, but students struggle to correctly use this notation to describe the movement of electrons in a reaction.¹⁶  Thus, we wanted to better understand how students visually focus on curved arrows and how this relates to the types of features they discuss and their reasoning patterns while judging the plausibility of proposed reaction mechanism.

When interacting with a representation, students need to be able to extract encoded information from a representation, make connections to prior instances, and extrapolate that knowledge to the new representation. Thus, when solving problems using representations students need to use abstraction. The representation mapping model developed by Hahn and Chater³¹  characterizes students’ abstraction by comparing the representations they use and generate while solving a problem. The relative level of abstractness of the students’ representations can be characterized by considering how concrete or how removed from surface features the students’ referents are for the students’ representations.¹⁴  With this framework, relative lower and higher levels of abstractness can be identified from the following indicators: (1) focusing on explicit vs. implicit features of the problem; (2) describing a sequence of events vs. focusing on properties of entities and explanations of events; (3) focusing on the structure of substances vs. their function; and (4) using specific vs. generalized terminology.¹⁴  Consider hypothetical students’ responses to proposing a mechanism for the reaction between the substances shown in Figure 1.1. While interacting with these structures a student could say “The oxygen on ethoxide would attach to the carbon that the bromide is attached to and that would kick out the bromide.” However, another student might say “The oxygen has a negative charge so it can act as a nucleophile and be attracted to the partial positive charge on the carbon that the bromide is attached to, kicking out the bromide.” The first student focused on the explicit feature of the atom symbols in the problem, whereas the second student focused on the concept of partial charge. They used that partial charge to explain the events they proposed whereas the first student described just a sequence of events. While describing this sequence of events the first student focused on the structure of the substances and the second student considered the function of those structures (i.e., act as a nucleophile). Additionally, the second student used more generalized terminology (nucleophile) and the first student used more specific terminology (ethoxide). In these examples, for each indicator of abstractness the first student had relatively lower levels of abstractness than the second student. Even though the second student demonstrated higher levels of abstractness, that does not mean the student was more successful. In both cases the students incorrectly assumed that a substitution process would predominate in this reaction over an elimination reaction. This abstractness framework^14,31,32  guided our investigation of the types of features students discuss while judging the plausibility of reaction mechanisms.

Although there are many frameworks to characterize student reasoning, we selected a reasoning framework that has specifically been used to characterize students’ reasoning when thinking about organic reaction mechanisms. Thus, our investigation of students reasoning was framed around them chaining reasoning strategies.³³  Darden defined chaining as “reasoning about one part of a mechanism on the basis of what is known or conjectured about other parts of a mechanism” (p. 362).³⁴  With their framework, chaining can be further categorized as “forward” or “backward.” Forward chaining occurs when the properties of an entity are used to infer subsequent mechanistic steps. For example, a student may note that a carbonyl-containing compound is present with a nucleophilic reagent and predict that nucleophilic addition will take place. Thus, the student is reasoning in the forward direction. In contrast, backward chaining uses knowledge of a future step in the mechanism to make inferences about prior steps. For example, if the starting material contains an alcohol functional group and the product does not, a student may determine that the hydroxyl group must function as a leaving group at some point, which further suggests a prior protonation step. Backward chaining can also be used to rule out a proposed mechanism if it would generate an implausible or unproductive intermediate.

Eye tracking technology can measure where on a visual stimulus a participant is looking and how long they look at the visual stimulus by recording eye fixations (pauses in eye movement). This research was guided by the eye-mind assumption of eye tracking, which presumes that when people fixate their visual attention on a referent, they are mentally processing that referent.³⁵  Thus, eye fixations are a good measure of where students are focusing their attention.³⁶  An eye tracking instrument can record the duration of these eye fixations on specific AOIs. This fixation duration (length of time that eye movement pauses on an AOI) can then be analyzed to compare how long a participant spends processing different areas of the stimulus.³⁷  Although students eye movements are highly correlated with their verbal descriptions, eye data are paired with verbal interviews to better understand students’ cognitive processes.²⁷  For example, correlation analysis can be used to compare fixation duration to other measures such as students’ accuracy at answering questions.²⁶ 

To investigate the types of features students paid attention to and considered while judging the plausibility of a proposed reaction mechanism, we explored the following research questions.

RQ1: What was the relationship between the types of chemical features (implicit/explicit) students discuss and the proportion of time they viewed curved arrow AOIs?

RQ2: What was the relationship between the chaining types of student reasoning (sequence, forward chaining, backward chaining) and the percentage of time they viewed curved arrow AOIs?

RQ3: What was the relationship between students’ success with judging the plausibility of reaction mechanisms and the percentage of time they viewed curved arrow AOIs?

Undergraduate students were recruited from a second semester organic chemistry course at a public university in the Rocky Mountain region of the United States. Twenty students participated in this study during the last four weeks of the Spring 2019 semester. Participants were offered extra credit for participating. The Institutional Review Board of the University of Northern Colorado approved this research.

In this mixed method study, quantitative data was collected using eye tracking and qualitative data was collected through interviews. The instrument used in this study was created by first reviewing lecture notes provided by the instructor of the course and students’ mid-term exam response. Based on these lecture notes and student generated mechanisms, eleven organic mechanistic steps (Figure 1.2) with curved arrows were created. Some steps were drawn in a plausible manner (Q1, 3, 5, 8 and 10), and others were implausible (Q2, 4, 6, 7, 9 and 11). The mechanistic steps were reviewed by the instructor of the course for their appropriateness. Within Tobii Pro Studio AOIs were pre-assigned to each starting material, curved arrow, and question prompt. Each structure and arrow had sufficient white space between them for the eye tracking instrument to distinguish these AOIs (1 cm for this instrument).³⁸  Each AOI used in this study can be found in the supplemental information. Student eye movements were captured using a Tobii T120 eye tracker (120 Hz), which recorded eye fixation durations on each AOI. During data collection, a five-point calibration was used. Data collection was piloted in the previous summer session with three students to check that the question format was understandable.

During data collection, students were shown each proposed mechanism one at a time and asked to rate each one as “plausible” or “implausible”. They moved through each problem at their own pace while the eye tracker captured their eye movements. During the eye tracking portion of data collection, students did not talk. After working through all the problems, students were interviewed and asked to explain their choice of “plausible” or “implausible” for each question. Students were asked follow-up questions to elaborate their reasoning (i.e., “You said x, what does that mean?”, “Can you tell me more about that?”, “How did you know that could/could not happen?”, “Is there anything else that tells you this is a plausible/an implausible step?”). Students could write while describing their thinking using a Livescribe pen. Audio recordings of interviews were transcribed.

Thematic analysis on the interview transcripts was done in NVivo to characterize students’ abstractness, reasoning patterns, and success at judging the plausibility of a mechanistic step.³⁹  We characterized students’ relative abstractness based on the terminology they used (specific vs. general) and the types of features students discussed (explicit vs. implicit). When students used the specific name of a substance, we coded that as specific terminology. When students discussed an entity with a more generalized name (i.e., alcohol, nucleophile) we coded that as generalized terminology. If students referenced a feature that was clearly indicated in the problem (such as an atom, charge, lone pair dots, etc.), it was coded as explicit. When students talked about a feature that was not clearly indicated (partial charge, carbons and hydrogens hidden in line structures, etc.), it was coded as implicit. Each of these was coded a maximum of once per problem that a student answered to represent if that code was present in that student’s response to that problem.

We were also interested in the type of justification provided by students for their answers. There were broadly three types of justification provided by students. First, some merely described the sequence of events as they understood them, without appealing to any properties of the starting material or product. Second, students employed forward chaining, making a prediction by analyzing the properties and reactivity of the starting material. Third, students also used backward chaining, imagining a future state or intermediate and using that conception to justify earlier mechanistic steps. Each of these was coded a maximum of once per problem that a student answered to represent if that code was present in that student’s response to that problem.

Finally, to compare students’ success at judging the plausibility of these mechanistic steps their verbal and written responses were scored to represent their performance on these problems, as if grading on an exam. This scoring scale ranged from 0 to 3 (0 = unsuccessful; 1 = mostly unsuccessful; 2 = mostly successful; 3 = successful). For example, a student scored 0 if they did not demonstrate a correct concept. This included arriving at the anticipated (canonically correct) response of plausible or implausible based on the material they learned in class but making this judgement for faulty reasons. A student scored a 1 if their overall response was incorrect, but they demonstrated at least one correct concept. For example, they might correctly describe that partial positive and partial negative charges attract, but incorrectly identify these charges throughout the problem. A student scored a 2 if their overall response was correct, but they demonstrated at least one incorrect concept. For example, they might correctly identify partial charges throughout the problem and that these charges attract but might not evaluate which partial charges are most likely to interact. A student scored a 3 if they demonstrated no errors based on the material they learned in their course. To establish intercoder agreement, both authors coded five interviews (25% of the data). Initially, there was 93% agreement and code applications were discussed until agreement was reached. The first author then coded the remaining interviews.

Eye tracking data was analyzed using Tobii Pro Studio. Total fixation duration on each AOI (the words of the prompt, structures of the starting materials, and curved arrows) were converted to percentages based on the total time a student spent on all AOIs. This was done to be able to make comparisons since each student worked through the problems at their own pace. Spearman’s correlations were run in SPSS to compare the proportion of time students fixated on curved arrow AOIs with the frequency of occurrence across the eleven problems of each type of reasoning pattern, type of feature (explicit/implicit) discussed, terminology used (specific/general), and their success. The Spearman correlation was selected over other correlation statistics because not all variables of interest were normally distributed. Eleven data sets from each of these 20 participants were analyzed to give a total of 220 data sets for this study.

To answer the research questions, we will first describe the features, terminology, reasoning, and AOIs students used. During these interviews, students paid attention to a variety of features (explicit and implicit) to judge the plausibility of these mechanisms. They used specific and generalized terminology. Some students described a sequence of events without an explanation for why that sequence of events could occur, whereas other students used forward or backward chaining to justify their decision of the plausibility of a proposed mechanism.

Students mentioned both explicit and implicit features of the problems. Examples of the explicit features students discussed included the atoms, bonds, lone pair dots, or charges shown in the problem. Examples of the implicit features students discussed included carbons and hydrogens hidden in line structures, electrons that were not explicitly shown, partial charges, hypothetical charges and bonds arising from proposed mechanisms and implicit properties of the entities such as electronegativity, acidity, and nucleophilicity, etc. For example, the following student discussed mainly explicit features:

This student paid attention to features that were explicitly shown on the problem such as an arrow, the atoms, and bonds. Another student discussed a combination of explicit and implicit features:

This student mentioned the explicitly shown features of the atoms present, but also considered the implicit partial charge of the atoms, which was not explicitly shown in the problem (Figure 1.3).

These students discussed specific and general terminology. Examples of specific terminology these students used included the names of substances involved such as hydronium, water, chloride, and methyl group. Examples of the general terminology students used included the terms acid/base, nucleophile/electrophile, leaving group, and functional group names. In the following two quotations, the first student used specific terminology (hydronium), whereas the second student used more general terminology (acid).

While judging the plausibility of these mechanistic steps, many students described a sequence of events, whereas other students explained why that sequence of events could or could not happen using either forward and/or backward chaining. Consider this student’s reasoning for judging a step to be plausible:

This student justified the plausibility of the mechanism by describing the steps they saw occurring. When asked follow-up questions, this student continued to justify their decision by just describing the sequence they saw. However, other students used properties of the substances involved to justify their reasoning. For example,

To this student, electrons were added to make a stable double bond in the product. This student imagined the product as a “stable” substance and determined the mechanism to be plausible based on this feature of the product. This student was using backward chaining. Other students focused on properties of the starting materials to justify their decision.

This student justified the plausibility of the mechanism step by describing the charge and nucleophilicity of the starting materials. This student used forward chaining to conclude the mechanism was plausible.

The average frequency of each type of terminology, feature, and reasoning across the eleven problems is presented in Table 1.1. For example, these students used forward chaining on average in 5.6 out of the 11 problems.

The proportion of time spent viewing curved arrows correlated with the use of generalized terminology and some implicit features, but not the types of reasoning students provided nor their success. Students spent most of their time viewing the structures of the starting materials (79%), compared to AOIs of the words of prompt (6%) and curved arrows (15%). There was some variability in the proportion of time students spent viewing the curved arrows, with some students viewing the curved arrows for a larger proportion of time (22% maximum) than other students (10% minimum).

A comparison the proportion of time students viewed curved arrow AOIs to their use of terminology, explicit/implicit features, reasoning, overall time spent, and success showed a few significant correlations (Table 1.2).

RQ1: What was the relationship between the types of chemical features (implicit/explicit) students discuss and the proportion of time they view curved arrow AOIs? Students who spent a larger proportion of their time viewing the arrows also tended to use general terminology (such as functional group names) and describe some implicit features of a problem such as partial charge. They tended to double check that the resulting structure would follow the octet rule.

RQ2: What is the relationship between the chaining types of student reasoning (sequence, forward chaining, backward chaining) and the percentage of time they view curved arrow AOIs? There was not a significant correlation between proportion of time spent viewing curved arrows and the type of reasoning the student demonstrated.

RQ3: What is the relationship between students’ success with judging the plausibility of reaction mechanisms and the percentage of time they view curved arrow AOIs? Students’ success at solving the problems did not appear to correlate with time spent viewing the curved arrows.

Student success correlated with the use of generalized terminology and discussion of implicit features but not the types of reasoning students provided nor the proportion of time viewing curved arrows. There was a range of students’ performance in judging the plausibility of these proposed mechanistic steps across the eleven problems (average score 1.7/3, minimum average score 0.3/3, maximum average score 2.6/3). A comparison of students’ success to their use of terminology, explicit/implicit features, reasoning, overall time spent, and proportion of time spent viewing curved arrows showed a few significant correlations (Table 1.2).

Students who were more successful at solving these problems also tended to use general terminology and discuss implicit entities. They tended to use functional group names, discuss the presence of acids or bases, and describe the actions of a nucleophile in a mechanism. However, there was not a significant correlation between students’ success and the proportion of time spent viewing curved arrows. Previous research has shown the more successful problem solvers tend to spend less time overall working through a problem.²⁶  Although there appeared to be a slight trend in students who spent more time working through the problems being less successful, this was not significant. Additionally, students’ success at solving the problems did not appear to correlate with time spent viewing the curved arrows.

While students judged the plausibility of proposed mechanism steps, we characterized the types of features they focused on verbally and how that related to their visual attention on curved arrows. Students in the study described either explicit or implicit features (or both) in their reasoning. The language they used included either specific or general terminology (or both). Unsurprisingly, students spent more of their time viewing the reactants displayed than the curved arrows. A comparison of the proportion of time students spent viewing arrows to the types of features they discussed showed that students who spent more time viewing the arrows also tended to discuss some implicit features (such as identifying functional groups or partial charges) and use general terminology (such as talking about a functional group or nucleophile) while justifying their assessment of the plausibility of the reaction mechanism. This could imply that students who consider implicit features and general terminology might find arrows as useful representations to process these concepts. Alternatively, focusing on curved arrow representations might help students to consider implicit features and general terminology. They tended to follow the flow of electrons and verbally double check that the resulting structure would fulfill the octet rule. This could imply that curved arrows were a useful tool for students to think through this concept. Additionally, we found that successful students tended to also discuss implicit entities and general terminology (such as identifying that an acid, or nucleophile, or functional group was present). This aligns with previous studies that have reported students who focus on implicit features can be successful at problem solving.^14,33,40 

Although we identified relationships between the proportion of time students viewed curved arrows and the types of terminology (specific/general) and entities (implicit/explicit) they discussed (RQ1), we did not identify any significant relationships between the proportion of time students viewed curved arrows and the types of chaining reasoning (sequence without explanation, forward chaining, backward chaining) they used (RQ2). Since the products were not shown, this may have influenced the types of reasoning strategies students used.²⁵  Additionally, we did not identify any significant relationships between the proportion of time students viewed curved arrows and their success at judging the plausibility of these mechanism steps (RQ3). As with many eye-tracking studies (i.e., Williamson et al.)²⁶  the small size of the sample in this study may have limited the results. These results have implications for both future research and teaching. For teaching, having students practice reaction mechanisms with curved arrows may not be enough to develop their reasoning patterns. Instead, it might be more productive to focus on building additional conceptual scaffolding to help students think through the flow of electrons and their reasoning.³  Since we did not see a relationship between the students’ reasoning pattern and the proportion of time they spent viewing curved arrows, future studies could explore students’ viewing patterns of curved arrows and other features of a problem such as atoms, bonds, lone pairs, and charges to understand how students consider the directionality of curved arrows. A limitation of this current study could be the visual complexity of the selected problems. Future eye tracking research could explore student reasoning with problems with a range of visual complexity. Additionally, further research is needed that goes into more detail of visual features in a reaction mechanism as this study could only compare students’ view of arrows and the whole structure of a starting material. This study could not go into detail within a structure such as the types of atoms, bonds, lone pairs, charges students focused on.

We thank the participants of this study and the instructor of the course for providing access to participants, sharing their lecture notes, and reviewing the instrument. Additionally, we would like to thank Michael Franklin for providing feedback in the initial stages of the project.