A Votable Concept Mapping Approach to Promoting Students' Attentional Behavior: An Analysis of Sequential Behavioral Patterns and Brainwave Data.
IntroductionThe purpose of this study is to explore the effects of the integration of concept maps and classroom polling systems on students' learning performance, brainwaves associated with attention, and behavioral sequences. Concept mapping has been applied to a variety of educational settings, such as classroom teaching (Chiou, Lee, & Liu, 2012; Sun & Lee, 2016), designing digital teaching materials (Adesope & Nesbit, 2013), in-class teaching activities (Jones, Ruff, Snyder, Petrich, & Koonce, 2012), in-field inquiry activities (Hwang, Wu, & Ke, 2011), and online inquiry activities (Hwang, Kuo, Chen, & Ho, 2014). Researchers have indicated that concept maps can facilitate learners' reasoning ability (Mih & Mih, 2011); moreover, providing well-constructed concept maps during the learning process improves the accuracy of learners' understanding of the knowledge (Redford, Thiede, Wiley, & Griffin, 2012). On the other hand, researchers have further emphasized the importance of facilitating peer interactions (Sun, Chen, Yeh, Cheng, & Lin, 2018), which are usually ignored in most concept mapping activities. Using a polling tool in the classroom helps teachers to attract students' attention, increase their engagement level, and obtain information regarding their understanding of the subject. In addition, the application of the polling system facilitates peer interaction; learners are able to compare their answers with those of their classmates, discuss discrepancies, and reassess their own answers (Gachago, Morris, & Simon, 2011). Therefore, this study combines the concept-mapping technique and a polling system installed on tablets to help learners to strengthen the links between concept nodes through the introduction of interactive voting activities into the construction processes of concept maps.
It is generally considered that bringing sufficient attention to learning activities ensures meaningful learning. In the study of Hwang, Yang, and Wang (2013), researchers introduced the concept mapping technique to a game-based learning system and found that the teaching method resulted in learners becoming better focused and taking greater initiative in learning activities. Nesbit, Larios, and Adesope (2007) utilized eye movement tracking devices to examine the manner in which students read concept maps. The results proved that well-designed concept maps optimize the efficiency of attention allocation.
Sequential behavioral analysis is a behavior research method that utilizes encoded behaviors to investigate time-based behavioral patterns of individuals and groups (Bakeman & Gottman, 1997, p. 14). Hou (2012a) compiled a log of students' operations on a large-scale multi-person online educational gaming platform to analyze learners' knowledge construction, peer interaction, and problem-solving processes. Hou (2015) investigated learners' behavioral patterns and flow states in game-based learning to understand the patterns of their interactive behavior during the learning process. On that account, the sequential behavioral analysis technique could be used to explore students' interactive behavior in the class, thereby leading to an in-depth understanding of learners' knowledge construction processes. Therefore, this study adopted the sequential behavioral analysis method to explore learners' attentional behavior during the concept mapping process.
To summarize, this study introduced a classroom polling system to the construction of concept maps in order to stimulate peer interaction, reflection, and discussion, improve students' understanding of the course knowledge, and enhance their attention during learning. The research framework of the present study is shown in Figure 1. The research questions are as follows:
* Are there any significant differences in the academic performance of learners with different motivational traits when different conceptual mapping strategies and tools are applied?
* Are there any significant differences in the patterns of learners' attentional behaviors when different conceptual mapping strategies and tools are applied?
* Are there any significant differences in the brainwave readings associated with attention when different conceptual mapping strategies and tools are applied?
Literature review
Integrating conceptual maps and the polling system, and the role of learning motivation
A concept map is a graphical tool that illustrates the relationships between concepts. Concept maps use nodes (usually circles and squares) to represent concepts, links (connecting lines) to indicate the relations between nodes, and labels (texts and symbols) to describe their relationships (Novak, 1984). Through systematic induction and organization, concept maps transform large-scale, complex knowledge into a visual map to help learners better understand the meaning of each concept through the layers and links between the nodes (Blankenship & Dansereau, 2000; Novak, 1990).
Polling systems (also known as "clickers") refer to a small voting tool that is commonly used during teaching. Specifically, teachers first use presentation tools (such as MS PowerPoint) to introduce a question and corresponding options to the students, and then use a polling system to collect anonymous votes on each option on behalf of the students. Polling systems are used to analyze learners' understanding of the knowledge given; they also serve as an in-class quiz and assessment tool and record learners' attendance (Cheesman, Winograd, & Wehrman, 2010; Prather & Brissenden, 2009; Sun, Martinez, & Seli, 2014). Gachago et al. (2011) pointed out that the application of classroom polling systems is conducive to attracting learners' attention and enhances their engagement.
Many scholars have combined concept mapping and technological devices in their research and found that the combination had a positive effect on learning results. The study of Hwang et al. (2014) showed that a fill-in-the-blank concept map improved learners' problem-solving abilities. Hwang et al. (2011) applied concept mapping to an outdoor learning activity by presenting the teaching material with concept maps on mobile devices, which allowed the learners to draw and modify the concept maps. The results showed that the teaching method enhanced learners' comprehension of the knowledge. However, if learners were not familiar with concept maps, they tended to have difficulties understanding the structural relations between concepts, which thereby affected their learning motivation (Blankenship & Dansereau, 2000). Sun and Lee (2016) found that, compared with students with low learning motivation prior to the course, students with high learning motivation tended to develop greater motivation for learning after participating in a course that used tablet computers to construct concept maps. For that reason, participants' learning motivation before the in- class quiz was measured, and cluster analysis was applied to divide the participants into groups; a comparative analysis was then employed to investigate the differences in the learning achievements of the different groups.
Expectancy-value theory is one of the most important theories related to learning motivation. Expectancy refers to individuals' beliefs and judgments regarding their ability to successfully complete a given task. Value refers to the incentive that drives individuals to engage in the task (Schunk, Meece, & Pintrich, 2013). This study used participants' performance expectations of the in-class quiz as a proxy variable for their motivational traits. In addition, since we only needed to measure participants' performance expectations in the quiz, a single-item questionnaire based on the definition of expectancy proposed by Schunk et al. (2013) was adopted.
One other notable learning motivation related factor is anxiety. Evaluative environments and timed test-taking conditions (such as quizzes or exams) accentuate the detrimental effects of anxiety. The study of Tsai, Lin, and Yuan (2001) revealed that learners with greater test anxiety are more likely to prefer to use the developed fill-in-the-blank concept maps. Batchelor (2015) introduced clickers in a calculus course and discovered that learners' engagement in the learning process and their expectations of the examination results are powerful predictors of their math anxiety. According to the aforementioned studies, anxiety is an indicator of affectivity and is associated with performance expectation. Therefore, anxiety was included as a variable in the cluster analysis of motivational traits. Considering the context of the in-class quiz in this study, a single-item questionnaire based on the definition of test anxiety proposed by Zeidner (1998) was used to measure participants' test anxiety.
Attentional behavior and attention-associated brainwaves during a votable concept mapping activity
Engaging sufficient attention in learning tasks is essential to achieving meaningful learning. Existing empirical studies have revealed that concept mapping techniques and classroom polling tools have a positive effect on students' engagement and attention in learning. Hwang et al. (2013) found that combining concept maps and game-based teaching tends to make learners more attentive in class and to participate more actively in the learning activities. The study of Nesbit et al. (2007) applied eye-movement tracking devices and discovered that a well-designed concept map can improve the efficiency of attention, allowing learners to allocate attentional information resources from the higher part of the hierarchy and the center of the network to more effectively complete knowledge construction. Sun (2014) employed physiological equipment to trace learners' brainwaves, and found that their attention was significantly enhanced during the voting process. Therefore, this study designed votable concept maps to enhance learners' attentional behavior and attention-related brainwaves through the interaction of the voting and knowledge construction processes of concept mapping.
Physiological data, such as detecting electrical activity in the brain with electroencephalographic (EEG) devices (Rebolledo-Mendez et al., 2009), reflects learners' attention levels from an objective perspective. Referring to the review of McDowd (2007), this study attempts to measure learners' attention from both a behavioral and physiological approach so as to acquire an in-depth understanding of changes in their attention level. Specifically, all participants' behaviors during the class were recorded for further encoding, and the brainwaves of three participants were measured throughout the entire experiment process.
Sequential behavioral analysis is a commonly applied method to investigate learning behavior that reveals the behavioral patterns of individuals and groups based on the sequence of encoded behaviors (Bakeman & Gottman, 1997; Sun, Kuo, Hou, & Lin, 2017). The process of learning how to use concept maps is considered a knowledge construction behavior, and can be analyzed by sequential behavioral analysis to gain a better understanding of how concept mapping enhances learning. Hwang et al. (2011) investigated the effect of mobile-facilitated concept mapping strategies on students' learning achievements and attitudes in ecology courses. Hwang et al. (2014) developed a computer-supported concept map-based teaching system which improves learners' problem-solving ability. Sun and Chen (2016) incorporated dynamic concept maps into a polling system, and investigated its effect on students' learning motivation and achievement. However, the majority of the studies that combine concept maps and technological devices focus on the results of learning and learning motivation, so the actual sequence of students' attentional behaviors during the concept mapping processes has yet to be explored. Therefore, this study uses sequential behavioral analysis to investigate participants' attentional behaviors during the voting and concept mapping activities.
In summary, previous studies have revealed that both concept mapping and polling systems have positive effects on learning performance, while the integration of concept maps and technological devices can further improve learners' comprehension of the knowledge (Hwang et al., 2014; Hwang et al., 2011). For that reason, this study hypothesized that polling systems can stimulate interactions in learning so that learners are able to construct more complete concept maps and thereby improve the learning effectiveness. The study of Sun and Lee (2016) revealed that learners' motivational traits influence the effectiveness of teaching strategies for tablet-facilitated concept mapping. Therefore, we also included performance expectations and anxiety as proxy variables for learning motivation in our study. Furthermore, since polling systems stimulate interactions in learning, learners' attention during the concept map construction process could be enhanced, which may lead to increased attentional behavior and enhanced attention-related brainwaves.
Research methods
Participants
The participants of the study were 33 graduate students registered in an educational research methodology course. Their in-class quiz results, attentional behavior, and attention-related brainwaves during the research period were collected. Excluding the data of students who did not complete the three-week session or provide the required information, the data of 29 participants were retained for analysis. In addition, the brainwave readings of three participants were collected for data analysis to gain a deeper understanding of the brainwave variations over the 3 weeks. The three participants (two males and one female) were volunteers who had given their consent to the collection of their brainwave data. The participants were divided into groups of three to four during the voting activities. Among the 29 participants, nine were male (31%) and 20 were female (69%). The average age was 26.34 years old (SD = 5.88).
Instructional design
The study lasted 3 weeks; each week involved 100 minutes of classes. Each class included a lecture and two to three group activities. Three participants were selected and their brainwaves monitored throughout the entire 3week period. Twenty minutes before the end of the last class of each week, participants were asked to rate their feelings of anxiety and performance expectations regarding the upcoming quiz; an in-class closed-book quiz was then conducted. In addition, at the end of the 3-week period, nine participants were invited to participate individually in a 20-minute face-to-face structured interview.
Three group activities were implemented during the research period. Specifically, the voting activities and use of tablets were implemented in week 1; the pen-and-paper (PnP) concept mapping activities were adopted in week 2; and votable concept maps were introduced in week 3. The polling system application employed by the study was the "Interactive, Feedback-Based In-Class Teaching" (iFIT) system, which has been applied to classroom voting activities and achieved positive results prior to our study (Sun et al., 2018; Sun & Lee, 2016). In the voting activity, a tablet was provided to each group. After the instructor finished a section of the teaching material, he would present a question related to the knowledge taught on the screen for group discussion. The participants were asked to discuss the options and use the provided tablet and application to vote anonymously for the answer that they most preferred. The votes were then transmitted to the back of the instructor's stage area, the total for each answer calculated automatically by the polling system, and the results presented on the screen. The instructor would then reveal the correct answer, explain the reasons, and help students to resolve any misunderstandings or confusion.
In the PnP concept mapping activity, the instructor would also present a question on the screen after each section. The participants were required to exchange viewpoints and ideas with their group and construct a concept map using the pen and paper provided to each group. Next, each group would take a photo of the constructed concept map with the tablet provided and send the photo to the instructor. The instructor would then display the concept map of each group on the screen, discuss the strengths and weaknesses of each map, and resolve any misunderstandings and errors the class might have. In the votable concept mapping activity, after each section, the instructor would present the question in the format of a fill- in-the-blank concept map with corresponding options. The participants were required to choose the best answer for each blank following a group discussion. The voting results would be presented to the class and the instructor would reveal the correct answer, explain the reasons, and resolve any misunderstandings or confusion related to the concepts. The questions used in the group discussion over the 3 weeks are shown in Figure 2. In order to avoid the carry-over effect, the contents of the three units ("threats to validity," "experimental design," and "research method") were independent, with no sequenced relationship between two units. The difficulty level of the quizzes was approximately equal. The procedure of the experiment is presented in Figure 3 and photos taken during the experiment are shown in Figure 4.
Instruments and Analysis
The item used to measure performance expectations was phrased as follows: "In the upcoming quiz, I expect my score to be--(from 0 to 100)." The item used to measure anxiety was written as follows: "From 0 to 100, what is your anxiety level regarding the upcoming quiz?" The in-class quizzes were provided by the instructor and included multiple-choice, fill-in-the-blank, and short-answer questions. Due to differences in the rated scores, the Z-scores of performance expectations and anxiety of the participants were computed, and cluster analysis was then applied to divide the participants into groups. In the first stage of the cluster analysis, Ward's method was used to determine the optimal number of clusters. In the second stage, the k-means clustering technique was applied to determine the allocation of participants among the groups, and yielded three groups: the "high-expectancy and high-anxiety group" (Group 1), the "medium-expectancy and low-anxiety group" (Group 2), and the "low-expectancy and medium-anxiety group" (Group 3); the number of participants in each group was 7, 9, and 13, respectively. Next, we applied Analysis of Variance (ANOVA) to test the validity of the clustering results. The results showed significant differences in both the performance expectations (F = 18.11, p < .001) and anxiety (F = 20.26, p < .001) of the three groups. The means of performance expectations and anxiety of Group 1 were 81.86 (SD = 9.70) and 85.71 (SD = 11.34), respectively; those of Group 2 were 74.78 (SD = 8.97) and 42.22 (SD = 17.87), respectively; and those of Group 3 were 57.69 (SD = 9.27) and 71.92 (SD = 12.84), respectively. The findings revealed distinctive differentiations in the motivational traits of the three groups, indicating that the clustering results had satisfactory validity. Due to the small size of the research sample, we adopted the Kruskal-Wallis H test, a nonparametric statistical method, instead of ANOVA to examine the significance level of the dissimilarities in the quiz results of the three groups.
We outlined the coding scheme of the attentional behavior of the participants. According to the learning tasks and required attention in the experiment, the behaviors were coded into three categories: "attentional behavior related to the course," "attentional behavior related to the activity," and "distracted behavior." The detailed coding method of the behaviors is shown in Table 1. Three observers were in charge of the behavior coding process by observing participants' behaviors recorded during the research period. The coded behaviors of three students during the polling system week were used to examine the reliability of the encoding process. Fleiss' kappa is a statistical indicator of the reliability of agreement between three raters when handling nominal scales (Fleiss, 1971). The Fleiss' kappa of the encoding process of the study was 0.65, suggesting that the consistency between the raters was substantial (Landis & Koch, 1977, p. 165).
Brainwave signals of three students were measured to monitor their attention level during the classes. NeuroSky MindWave headsets were used in this study, as they have been proven able to effectively monitor users' attention levels through brainwave signals (Crowley, Sliney, Pitt, & Murphy, 2010; Rebolledo-Mendez et al., 2009). Based on the nature of the teaching activities, we divided the entire course into three stages, classroom lectures, group activities, and explaining the answers after the activity, and analyzed the EEG data at each stage accordingly. We used the first 3 minutes of the EEG data of each participant as the baseline for their attention levels, then calculated the percentage of the attention levels that exceeded the baseline and plotted the EEG diagrams for the selected participants accordingly. Lastly, we combined the EEG observations with the results of the sequential analysis of attentional behavior to further explain the variations in brainwaves.
Results
Comparison of the quiz results of the three groups
The Kruskal-Wallis test results showed no significant differences in the quiz results of the three groups during the polling system week ([chi square] = 3.09, p = .21). However, significant differences were found in the quiz results of the three groups in the PnP concept map week ([chi square] = 6.56, p = .04). A post-hoc comparison revealed that the mean rank of quiz scores of Group 3 (12.46) was significantly better than that of Group 1 (12.21). The quiz results of the three groups in the votable concept map week showed no significant differences ([chi square] = 1.23, p = .54).
Sequence of attention behavior over the 3 weeks
The adjusted residuals (Z-scores) of the originally coded behavioral data are presented in Table 2. In the table, the initial behaviors are listed in the second column of each row, and subsequent behaviors in the first row of each column. If the Z-score between two behaviors was greater than 1.96, then the sequential relationship between the behaviors can be considered statistically significant (p < .05) (Bakeman & Gottman, 1997; Hou, 2012b). The sequence patterns of participants' attentional behaviors during the 3 weeks are demonstrated in Figure 5.
As shown in the figure, in the polling system week, participants were found to be engaged in group discussion after the voting activities, and the behavioral sequence of "discussing following voting" (3 [right arrow] 4) reached statistical significance. However, the behavioral sequence of "voting following discussion" (4 [right arrow] 3) was not statistically significant. Since during the PnP concept map week the tablets were used to take and send photos rather than being used as voting instruments, no significant sequential relations were found between interaction with group members (4) and interaction with the tablet (3). Instead, attention to the concept map (5) and interaction with group members (4) were found to have significant sequential relations in both directions (4 [right arrow] 5 and 5 [right arrow] 4). In addition, the participants were found to show attention to the instructors' explanation of the answer and analysis of the concept map constructed by each group (3 [right arrow] 6), and the sequential relationship was statistically significant. However, participants were also found to be shifting between listening to the instructor and other distracted behaviors (6 [right arrow] 9, 9 [right arrow] 6), with notable distracted behaviors, such as looking around the room, staring blankly into space, and looking for irrelevant items. The behavioral sequence of "discussing the following votes" (3 [right arrow] 4) was also found to be statistically significant in the votable concept map week. Moreover, interaction with the tablet (3) and interaction with group members (4) were both found to lead to attention paid to the instructor's explanation of the answers (3 [right arrow] 6 and 4 [right arrow] 6), while listening to the instructor's explanation of the answers (6) was also found to effectively enhance the interaction with the tablet and group members in the next activity (6 [right arrow] 3 and 6 [right arrow] 4). The attentional behaviors during the activity and following the activity in the votable concept map week appeared to be able to facilitate one another, thus forming a virtuous cycle.
Differences in the brainwave signals over the 3 weeks
Table 3 and Figures 6 and 7 illustrate the percentage of the brainwave signals collected from the three participants (A, B, and C) that exceeded the baseline (hereinafter, attention indicator). Figure 6 is a comparison of the week-on-week changes in the attention indicator. It can be seen from the figure that A's and C's attention indicators during the classroom lectures were the highest in the PnP concept map week (50% greater than the baseline), while B's attention indicator was the highest in the polling system week (60% greater than the baseline). In the group activities, the attention indicators of A, B, and C were the highest in the PnP concept map week, the votable concept map week, and the polling system week, respectively. When the instructor was explaining the answers following the activities, the attention indicators of A and B were highest in the PnP concept map week, while that of C was highest in the polling system week. From the perspective of the entire class, the attention indicators of A and C were highest in the PnP concept map week and that of B was highest in the polling system week. Figure 7 compares the three participants' attention indicators at different stages of the course in the same week. In the polling system week, the attention indicators of A and B were highest in classroom lectures (30% and 50% greater than the baseline, respectively), while that of C was highest when the instructor was explaining the answers after the activities (70% greater than the baseline). In the PnP concept map week, A's attention indicator was highest during the classroom lectures, B's during the group activities, and C's when the instructor explained the answers after the activities. In the votable concept map week, both A's and B's attention indicators were highest during the group activities, while C's remained highest when the instructor explained the answers after the activities.
Figure 8 illustrates the changes in B's attention indicator during the teaching activities in the polling system and votable concept map weeks. It can be seen from the figure that in the polling system week, this indicator was unstable during the group activities as well as when the instructor was explaining the answers after the activities. In the votable concept map week, fewer fluctuations in the indicator were observed during the group activities and when the instructor was explaining the answers after the activities, indicating that B's attention level was relatively more stable in the votable concept map week than in the polling system week.
In general, the participants' attention level was highest in the PnP concept map week. Participants were found to invest more attention in the lecture and the explanations after the group activity, when a PnP concept mapping activity was implemented. In weekly terms, the participants' attention levels were highest during the classroom lecture stage in the polling system week and the group activity stage in the votable concept map week; however, no specific stage received additional attention from the participants in the PnP concept map week. In addition, compared with the voting activities in the polling system week, participants showed a more stable level of attention during the votable concept mapping activities, as well as when the instructor was explaining the answers after the activities in the votable concept map week (Figure 8).
Discussion
The purpose of this study was to integrate the concept mapping technique and polling system as a teaching strategy and explore their impact on in-class quiz results, attentional behavior, and brainwaves associated with the attention of students with various motivational traits. The results showed that the mean rank of the quiz results of the low-expectancy and medium anxiety group (12.46) was significantly better than that of the high-expectancy and high-anxiety group (12.21). Past research has confirmed that combining concept mapping techniques and technological devices that provide learners with the opportunity to construct concept maps in a fill-in-the-blank format can improve learners' problem-solving ability and enhance their comprehension of the knowledge (Hwang et al., 2014; Hwang et al., 2011). However, the study of Sun and Lee (2016) suggested that, compared with concept mapping activities that use tablet computers, concept mapping activities that use pen and paper are able to significantly improve the post-test motivation of learners with lower initial motivation. Similar results were obtained in this study: The PnP concept mapping activity was found to be able to improve the quiz scores of participants from the low-expectancy and medium anxiety group. Nesbit et al. (2007) found that concept maps could help learners to improve the utility of attentional resources and construction of knowledge. Therefore, this study speculated that the results could be attributed to the scaffolding structure of concept maps, which can help learners with low self-confidence and a certain level of anxiety allocate their attentional resources more efficiently. Thus, they are able to participate in the group discussions more actively and construct concept maps with other group members. As a result, their learning performance was improved.
The results of this study revealed that during the first week, when the voting activities were introduced, there was a significant sequential relation between voting and group discussion ("discussing after voting"), and the brainwave readings showed that classroom lectures received the highest level of attention. During the second week, when the PnP concept mapping activities were implemented, an apparent repetition of two behaviors, "discussing with group members" and "constructing concept maps," was observed, showing that the concept maps were completed through multiple discussions among group members. In addition, after taking photographs of the constructed concept maps and sending them to the instructor, participants were found to pay greater attention to the instructor's explanation of the answers. However, while listening to the instructor's explanation, participants also manifested "other distracted behaviors." The brainwave readings showed that classroom lectures received the highest level of attention. During the third week, when votable concept mapping activities were applied, the behavioral sequence of "discussing after voting" was observed again. In addition, interactions with both the tablet and group members were found to lead to greater attention shown to the instructor's explanation after the activities. More importantly, it was observed that listening to the instructor's explanation tended to lead participants to interact with the tablet and group members in the next activity. These findings show that votable concept maps can stimulate learners' interactions and enhance their attention to the instructor's explanations, forming a virtuous cycle of learning behavior. The brainwave readings showed that group activities received the highest level of attention throughout the week.
Sun (2014) discovered that learners' attention-related brainwaves were significantly enhanced when participating in activities involving interactive response systems (IRSs). The same result was also reached in this study. To quote participant B's comments on the votable concept maps during the interview, "... and then answer the question proposed by the instructor; for this stage, since there were images to supplement the information previously given by the instructor, [the concept of] this part became quite clear [to me]." It can be seen that a votable concept map can clearly present the knowledge structure and enhance learners' understanding of the topics, and that participants' attention-related brainwaves were significantly strengthened during the voting activity. One possible reason for this result is that the votable concept map can stimulate learners' attention-related brain regions, which leads to their attentional behavior when listening to the instructor's explanation and more active engagement in interactive discussions of the next activity. Hwang et al. (2013) discovered that combining concept maps and game-based teaching could effectively enhance learners' learning achievement and mental effort in learning activities.
The findings in the PnP concept mapping activities of this study were the same. During the group activities, the learners were found to be switching between drawing concept maps and discussing with group members. However, it was also found that after the concept mapping activity had finished, learners manifested noticeable distracted behaviors, such as looking around the room, staring blankly into space, and looking for irrelevant items, when the instructor was explaining the answers. The results of the interviews revealed that participants were not satisfied with the clarity of the projected concept maps constructed by each group. For example, some participants claimed that "the instructor seemed to have presented ours on the screen, however, since the words are really small, we cannot see what they are about ..."; "because the photos are not clear; [I] hope that other methods can be used [in the future]"; and "the concept maps drawn by us are not clear." Thus, one likely explanation for the distracted behaviors after the group activity observed in the PnP concept map week is the lack of clarity of the concept maps projected on the screen, which led to the learners experiencing difficulties concentrating on the screen, thus leading to their distracted behaviors.
Conclusion and future research
This study investigated the effects of integrating concept maps and a polling system in teaching on learners' quiz results, attentional behavior, and brainwaves associated with attention. The conclusions of the study are as follows: (1) When PnP concept mapping activities were implemented, the mean rank of the participants' quiz scores from the low-expectancy and medium-anxiety groups were significantly better than that from the high-expectancy and high-anxiety group. (2) When PnP concept mapping activities were carried out in class, the learners were found to be highly attentive during the interactive activities; however, their attention was distracted when the instructor was explaining the answers. (3) During the class with the use of votable concept mapping activities, the attention level measured through brainwave signals was the greatest compared with all group activities. (4) During the class with the application of votable concept mapping activities, learners' attentiveness to the discussions of one activity tended to lead to attentive behavior when the instructor was explaining the answers, which was likely to lead to their attentive behavior in the next discussion of the next activity, forming a virtuous behavioral cycle. In short, PnP concept mapping activities could improve the learning performance of students with low learning motivation, leading to attentional behavior and active participation in discussion and interaction. However, learners might be distracted when the instructor starts to explain the answers after the activities. Votable concept mapping activities, on the other hand, are not only conducive to promoting attentional behavior during learning activities, but also encourage learners to concentrate on the instructor's explanation following the activities. On that account, votable concept mapping effectively enhances learners' attentional behavior prior to and following the activities.
The limitations of this study include the small sample size (students from only one class were recruited as participants), the short research period, the limited number of participants used for brainwave data collection (only three students), and the limitations of the reliability and validity of the research instruments. Therefore, the findings of this study should be used with caution. We suggest that further studies expand the size of the research sample and extend the research period to improve the reliability of the results. In addition to introducing a control group in the research design, future studies can also attempt to use a counterbalanced design that randomly assigns participants into groups to minimize the carry-over effect and achieve more distinguishable results by implementing various votable concept-mapping activities in different orders. In future research, it is recommended that researchers include a preliminary test, so that the participants' prior knowledge regarding each subject may be included in the analysis. Future studies involving votable concept maps are suggested in order to introduce more comprehensive measurement scales that can examine learners' motivation and learning performance, so as to better understand the influence of the teaching method on the learners' performance and different motivational traits. In terms of experimental devices, future studies should increase the number of wearable brainwave headsets so as to collect EEG data from more learners and conduct a more in-depth analysis of changes in their attention levels. Lastly, further studies can also incorporate group competition activities to stimulate learners' participation and concentration and thereby enhance their attentional behavior in the learning process after the activities have been completed.
Acknowledgements
This research was supported by the Ministry of Science and Technology in Taiwan through Grant numbers MOST 105-2511-S-009-013-MY5 and MOST 104-2511-S-009-008- MY3. The authors would like to thank the students who participated in this study, Katherine Pin-Chen Yeh, Afra Lin, Hsueh-Er Tsai, Chih-Hsuan Chao, and Yen-Ling Chen for their assistance in conducting this research, Shu-Kai Yang for his support for the smooth implementation of the study's iFIT app, and all of the reviewers for their useful comments.
Reference
Adesope, O. O., & Nesbit, J. C. (2013). Animated and static concept maps enhance learning from spoken narration. Learning and Instruction, 27, 1-10. doi:10.1016/j.learninstruc.2013.02.002
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An Introduction to sequential analysis (2nd ed.). New York, NY: Cambridge University Press.
Batchelor, J. (2015). Effects of clicker use on calculus students' mathematics anxiety. PRIMUS, 25(5), 453-472.
Blankenship, J., & Dansereau, D. F. (2000). The Effect of animated node-link displays on information recall. The Journal of Experimental Education, 68(4), 293-308. doi:10.1080/00220970009600640
Cheesman, E. A., Winograd, G. R., & Wehrman, J. D. (2010). Clickers in teacher education: Student perceptions by age and gender. Journal of Technology and Teacher Education, 18(1), 35-55.
Chiou, C. C., Lee, L. T., & Liu, Y. Q. (2012). Effect of Novak colorful concept map with digital teaching materials on student academic achievement. Procedia--Social and Behavioral Sciences, 64, 192-201.
Crowley, K., Sliney, A., Pitt, I., & Murphy, D. (2010, July). Evaluating a brain-computer interface to categorise human emotional response. Paper presented at the 2010 IEEE 10th International Conference on Advanced Learning Technologies (ICALT), Sousse, Tunisia.
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378-382.
Gachago, D., Morris, A., & Simon, E. (2011). Engagement levels in a graphic design clicker class: Students' perceptions around attention, participation and peer learning. Journal of Information Technology Education: Research, 10(1), 253-269.
Hou, H.-T. (2012a). Analyzing the learning process of an online role-playing discussion activity. Educational Technology & Society, 15(1), 211-222.
Hou, H.-T. (2012b). Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG). Computers & Education, 58(4), 1225-1233. doi:10.1016/j.compedu.2011.11.015
Hou, H.-T. (2015). Integrating cluster and sequential analysis to explore learners' flow and behavioral patterns in a simulation game with situated-learning context for science courses: A Video-based process exploration. Computers in Human Behavior, 48, 424-435. doi:10.1016/j.chb.2015.02.010
Hwang, G.-J., Kuo, F. R., Chen, N. S., & Ho, H. J. (2014). Effects of an integrated concept mapping and web-based problem-solving approach on students' learning achievements, perceptions and cognitive loads. Computers & Education, 71, 77-86.
Hwang, G.-J., Wu, P. H., & Ke, H. R. (2011). An Interactive concept map approach to supporting mobile learning activities for natural science courses. Computers & Education, 57(4), 2272-2280.
Hwang, G.-J., Yang, L.-H., & Wang, S.-Y. (2013). A Concept map-embedded educational computer game for improving students' learning performance in natural science courses. Computers & Education, 69, 121-130.
Jones, B. D., Ruff, C., Snyder, J. D., Petrich, B., & Koonce, C. (2012). The Effects of mind mapping activities on students' motivation. International Journal for the Scholarship of Teaching and Learning, 6(1), 1-21.
Landis, J. R., & Koch, G. G. (1977). The Measurement of observer agreement for categorical data. International Biometric Society, 33(1), 159-174.
McDowd, J. M. (2007). An Overview of attention: Behavior and brain. Journal of Neurologic Physical Therapy, 31(3), 98-103.
Mih, C., & Mih, V. (2011). Conceptual maps as mediators of self-regulated learning. Procedia--Social and Behavioral Sciences, 29, 390-395.
Nesbit, J. C., Larios, H., & Adesope, O. O. (2007, June). How students read concept maps: A Study of eye movements. Paper presented at the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (ED-MEDIA 2007), Vancouver BC, Canada.
Novak, J. D. (1984). Learning how to learn. New York, NY: Cambridge University Press.
Novak, J. D. (1990). Concept maps and vee diagrams: Two metacognitive tools to facilitate meaningful learning. Instructional Science, 19(1), 29-52. doi:10.1007/BF00377984
Prather, E. E., & Brissenden, G. (2009). Clickers as data gathering tools and students' attitudes, motivations, and beliefs on their use in this application. Astronomy Education Review, 8(1), 01010301- 01010310. Retrieved from http://access.portico.org/Portico/#!journalAUSimpleView/tab=PDF?cs=ISSN_15391515 ?ct=EJournal%20Content?auId=ark:/27927/pgg3ztfc67p
Rebolledo-Mendez, G, Dunwell, I., Martinez-Miron, E. A., Vargas-Cerdan, M. D., De Freitas, S., Liarokapis, F., & Garcia-Gaona, A. R. (2009, July). Assessing neurosky s usability to detect attention levels in an assessment exercise. Paper presented at the International Conference on Human-Computer Interaction, San Diego, CA.
Redford, J. S., Thiede, K. W., Wiley, J., & Griffin, T. D. (2012). Concept mapping improves metacomprehension accuracy among 7th graders. Learning and Instruction, 22(4), 262-270.
Schunk, D. H., Meece, J. L., & Pintrich, P. R. (2013). Motivation in education: Theory, research, and applications. Upper Saddle River, NJ: Pearson.
Sun, J. C.-Y. (2014). Influence of polling technologies on student engagement: An Analysis of student motivation, academic performance, and brainwave data. Computers & Education, 72, 80-89. doi:10.1016/j.compedu.2013.10.010
Sun, J. C.-Y., & Chen, A. Y.-Z. (2016). Effects of integrating dynamic concept maps with Interactive Response System on elementary school students' motivation and learning outcome: The Case of anti- phishing education. Computers & Education, 102, 117-217. doi: 10.1016/j. compedu.2016.08.002
Sun, J. C.-Y., Chen, A. Y.-Z., Yeh, K. P.-C., Cheng, Y.-T., & Lin, Y.-Y. (2018). Is group polling better? An Investigation of the effect of individual and group polling strategies on students' academic performance, anxiety, and attention. Educational Technology & Society, 21(1), 12-24.
Sun, J. C.-Y., Kuo, C.-Y., Hou, H.-T., & Lin, Y.-Y. (2017). Exploring learners' sequential behavioral patterns, flow experience, and learning performance in an anti-phishing educational game. Educational Technology & Society, 20(1), 45-60.
Sun, J. C.-Y., & Lee, K.-H. (2016). Which teaching strategy is better for enhancing anti-phishing learning motivation and achievement? The Concept maps on tablet PCs or worksheets? Educational Technology & Society, 19(4), 87-99.
Sun, J. C.-Y., Martinez, B., & Seli, H. (2014). Just-in-time or plenty-of-time teaching? Different electronic feedback devices and their effect on student engagement. Educational Technology & Society, 17(2), 234-244.
Tsai, C. C., Lin, S. S., & Yuan, S. M. (2001). Students' use of web-based concept map testing and strategies for learning. Journal of Computer Assisted Learning, 17(1), 72-84.
Zeidner, M. (1998). Test anxiety: The State of the art. New York, NY: Springer Science & Business Media.
Jerry Chih-Yuan Sun (1) *, Gwo-Jen Hwang (2), Yu-Yan Lin (1), Shih-Jou Yu (1), Liu-Cheng Pan (1) and Ariel Yu-Zhen Chen (1)
(1) Institute of Education, National Chiao Tung University, Taiwan // (2) Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, Taiwan // [email protected] // [email protected] // [email protected] // [email protected] // [email protected] // [email protected]
* Corresponding author
Caption: Figure 1. Research framework
Caption: Figure 2. Polling questions used for group discussion
Caption: Figure 3. Experimental design
Caption: Figure 5. Sequence patterns of participants' attentional behavior during the 3 weeks
Caption: Figure 6. Week-by-week comparison of three participants' attention indicators during each course stage
Caption: Figure 7. Comparison of three participants' attention indicators during all course stages in each week
Caption: Figure 8. Changes in participant B's attention levels during teaching activities in the polling system and votable concept map weeks
Table 1. Coding scheme of attentional behavior
Code Definition Examples
Theme A: Attentional Behavior Related to the Course
1 Behaviors that show Looking at the
attention to the instructor,
instructor listening while the
instructor is
talking
2 Behaviors that show Checking the
attention to the teaching material,
teaching materials taking notes
Theme B: Attentional Behavior Related to the Activity
3 Behaviors involving Using the tablet to
interaction with the answer questions and
tablet research information
4 Behaviors involving Talking and
interaction with discussing with
group members group members
5 Behaviors that show Constructing and
attention to the modifying the
concept map concept map on paper
(referring to the
activity in the week
of PnP concept map)
6 Behaviors that show Listening to the
attention to the instructor's
instructor's explanation,
explanation of the checking the
answers textbook, checking
the teaching
materials, looking
at the presentation
Theme C: Distracted Behavior
7 Distracted behaviors Using any irrelevant
involving digital digital devices in
devices class (such as smart
phones and MP3s)
8 Distracted behaviors Chatting with
involving classmates classmates on
irrelevant topics
9 Other distracted Looking around the
behaviors room (not at the
textbook and
teaching material),
staring blankly into
space, searching for
irrelevant items
Table 2. Z-Score table of participants' attentional behavior
during the 3 weeks
Weeks 1 2 3 4 5
Polling 1 115.77 -76.14 -25.12 -33.26 This
System 2 -63.65 117.77 -19.13 -23.74 code
3 -15.95 -14.16 115.55 3.34 * only
4 -21.00 -18.07 0.95 116.15 appears
6 -25.61 -22.37 -6.17 -8.27 in the
7 -3.81 -6.61 -1.78 -2.53 PnP
8 -11.48 -10.20 -3.48 -5.86 concept
9 -12.46 -11.95 -4.02 -3.10 map
week
PnP 1 95.15 -43.20 -8.11 -40.36 -17.23
Concept 2 -44.45 100.87 -8.80 -43.75 -18.41
Map 3 -5.67 -5.07 98.13 -0.53 -1.34
4 -30.21 -31.45 1.80 124.78 7.76 *
5 -11.46 -11.61 0.17 5.60 * 112.69
6 -24.96 -25.12 0.12 -6.64 -5.69
7 -5.55 -6.56 -0.78 -3.63 -1.48
8 -4.68 -9.29 -1.78 -8.04 -3.38
9 -7.11 -13.09 -2.31 -5.77 -4.39
Votable 1 101.25 -55.79 -19.01 -23.05 This
Concept 2 -50.00 104.47 -19.27 -20.20 code
Map 3 -15.17 -14.37 102.76 8.76 * only
4 -15.96 -15.19 -0.13 94.69 appears
6 -24.18 -22.17 3.64 * 3.27 * in the
7 -7.55 -8.99 -3.17 -3.55 PnP
8 -15.55 -16.69 -6.94 -7.83 concept
9 -13.95 -13.56 -5.62 -3.66 map
week
Weeks 6 7 8 9
Polling 1 -44.75 -6.44 -19.63 -20.01
System 2 -32.28 -9.24 -13.54 -18.19
3 -2.86 -2.28 -3.86 -4.22
4 -1.37 -3.02 -5.92 -3.67
6 126.99 -3.82 -7.03 -7.00
7 -3.26 95.55 -1.77 -1.20
8 -7.02 -2.24 85.94 0.30
9 -6.56 -0.80 -1.22 96.05
PnP 1 -35.73 -8.67 -9.14 -8.57
Concept 2 -38.14 -10.36 -13.30 -21.56
Map 3 3.01 * -0.78 -1.73 -2.21
4 -4.88 -4.03 -8.85 -6.14
5 -4.78 -1.49 -3.34 -3.77
6 122.77 -3.30 -6.89 3.53 *
7 -3.07 136.20 -1.91 -2.44
8 -6.72 -1.96 80.57 -0.62
9 2.34 * -2.11 1.50 64.70
Votable 1 -39.39 -11.15 -22.87 -21.74
Concept 2 -33.86 -14.71 -24.79 -18.99
Map 3 6.30 * -3.22 -6.78 -4.56
4 12.80 * -3.62 -6.71 -2.93
6 113.56 -5.29 -10.07 -7.08
7 -5.16 110.66 -3.99 -1.16
8 -10.04 -3.59 99.15 -7.25
9 -6.37 -1.46 -5.68 100.65
Note. 1 = Behaviors that show attention to the instructor; 2 =
behaviors that show attention to the teaching materials; 3 =
behaviors involving interaction with the tablet; 4 = behaviors
involving interaction with group members; 5 = behaviors that show
attention to the concept map; 6 = behaviors that show attention to
the instructor's explanation of the answers; 7 = distracted
behaviors involving digital devices; 8 = distracted behaviors
involving classmates; and 9 = other distracted behaviors. * p <
.05.
Table 3. Frequency of participant attention greater than the baseline
Stages of the course
Participants Weeks Classroom Group
Lecture Activity
A Polling System 34.43% 26.27%
PnP Concept Map 73.46% 49.83%
Votable Concept Map 16.56% 31.29%
B Polling System 52.90% 46.24%
PnP Concept Map 33.20% 48.40%
Votable Concept Map 35.41% 54.26%
C Polling System 53.03% 61.83%
PnP Concept Map 63.31% 41.12%
Votable Concept Map 46.70% 40.36%
Stages of the course
Participants Weeks Explaining the
Answers after the
Activity
A Polling System 10.32%
PnP Concept Map 41.18%
Votable Concept Map 26.32%
B Polling System 36.52%
PnP Concept Map 38.46%
Votable Concept Map 29.41%
C Polling System 73.33%
PnP Concept Map 70.08%
Votable Concept Map 61.39%
Stages of the
course
Participants Weeks The Whole
Course
A Polling System 31.64%
PnP Concept Map 60.63%
Votable Concept Map 20.90%
B Polling System 49.26%
PnP Concept Map 35.88%
Votable Concept Map 36.60%
C Polling System 57.62%
PnP Concept Map 62.62%
Votable Concept Map 46.72%
Figure 4. Photos taken during the experiment
Polling System
Group discussion during the voting
activity
PnP Concept Map
Drawing a concept map with pen on a
piece of paper, taking a photo, and
sending it to the instructor.
Votable Concept Map
The instructor explaining a question used
for the Votable Concept Map activity
Polling System
The instructor presenting the voting
results and explaining the correct answer
to a question.
PnP Concept Map
The instructor discussing and evaluating
the answers sent by students in the PnP
Concept Map activity with the class.
Votable Concept Map
Voting for the answer to be filled m in
the concept map
 Printer friendly
 Cite/link
 Email
 Feedback
| |
| Author: | Sun, Jerry Chih-Yuan; Hwang, Gwo-Jen; Lin, Yu-Yan; Yu, Shih-Jou; Pan, Liu-Cheng; Chen, Ariel Yu-Zhen |
|---|---|
| Publication: | Educational Technology & Society |
| Article Type: | Report |
| Date: | Apr 1, 2018 |
| Words: | 8039 |
| Previous Article: | How Competition in a Game-based Science Learning Environment Influences Students' Learning Achievement, Flow Experience, and Learning Behavioral... |
| Next Article: | Using a Learner-Topic Model for Mining Learner Interests in Open Learning Environments. |
| Topics: | |