Analyzing the Discourse of Geogebra Collaborations

This is a position paper presenting a perspective on fundamental assumptions about doing, teaching and learning mathematics in the presence of computer and communicative technologies. Doing, teaching and learning mathematics are activities that centrally involve discourse. Computer and communication technologies can facilitate collaborative interactions around mathematical topics. This can make the processes of doing, teaching and learning mathematics visible to researchers in the traces of small-group interaction. Analysis of the discourse can reveal processes of mathematical group cognition. We argue for a view of mathematics as discourse and for a specific set of complementary approaches to analyzing collaborative math discourse. 1. MATH AS DISCOURSE For most non-mathematicians, arithmetic provides their paradigm of math. Learning math, they assume, involves memorizing facts like multiplication tables and procedures like long division. But for mathematicians, math is a matter of defining new concepts and arguing about relations among them. Math is a centuries-long discourse, with a shared vocabulary, ways of symbolically representing ideas and procedures for defending claims. It is a discourse and a set of shared practices. Learning to talk about math objects, to appreciate arguments about them and to adopt the practices of mathematical reasoning constitute an education in math (Livingston, 1999; Sfard, 2008). To mathematicians since Euclid, math represents the paradigm of creative intellectual activity. Its methods set the standard throughout Western civilization for rigorous thought, problem solving and argumentation. We teach geometry to instill in students a sense of deductive reasoning. Yet, too many people end up saying that they “hate math” and that “math is boring” or that they are “not good at math” (Boaler, 2008; Lockhart, 2009). They have somehow missed the true experience of math cognition—and this may limit their lifelong interest in science, engineering and technology. According to a recent “cognitive history” of the origin of deduction in Greek mathematics (Netz, 1999), the primordial math experience in 5 and 4 Century BC was based on the confluence of labeled geometric diagrams (shared visualizations) and a language of written mathematics (asynchronous collaborative discourse), which supported the rapid evolution of math cognition in a small community of math discourse around the Mediterranean that profoundly extended mathematics and Western thinking. GeoGebra
NA2010
 July
27‐28
2010
 Ithaca
College,
Ithaca,
NY,
USA
 The vision behind our Virtual Math Teams (VMT) Project (Stahl, 2009b) is to foster communities of math discourse in online communities around the world. We want to leverage the potential of networked computers and dynamic math applications to catalyze groups of people exploring math and experiencing the intellectual excitement that Euclid’s colleagues felt—leveraging emerging 21 Century media of shared math visualization and collaborative math discourse. Classical training in school math—through drill in facts and procedures—is like learning Latin by memorizing vocabulary lists and conjugation tables: one can pass a test in the subject, but would have a hard time actually conversing with anyone in the language. To understand and appreciate the culture of mathematics, one has to live it and converse with others in it (Papert, 1980). Math learners have to understand and respond appropriately to mathematical statements by others and be able to critically review and constructively contribute to their proposals. The VMT Project is designed to create worlds and communities in which math can be lived and spoken. The learning sciences have transformed our vision of education in the future (Sawyer, 2006; Stahl, Koschmann & Suthers, 2006). New theories of mathematical cognition and math education, in particular, stress collaborative knowledge building, problem-based learning, dialogicality, argumentation, accountable talk, group cognition (Stahl, 2006) and engagement in math discourse. These approaches place the focus on problem solving, problem posing, exploration of alternative strategies, inter-animation of perspectives, verbal articulation, argumentation, deductive reasoning, and heuristics as features of significant math discourse (Powell, Francisco & Maher, 2003). By articulating thinking and learning in text, they make cognition public and visible. This calls for a reorientation to facilitate dialogical student practices as well as requiring content and resources to guide and support the student discourses. Teachers and students must learn to adopt, appreciate and take advantage of the visible nature of collaborative learning. The emphasis on text-based collaborative learning can be well supported by computers with appropriate computer-supported collaborative learning (CSCL) software. Students learn math best if they are actively involved in discussing math. Explaining their thinking to each other, making their ideas visible, expressing math concepts, teaching peers and contributing proposals are important ways for students to develop deep understanding and real expertise (Cobb, 1995). There are few opportunities for such student-initiated activities in most teacher-centric classrooms. The VMT chat room provides a place for students to build knowledge about math issues together through intensive, engaging discussions. Their entire discourse and graphical representations are persistent and visible for them to reflect on and share. 2. COLLABORATIVE MATHEMATICS The argument against discourse-based approaches to math education—like inquiry learning and collaborative learning—is generally that students must first learn the basic facts before they can speculate on their own. The major worry expressed about learning through peer discourse is that the group of students will come up with the wrong answer or an incorrect theory. The proposed solution is that education must “go back to the basics” and focus on delivering the basic facts of each field to all the students first, and then, if there is time left over, allow students to discuss their own ideas based on the GeoGebra
NA2010
 July
27‐28
2010
 Ithaca
College,
Ithaca,
NY,
USA
 foundation of knowledge of these facts. Mathematics is taken as the clearest example of this argument. Make sure that students have memorized their number facts first, then drill them on applying algorithmic manipulations such as long division. If there is any place for discovery learning, it must come later. Of course, there is never extra time because once the facts of one area of math have been practiced, it is time to move on to the next in a never-ending sequence of math areas (Boaler, 2008; Lockhart, 2009). Similarly, with science, the approach is to have students memorize the basic terminology and facts of one scientific field after another. The assumption is always that there is a fixed body of factual knowledge that forms the uncontested basics of each field of math and science. However, neither math nor science works that way in reality. Each actual field of math and science has evolved and grown through controversy and over-turning of one position after another. Math and science are the products of inquiry, dialog and controversy at the level of the creation of individual results and at the level of the formulation of theories for whole areas. For instance, the expansion of the concept of number in the history of math proceeded through the repeated criticism of the limits of each historical concept: from the integers to rationals, to irrationals, to imaginary and complex, to transfinite, to infintesimal, to hyperreal, .... (Lakoff & Núñez, 2000). If one follows a particular theorem, such as Lakatos’ (1976) study of refutations of proofs of Euler’s theorem, one sees that historical progress in professional mathematics proceeds not by collecting more and more facts, but by reconceptualizations and constructive criticism. Individual proofs of professional mathematics also proceed through complex paths of inquiry, speculation and critique—although this path of discovery is obscured in the linear logic of published presentations. An interesting example of innovative mathematical proof arose this past year when Timothy Gowers, a renowned professional mathematician, invited others to participate in a virtual math team effort to find a new proof for a theorem which had only been proven until then in a very indirect and obscure way (Polymath, 2010): The work was carried out by several researchers, who wrote their thoughts, as they had them, in the form of blog comments at http://gowers.wordpress.com. Anybody who wanted to could participate, and at all stages of the process the comments were fully open to anybody who was interested. This open process was in complete contrast to the usual way that results are proved in private and presented in a finished form. The blog comments are still available, so although this paper is a polished account of the DHJ argument, it is possible to read a record of the entire thought process that led to the proof. (p. 4) As Gowers (Gowers & Nielsen, 2010) observed from a look at the trace of the collaborative effort, even at the highest levels of math problem solving, consideration of false starts is integral to the process: The working record of the Polymath Project is a remarkable resource for students of mathematics and for historians and philosophers of science. For the first time one can see on full display a complete account of how a serious mathematical result was discovered. It shows vividly how ideas grow, change, improve and are discarded.... Even the best mathematicians can make basic mistakes and pursue many failed ideas. (p. 880) GeoGebra
NA2010
 July
27‐28
2010
 Ithaca
College,
Ithaca,
NY,
USA
 3. ANALYZING DISCOURSE In the VMT Project, we study the traces of online collaborative interactions of small groups of students discussing math topics in order to observe the methods of students engaged in math problem-solving discourse (Stahl, 2006; 2009b). We use various approaches to analyzing the discourse. In order to work effectively together, students must make their thinking visible to their collaborators. They can do this in many ways, dependent upon the affordances of the online environment. The VMT environment, for instance, supports chat texting, shared whiteboard drawing, GeoGebra constructions, graphical referencing, wiki postings and math symbols. Because the VMT system captures a complete trace of the group interactions, the thinking that the students make visible to each other is also visible to researchers. One approach that we take to the analysis of student interactions is to conduct data sessions in which a group of researchers collaboratively view the log of what took place in a VMT chat room and slowly step through the interaction (see Section 4 below). This way, we get interpretations of what took place, as seen from the various personal perspectives of researchers with different methodological training. Building on such relatively informal observations, individual researchers can then look more systematically at the trace data and develop analyses of the student-student interactions using concepts and techniques of conversation analysis (Schegloff, 2007), as adapted to online math discourse. Also, we can look at the relations among the students through social-network analysis (see Section 5 below). This way we can quantitatively measure the different roles (e.g., leaders and responders) in the discourse of different groups during various sessions. We can see what the lines of communication were and we can correlate social roles with other characteristics, including measures of math learning. A third approach is to code individual lines of chat for different kinds of interactional moves that may be of interest (see Section 6 below). Then, statistical analysis can reveal patterns in the discourse. In addition, we can correlate individual student learning with characteristics of the chats. For instance, we might compare math test results of individual students before and after the VMT sessions to see who learned the most and then see which groups contained students who learned more or less than students in other groups. Knowing how well students in different groups learned, we can compare the statistical characteristics of the discourse in the different groups. 4. CONVERSATION ANALYSIS AND DISCOURSE ANALYSIS In Group Cognition (Stahl, 2006), we argued that we do not yet have a science of small groups. Current approaches in education, psychology and related fields focus either on the individual or the community, but not on the intermediate small group as the unit of analysis. For instance, most discussions of small groups either reduce group phenomena to individual behaviors or to cultural factors. The VMT Project has been trying to define in a preliminary way a science of groups appropriate to understanding computersupported collaborative learning (Stahl, 2010a). We are interested in the specifically group-level phenomena. Focused on the group unit of analysis, our approach adopts the analytic approach of Conversation Analysis (CA) and adapts it from informal social conversation of mainly dyads to online, task-oriented interaction of small groups; in the VMT case, the groups are usually four or five high school students discussing mathematical relationships, using text chat and a shared whiteboard. GeoGebra
NA2010
 July
27‐28
2010
 Ithaca
College,
Ithaca,
NY,
USA
 In the past year, we have been trying to apply CA techniques in a systematic way to the coding of VMT chat logs (Stahl, 2009a; 2010b). In doing so, we have begun to suspect that these CA techniques are at too fine-grained a level to capture the most important group-cognitive processes in small-group problem solving. While it is true that the adjacency-pair structure on which CA analysis focuses provides much of the interactional fabric of small-group cognitive work, (a) it is at too detailed a level to describe the important methods of mathematical group cognition, (b) it is often deviated from in the complexity of text chat by multiple participants and (c) it fails to capture the larger problem-solving processes that are fundamental to mathematical tasks. At the other extreme, Discourse Analysis (DA) (Gee, 1992) is too high-level, oriented toward the socio-cultural issues, such as power relationships and gender. Just as we have previously maintained that a small-group-level science of group cognition is needed to fill the theoretical lacuna between individual-level psychology and community-level social science, we now propose that an analytic method is needed that fills the gap between CA and DA. We call this new method Group-Cognition Analysis (GCA). It builds on the adjacency-pair structure fore-grounded by CA, but looks at the longer sequences that are so important to mathematical problem solving and explanation. Unfortunately, GCA is extremely time consuming and involves tedious, detailed, multi-dimensional analysis of the words, references and utterances that go into longer sequences; therefore, we are interested in computer-supported statistical analysis and automated coding to assist and complement this analysis process. 5. ANALYZING INTERACTION STRUCTURE To complement the ethnomethodologically informed interaction analysis, we will analyze VMT chat logs using content analysis (Krippendorff, 2004) and social-network analysis (Wasserman & Faust, 1992). The content analysis will be executed using the following two rubrics. The unit of analysis for this work will be a complete unit of group conversation. The first rubric will evaluate the development of group identity within the small groups, using Tajfel’s (1978) description of group communication as inter-group, inter-personal, intra-group and interindividual. Inter-group communication is communication across groups, and only rarely occurs in VMT data. Inter-personal communication takes place between two individuals. Intra-group communication is within the group, where all members participate in the dialogue. An utterance addressing an individual member in the presence of the whole group is coded as inter-individual communication. The second rubric will evaluate trace data for knowledge co-construction using a rubric developed by Gunawardena et al (1997). Two raters will score the conversations on these rubrics and measure interrater reliability using Krippendorf’s alpha (2004). This type of analysis has been performed by Goggins (2009) on asynchronous communication records. The contrast with the results from synchronous chat data will provide a helpful comparison of synchronous and asynchronous knowledge co-construction in small groups. Social-network analysis will be performed on group interactions in order to determine if there are patterns of networked interaction that correspond with the development of group identity or the co-construction of knowledge. The resulting networks will be bi-partite (users and objects) and regular. Since the networks GeoGebra
NA2010
 July
27‐28
2010
 Ithaca
College,
Ithaca,
NY,
USA
 in online chats are closed and small, we will focus our analysis on small network evolution over time and on elaborating semantically meaningful measures of tie strength. Tracking longitudinal evolution will involve developing a time-series set of network views, possibly addressing the state of the network as a feature that contributes to other forms of analysis. We will also explore the advantages of deriving measures of tie strength from the results of machine-learning algorithms, response-time lag and length of sustained interaction between pairs of group members. 6. AUTOMATED LANGUAGE ANALYSIS In recent years, the computer-supported collaborative learning community has shown great interest in automatic analysis of data from collaborative-learning settings, building on and extending state-of-the-art work in text mining from the language-technologies community. Automatic analysis approaches as we know them today are only capable of identifying patterns that occur in a stable and recognizable way. Although those patterns can be arbitrarily complex, there are limitations to contexts in which an approach of this nature is appropriate. These approaches are most naturally usable within research traditions that value abstraction and quantification. The most natural application of such technology is within traditions that employ coding-and-counting approaches to analysis of verbal data. Thus, we do not see this at all as a replacement for the two frameworks discussed above, but as a synergistic approach. By nature, empiricalmodeling approaches involving statistics and machine learning are mainly useful for capturing what is typical. In contrast, within many qualitative-research traditions, it is the unusual occurrences and practices that are worthy of study. Thus, it is unlikely that such technology would be directly usable for producing the kind of findings that are valued within those traditions. However, what it may be able to assist with is finding the unusual occurrences within a mass of data, which might then be worthy of study in a more qualitative way. Machine-learning algorithms can learn mappings between a set of input features and a set of output categories, allowing us to automatically generate coded categories for input utterances. Language-analysis software does this by using statistical techniques to find characteristics of hand-coded “training examples” that exemplify each of the output categories. The goal of the algorithm is to learn rules by generalizing from these examples in such a way that the rules can be applied effectively to new examples. In order for this to work well, the set of input features provided must be sufficiently expressive, and the training examples must be representative. Once candidate input features have been identified, analysts typically hand code a large number of training examples. The previously developed TagHelper tool set (Rosé et al., 2008) and more recent SIDE tool set (Mayfield & Rosé, to appear) both have the capability of allowing users to define how texts will be represented and processed by making selections in their GUI interfaces. In addition to basic textprocessing tools such as part-of-speech taggers and stemmers—which are used to construct a representation of the text that machine-learning algorithms can work with—a variety of algorithms from toolkits such as Weka (Witten & Frank, 2005) are included in order to provide many alternative machinelearning algorithms to map between the input features and the output categories. Based on their understanding of the classification problem, machine-learning practitioners typically pick an algorithm that they expect will perform well. Often this is an iterative process of applying an algorithm, seeing GeoGebra
NA2010
 July
27‐28
2010
 Ithaca
College,
Ithaca,
NY,
USA
 where the trained classifier makes mistakes, and then adding additional input features, removing extraneous input features or experimenting with algorithms. SIDE, in particular, includes an interface for supporting this process of error analysis, which aids in the process of moving forward from a sub-optimal result. Our automatic analysis technology is extensively discussed in our recent article investigating the use of text-classification technology for automatic collaborative-learning process analysis (Rosé et al., 2008).