Mining Negotiation: Using Writing Analytics to Understand Decision-Making and Consensus-Building in Student Peer Review
Susan Lang (Ohio State University, US) and Scott Lloyd Dewitt (Ohio State University, US)
In his landmark 1984 College English article, “Collaborative Learning and the ‘Conversation of Mankind’,” Kenneth Bruffee sought to ground collaborative learning in composition studies, drawing considerable attention to peer review, the task of asking students to read and respond to each other’s writing. Bruffee’s premise, that “understanding . . . the complex ideas that underlie collaborative learning can improve its practice and demonstrate its educational value” (546), outlined three areas of student learning that he asked composition curriculum specialists to take note of:
Collaboration in conversation (550).
Collaboration in authentic social contexts (551).
Collaboration in establishing knowledge (555).
Consensus is often cited as a key outcome of the complex processes of a collaborative learning situation. John Trimbur argues that students who truly attempt at reaching consensus are those who realize that they need “to take their ideas seriously, to fight for them, and to modify or revise them in light of others’ ideas” (Wiener 55). In other words, collaborative learning is “intellectual negotiation,” not merely each student doing his or her part to add to the completed project (Wiener 55). Further, Trimbur complicates consensus by defining it in relationship to Habermasian notions of dissensus: “The consensus that we ask students to reach in the collaborative classroom will be based not so much on collective agreements as on collective explanations of how people differ, where their differences come from, and whether they can live and work together with these differences. . . . Consensus does not appear as the end or the explanation of the conversation but instead as a means of transforming it” (610-12).
This presentation picks up questions we raised at the Writing Analytics Conference in St. Petersburg, FL in January 2018 about the ways in which negotiation, decision-making, and consensus-building are represented in students’ peer review. We are mining and analyzing data from a large corpus of student writing where students participate in an inter-section, cross-campus, anonymous peer review of student manuscripts submitted for publication on a Webzine. While reviewing manuscripts, students compose substantial review memos that evaluate the strengths and weaknesses of the writing. The review process is divided in two steps. First, students write detailed, individual review memos that argue for one of three publication decisions: Accept with Minor Revisions, Revise and Resubmit, and Reject. Second, students work in groups of at least three and come to consensus on their publication decision; they write a group memo that reflects that conversation.
We ask the following questions about this corpus of student peer review:
·In their review memos, how do student represent consensus building and intellectual negotiation? In their review memos, do students represent dissensus?
·In their review memos, what are the similarities and differences between individually authored memos that reflect a single reviewer’s decision and collaboratively authored memos that reflect a group decision?
Using Provalis Research’s Word Stat and QDA Miner, we are engaged in a four-tiered mining and analysis of our corpus
1. We will mine the corpus for review portfolios (3+ individual memos and a collaborative memo) where there is high discrepancy between individual reviewers and group decisions.
2. We will examine/code those portfolios for language that represents decision making processes, negotiation, and consensus building.
3. We will return to the entire corpus to look for examples of words/word strings/passages where students represent decision-making, negotiation, and consensus-building.
4. We will attempt to conclude where students are effective and ineffective at representing decision-making, negotiation, and consensus-building and propose implications for the teaching of writing.