Good Online Conversation:
Building on Research to Inform Practice

Lorraine Sherry, Ph.D.
Research Associate
RMC Research Corporation
Denver, Colorado

Fern Tavalin, Ph.D.
Project Director
The WEB Project
Montpelier, Vermont

Shelley H. Billig, Ph.D.
Vice President
RMC Research Corporation
Denver, Colorado


Abstract

With the increase of Internet and WWW usage in the classroom, instructional activities are now being implemented that use online conversation for knowledge building. Current research with The WEB Project, a Technology Innovation Challenge Grantee, subtitled "Creating a WEB of Evidence of Student Performance in Nonverbal Inquiry and Expression" can help inform practice in this area. This paper presents a synthesis of current research on face-to-face and online conversation and a short analysis of episodes of online conversations captured from The WEB Exchange (www.webproject.org), together with a few perceptions from a student focus group. Lessons learned and implications for practice are then presented.

Introduction

In a recently published article in the Review of Educational Research (Fabos & Young, 1999), the authors question the conclusions of researchers who are quick to enter discussions about skill, social, and economic benefits of educational telecommunications exchanges without a thorough review of conversational research and a critical analysis of online discourse. They then challenge educational researchers to step back, to critically evaluate their enthusiastic rhetoric, and attempt to understand the complex contextual framework in which these online exchanges occur. This paper is a response to Fabos and Young's call to "focus on the content of particular projects, why they hold promise, and how they can be used to meet specific educational goals".

The context of this study is The WEB Project, a Technology Innovation Challenge Grantee, now in its fifth and final year. Created in 1995, the project's guiding vision is "to improve student performance in the arts and humanities and social sciences that stems from the Vermont Framework, using technology, professional development, and community involvement as strategies to promote the use of multiple forms of evidence of student learning and public reporting of information". At present, twelve cooperating middle and high schools are joined by several statewide initiatives including the Vermont MIDI Project, Art Responding Through Technology, the Vermont Center for the Book, the Vermont Arts Council, and the Vermont Historical Society.

The project promotes in-person and online discussions of student work among students, teachers, administrators, and community members that center on the Vermont standards via a website known as The WEB Exchange ( HREF= . The WEB Exchange contains forums dedicated to sharing student works of art, music, and multimedia for feedback and critique from participating teachers, mentors, and students, and to hosting curriculum-related online conversations. While students post and discuss their work and receive constructive feedback from other members of the online community, teachers engage in online discussions of student work, rubric development, and problems that they encounter as they implement technology-based instruction. In essence, The WEB Project has helped to form communities of learners among teachers; among students; and between students, teachers, and community mentors.

The project leadership and the evaluation team examine The WEB Exchange projects and discussion sections on a regular basis. They are particularly interested in finding out what characterizes "good" online conversation in its various genres. They look for evidence of problem-solving, informed decision making, and depth of both student- and teacher-facilitated discussions, especially those episodes that extend the conversation beyond a simple question-answer interaction and examine complex problems from multiple perspectives. This paper presents the results of some of those investigations.

What Is Conversation?

The Purpose of Conversation

When addressing online conversation, the terms "dialogue", "discussion", and "conversation" are often used interchangeably. This paper will attempt to clarify some of the subtle distinctions among them, describe how they work, and present some current research findings regarding both online and face-to-face conversations that take place within educational settings. This is especially important for The WEB Project and its participating initiatives, since many educational technology projects rest on the concept of an ongoing, constructive dialogue that links students, teachers, experts, and the wider community to facilitate systemic educational reform.

The word "conversation" comes from the Latin word "convertere" -- to turn around. It may also be interpreted as "to take turns". Jenlink and Carr (1996) identified three broad purposes of conversation:

Transacting: conducted for the purpose of negotiation or exchange within an existing problem setting;

Transforming: conducted when individuals suspend their own personal opinions or assumptions and their judgment of others' viewpoints; and

Transcendent: where the purpose is to move beyond or "leap out" of existing mindsets.

The Nature of Conversation

The purpose of conversation, whether online or face-to-face, will ultimately drive the type of interactions that will ensue. Jenlink and Carr (1996) also identified four types of conversation: dialectic, discussion, dialogue, and design.

Dialectic conversation focuses on framing a logical argument for distilling the truth. It is a scientific approach, a disciplined inquiry into whatever is being examined. This is exemplified by a few of the reflective student postings on phase 2 of The WEB Project's Civic Discourse discussions, specifically the "What is leadership?" thread.

Discussion conversation is the forum in which many people advocate for their own individual positions. The word discussion has the same root as percussion and concussion -- to break apart. Discussion promotes advocacy of firmly held opinions, sometimes backed up by facts, sometimes Project's Civic Discourse.

Dialogue conversation focuses on constructing meaning through sharing multiple perspectives. It is a community-building form of conversation. Supportive messages, such as the listserv messages described by Herrmann (1995) are an important part of online dialogue: they are the "glue" that holds the conversation together. Dialogue lacks a stable goal. Instead, its purpose is to create a setting where conscious, collective mindfulness can be maintained (Bohm, 1990). This is exemplified by the four to five level discussion threads on phase 2 of the Civic Discourse discussion, "What is leadership?", ending with the liberal-conservative summary.

Design conversation is goal-related and focuses on creating something new. According to Isaacs (1996), it goes beyond the suspension of personal opinions, transforming the participants and deepening their capacity to hear and inquire into perspectives vastly different from their own. This is exemplified by the interactive postings on The WEB Exchange in which students describe projects, post them, and request feedback on specific design points. Then, mentors respond, and students selectively filter these comments and use them to refine and re-post their products. The WEB Project has developed a simple three-point rubric to help guide mentors through giving basic feedback. Whereas level 1 feedback is general in nature, level 3 feedback accurately describes the area being discussed, using technical vocabulary and giving detailed examples, references, connections or responses to general insights.

Discussion is the most familiar and pragmatic. Dialogue is also pragmatic, but less common. Dialectic and design are more disciplined conversations. All may be appropriate, depending on how the conversation is linked to the curriculum. As expected, discussion and dialogue are found on The WEB Exchange "discussion" section -- primarily Civic Discourse and Taking A Stand. Dialogue and design conversations are most frequently seen in the "projects" section.

What Motivates Participation?

Face-to-Face Conversations

David Bohm: The theory of dialogue

Bohm has written extensively on the subject of dialogue. In his seminal 1990 paper, he stresses that in good dialogue, both the individual and the group must suspend their assumptions and preconceived notions. This is often difficult, because assumptions are generally tacit. However, since society is based on shared meanings that are often incoherent, individuals bring to the group or "microculture" a corresponding incoherence.

If all meanings can be negotiated and agreed upon, the group may then be able to work toward coherence. Group coherence is a similar concept to Crook's (1994) "longitudinal continuity", which can be loosely defined as a shared base of beliefs, values, assumptions, and experiences. As a result of this process, Bohm suggests that the individuals may decide to drop a lot of their meanings, which means that they must first apprehend them. In other words, in order to change a viewpoint or perspective, one must first be aware of its existence.

Bohm (1990) also presents the problems and frustrations that arise as dialogue participants attempt to make sense of multiple perspectives. Some statements may make them angry. Assumptions may be revealed and challenged. Opinions of others may seem outrageous. People may be frightened if there is no leader, no topic, and nothing "to do". Bohm therefore believes there has to be a goal or reason for the dialogue. That reason should be strong enough to get the participants through all of these frustrations.

Just as Deming (1994) said, "every system must have an aim; without an aim, there is no system", the same thing holds for Bohm's reason for the dialogue. The reason must be clear, and it must be communicated to all. For The WEB Project's related activities, dialogue is aligned with the purpose for setting up the discourse in the first place, that is, to improve student performance according to standards selected from the Vermont Framework of Standards and Learning Opportunities and the Vital Results: content standards related to history, research, elements and principles of design; and process standards such as reading comprehension, informed decisions, responding to text, the ability to define and solve artistic problems, and the like. The standards flesh out the reason or goal of the dialogue.

How misinterpretations arise and how to resolve them

Bohm recognizes that problems arise as participants try to make sense of the dialogue; this sense making often leads to misunderstandings. This is the basic theory behind Senge's(1994:246-247) "Left-hand column exercise" -- in other words, to distinguish "what I think I heard" (based on my own tacit assumptions and selective filtering) from "what the person actually said". Senge highlights an important problem that arises in dialogue: misinterpretation of the speaker's words or intent. He refers to this as the "Ladder of Inference". Four common misconceptions occur in personal communication:

Our beliefs are the truth;

The truth is obvious;

Our beliefs are based on real data; and

The data we select are the real data.

(Senge, 1994:242.)

In other words, the human mind uses selective encoding, (i.e., distinguishing relevant from irrelevant information). It is when preconceived notions "fog" this selective filter that misconceptions arise. Senge (1994:245) suggests three ways to improve personal communication:

Reflection: becoming more aware of one's own thinking and reasoning;

Advocacy: making one's thinking and reasoning more visible to others; and

Inquiry: inquiring into others' thinking and reasoning.

Kintsch's research on text comprehension

Because the dialogue on The WEB Project is in text form, and therefore is devoid of social cues, another important issue arises: How does the reader -- or listener -- interpret text? Based on his previous research, Kintsch (1989) deals with text-based and situation models. A text- based model is the mental representation of the text that a reader or listener constructs in the process of comprehension (or, in more technical terms, "expresses the semantic context of the text"). In contrast, a situation model is a mental representation of the situation described by the text, whether it be a map, an arithmetic structure, or an operating procedure (Kintsch, 1989: 26). In situational models, subjects make inferences that go beyond the propositions explicitly stated in the text. This is the essence of reading comprehension, in which students analyze and interpret features of a text and make connections between parts of a text and their own personal or vicarious experiences.

Research on dialogue in the reading classroom: Reciprocal teaching

Brown and Palincsar's (1989) work with reciprocal teaching in children's reading classrooms deals with text comprehension and is immediately applicable to the discussions on The WEB Exchange. Reciprocal teaching was designed to provide a simple introduction to group discussion techniques aimed at understanding and remembering text content. It is based on Vygotsky's "Zone of Proximal Development" or ZPD (Vygotsky, 1978). The ZPD is the area of knowledge and skill that the student will not master if left to his/her own devices, but will be able to master with relative ease if given instruction and assistance. Learning takes place in a cooperative learning group that features guided practice in applying four concrete strategies to that task: questioning, clarifying, summarizing, and predicting (Brown & Palincsar, 1989:413).

The teacher models and externalizes mature comprehension activities, focuses group energies on a clear instructional goal, and provides supportive feedback (suggestions and help). Higher level functions are gradually turned over to the students as the teacher removes the scaffolding and fades from constant, directive interaction. The group benefits from the increased range of expertise of its members' combined knowledge. Conflict or criticism forces the group to defend or elaborate solutions, resulting in a more mature resolution.

Discourse analysis

Wells (1996) analyzed "teacher talk" in many classrooms. Traditional teacher talk usually comprises a demand-acknowledge or a demand-give-acknowledge cycle. These terms are defined as follows:

Demand (question or probe: strongly prospective and requires a "give" in response);

Give (less prospective: expects but does not require a response); and

Acknowledge (always occurs in response to a more prospective move but itself expects no further response).

Though the terminology is different, the meaning is the same as The WEB Project's request-respond-reply cycle. To increase dialogue, the follow up move to a "demand" is extended. Extensions include suggestion, reformulation, informing, indirect challenge, qualification, justification, clarification, exemplification, explanation, expansion, and the like. (Wells, 1996:87). Sinclair and Coulthard (1975) propose three categories of follow up (or acknowledge) moves: accept/reject, evaluate, and comment. As mentioned above, Brown and Palincsar use a specific pattern of moves in reciprocal teaching: questioning, summarizing, clarifying, and predicting.

These and other "moves" have been identified and used by many researchers to facilitate face-to-face conversation. However, both student-mentor interactions on The WEB Exchange and student-student and student-teacher interactions in the participating schools indicate that rich conversation is not always describable in terms of these simple discursive interactions.

In Wells' description of "teacher talk", teacher-student conversational exchanges can be independent or dependent. In a dependent exchange, the teacher makes connections to other bodies of knowledge, supplies "proper" information, and requests the student to reply. This characterizes didactic teaching. In an independent exchange, the student is required to perform the very same functions as are performed by the teacher, thereby increasing performance in the ZPD.

Progressive discourse

Bereiter (1994) argues against the concept of specific "moves" such as those described by Wells. He argues that collaborative knowledge building does not happen by following rules. Bereiter cites Kintsch's (1989) mental models that are used for text comprehension. The textual propositions settle down into a pattern that makes sense to the reader. Bereiter makes a clear distinction between "making sense of text" and some predetermined set of moves that constitutes the abstract process of "making sense".

Pattern recognition has been identified by Perry (1992) under the broad heading of "analysis". It is an analytical thinking skill that is sometimes referred to as extension. It is also associated with finding relationships (comparing and classifying ideas) and problem solving.

Bereiter (1994) also wrote about progressive discourse, which leads to building knowledge collaboratively via dialogue, usually in scientific communities. Participants in the discourse must make four commitments:

To work toward common understanding, satisfactory to all;

To frame questions and propositions in ways that allow evidence to be brought to bear on them;

To expand the body of collectively valid propositions; and

To allow any belief to be subjected to criticism if it will advance the discourse.

This last commitment requires a willingness to sacrifice any belief in the interest of progress.

Progressive discourse moves students from analytical thinking skills to synthetic or creative thinking skills. Perry (1992) classifies synthetic thinking skills as production (thinking of many ideas), imagination (thinking of new ideas), and invention (creating a new product). This distinction between analytical and synthetic thinking skills is similar to the distinction between dialogue and design conversation.

Problem solving

Much of the design conversation carried on in The WEB Exchange involves problem solving. One of the standards in the Vermont Frameworks states that "Students use art forms to communicate showing the ability to define and solve artistic problems with insight, reason, and technical proficiency". In so doing, students look for feedback from mentors when they wish to revise their product in a specific, intentional way, but lack the knowledge and skills to do it. Polya (1957) presents guidelines for problem solving: understand the problem, devise a plan, carry out the plan, and look back. Similarly, Bransford and Stein (1985) described the IDEAL method of problem solving: identify the problem, define it, explore possible strategies for solving it, act on those strategies, and look at the effects of your efforts.

Mentors on The WEB Exchange assist students in defining and understanding the problem, and often suggest alternative strategies for solving it. It is up to the students to explore these possible strategies, filter them according to their intent in creating their work, evaluate which strategy (or strategies) might prove useful in their particular situation, and then reply to the mentor with a description of their revised product. The process of posting a revised work acts as a "give" move in Wells' taxonomy, and often sparks enthusiastic feedback from mentors and other students, with additional suggestions for further improvement.

Cognitive apprenticeship

Collins, Brown, and Newman's (1989) work with cognitive apprenticeship involves close communication between experts and novices in an authentic context. Novices progress along a path to expertise by progressively refining authentic products and processes, under the mentorship of experts. Electronically, this is what is happening with the various electronic arts and multimedia projects facilitated by The WEB Project. The mentors are the "experts" and the students are the "novices", although many students soon surpass their mentors in specific domain expertise as a result of participating in The WEB Project.

Cognitive apprenticeship, like any apprenticeship, involves observation of the experts in action, coaching of novices by experts, and successive approximation to expert work as novices gain expertise. Coaching involves hints, scaffolding, feedback, modeling, reminders, and new tasks aimed at bringing performance close to expert performance. Apprenticeship is embedded in a subculture in which members are all participants in the target skills, resulting in continual access to models of expertise in use. Conceptual and factual knowledge are exemplified and situated in the context of their use. The teacher models expert strategies in context. Learners and experts alternate in task performance in a shared, problem-solving environment.

In the cognitive apprenticeship model, modeling, coaching, scaffolding, and fading by experts are important. So is articulation of reasoning, knowledge, or problem-solving processes. Ongoing dialogue and design conversations between students and mentors help students explain their intent in creating the piece and begin to focus on the specific problems they need to solve in order to refine it until they are satisfied with their final product.

Summary of key concepts in face-to-face conversations

Bohm (1990) and Crook (1994) both stress that good dialogue depends on group coherence. If the group is not coherent, then the conversation can degenerate into a discussion rather than a true dialogue. Their view is consonant with Bereiter's (1994) concept of progressive discourse: individuals and the group as a whole must both apprehend and suspend tacit assumptions and preconceived notions before shared meanings can be negotiated and the conversation can progress. Senge (1994) suggests three ways of clarifying shared meanings and avoiding misconceptions: becoming more aware of one's own thinking and reasoning, making one's thinking and reasoning more apparent to others, and inquiring into others' thinking and reasoning processes.

Applying this to The WEB Project, it is important to realize that a good deal of knowledge-building and problem-solving conversation occurs face-to-face in the classroom and is not simply limited to The WEB Exchange. Students bring to the mediated environment those skills that they have already practiced in the classroom. The cognitive apprenticeship model stresses close communication between experts and novices in an authentic context in which novices observe experts in action, are coached by experts, and gain expertise by solving problems and performing realistic tasks. Key ideas contained in the cognitive apprenticeship model are observation of expert behavior, coaching, scaffolding, feedback, and modeling. This is quite different from what occurs in a traditional classroom that stresses recall and recitation of information.

Thus, it is important for teachers and mentors to move beyond what Wells (1996) defines as simple episodes of "teacher talk" (i.e., a simple question/answer interaction) and request students to explain, expand upon, justify, challenge, or otherwise expand upon facets of the conversation that have already occurred. Both Kintsch's (1989) and Brown and Palincsar's (1989) work in text comprehension furnish teachers and mentors with specific strategies that can help students deepen discussions of texts such as questioning, clarifying, summarizing, and predicting actions and events, and making connections between episodes in the text and their own personal experiences. Moreover, Polya's (1957) guidelines for problem solving can serve as a guide for resident artists and mentors as they assist students in defining and understanding artistic problems and suggest alternative strategies for solving those problems in order to produce a more polished final product.

Case Studies of Mediated Conversations

Knowledge-building communities: CSILE

Bereiter was the originator of Computer Supported Intentional Learning Environments (CSILE). The conceptual bases of CSILE come from research on intentional learning (student ownership of goals), process aspects of expertise (progressive problem solving, cognitive apprenticeship), and discourse in knowledge-building communities. According to Brown (1994) in her description of communities of learners, construction of knowledge is a collective goal. Electronic conferencing enables the community of learners to communicate in virtual space. Because this activity increases the collective knowledge, continued adaptation requires contributions beyond what is already known, thus producing "nonasymptotic learning" (learning without bounds). Key points of knowledge-building discourse within the CSILE learning environment are:

Focus on problems and depth of understanding;

Decentralized, open, collective knowledge-building; and

Expansion to the broader knowledge community.

Students initiate and contribute to the discourse, which is supported electronically. CSILE facilitates small group interaction, diverse response to ideas, reflection and metacognition, independent thought, progressive results, and the publication/review process.

The CoVis Project

Pea is one of the founders of the "Collaborative Visualization" (CoVis) project at Northwestern University. The project is housed in the "Collaboratory" -- a learning environment that enables students to generate and visualize solutions to physical problems and to communicate with one another in a common electronic forum.

According to Pea (1994), mediated conversations allow the participants to keep raising the standard and facilitate a form of interpersonal communication that he terms "transformative". In this view, teachers do not simply transmit information to the students from a predefined body of knowledge, nor is that communication purely "ritual" in nature, (i.e., directed toward maintaining the culture of the group through time as the learners participate in the conversation). The body of knowledge actually expands and is transformed by the combination of student/mentor talk, in an open dialogue. In the process, teachers and mentors co-construct knowledge, and student contributions become more expert in nature. This partly explains the process whereby students on The WEB Exchange begin to exceed their mentors' expertise as they progress from novices to experts.

Mediated collaborative knowledge-building in England

Crook writes about computers and the collaborative experience of learning. Crook (1994) says that the type of "teacher talk" that Wells (1996) describes is necessary to interpret and organize students' experiences. However, its downfall is that teachers may interpret them in an official, adult, sanctioned way, not as constructed by the students from their own experiences. True discourse is based on intersubjective understandings, incorporating prior knowledge as well as present experiences. Crook describes structured interactions with online resources -- these may provide participants with important reference points for their common grounding experiences.

Prior to Crook's seminal work, common grounding was studied in moment to moment conversation, but not across sustained and orchestrated patterns of talk in shared time and space, (i.e., that which is made possible by email conferencing or discussions). In Crook's view, asynchronous online dialogue facilitates "longitudinal continuity" among the members of the group: (i.e., the creation and negotiation of shared meaning and experiences, irrespective of time or geographical separation).

Crook (1994) also discusses peer to peer construction of knowledge. Like Pea (1994), Crook argues that expertise is developed dynamically through continuing participation in the learning community's discourse, rather than simply through the individual student's possession of a knowledge base and a set of problem solving skills. He also posits that the possibility of creating a shared cognitive context depends upon the participants' mutual appropriation of motives, intentions, and understanding.

These thoughts are in line with Bohm's (1990) concept of dialogue. Moreover, they are supported by Sherry and Myers' (1998) empirical research on collaborative design, in which mediated conversations provided a framework for sharing goals, negotiating and sharing understandings, and developing a common ground of knowledge. There are four key activities that comprise Sherry and Myers' collaborative design and learning process:

Authentic task (identifying and taking ownership of a task whose cognitive challenges are consistent with the target design environment);

Knowledge development (building a relevant, common base of knowledge, shared experiences and understandings, and associated skills);

Research (formulating questions concerning the design process and the group dynamics that emerge as a result of working on the authentic task); and

Reflection (consciously monitoring and making explicit the team members' own cognitive orientations and processes in relationship to the group's goal).

Crook (1994) identified three group processes that emerge a result of engaging in these activities:

Articulation (self talk leading to metacognition, facilitated by expert mentoring that aims to expand knowledge and skills in the ZPD);

Conflict leading to cognitive restructuring (similar to the reciprocal teaching philosophy of Brown and Palincsar); and

Co-construction (co-construction of meaning by synthesizing distributed expertise and knowledge from the group: similar to Lave and Wenger's concept of legitimate peripheral participation).

These processes are very different from having the teacher guide meaning making in the usual classroom. In Crook's scenario, peers who come into conflict will be prompted to resolve arguments and engage in reflection, rather than deferring to authority. This is the sort of learning outcome that is addressed by Vermont Standard 3.7, Informed Decisions, and Vermont Standard 5.13, Responding to text -- specifically the sections referring to recognizing others' points of view and assessing decisions From others' perspectives. It also is consonant with Perry's (1992) third evel of thinking skills: evaluation.

Online book discussions in Norway

Mehus (1995) conducted research on the use of e-mail in the literature classroom. His aim was to understand the possibilities of e-mail in establishing an international community of high school students collaborating on literary texts.

There is a remarkable similarity between Mehus' work at the secondary school level and the Taking A Stand book discussions at the middle school level. Mehus' cooperating schools agreed on subjects, books, and themes that could be included in all the students' study programs, frequently in cooperation with the students themselves. There were no guidelines as to how to save or administer the files. Each part of the project lasted three to four weeks and included the following activities:

Students discussed the text in class;

Working in groups, students wrote on various aspects of the text;

They e-mailed their writings to other participating schools;

Students freely commented on the contributions they got; and

Student responses were transmitted to the other participants.

Among the outcomes were:

The amount of writing in different genres or language functions was considerably larger than in traditional classes (similar to Wang's findings in 1996 regarding e-mail journaling in an ESL classroom);

The students showed their increased motivation for writing by their curiosity for new vocabulary that was then used in their e-mail messages; and

Teachers became aware of some problems such as access and other technical problems, handling great amounts of text both at the creative and at the reception stages, and the social dynamics of the networked groups.

Lessons learned from Mehus' study included:

There is plenty of room for experimentation when it comes to subject matter and points of focus;

The amount of the text on the network should be reduced by spending more time on collective work and limiting the size of individual responses;

Less attention should be paid to the correctness of language in the messages;

The students should be more involved in the selection of texts; and

Everything done in the project should have direct relevance to the courses in question.

Structural aspects of the discussion were important:

Ample time needs to be spent in finding/discussing various approaches to the project

Deadlines must be strictly observed;

The project leader's role should be discussed thoroughly from the outset;

Schools must be matched carefully by academic level and motivation;

Students must be trained in the use of the media before the project begins; and

Concentrated, intense periods of collaboration are preferable, such as four-five weeks in spring and autumn.

Mehus' lessons learned are quite similar to the responses from students who were participating in the Taking A Stand online book discussions.

Group-based project work in The Netherlands

Collis studies Web environments for group-based project work in Europe. These mediated environments allow collaborative groups to tackle tasks that are too complex for one individual to undertake; provides learning experiences in group interaction; and provides opportunities for students to articulate and defend their ideas, reach group consensus, and manage their work flow as they design and create a product. (See Collis, et al., 1997).

Major variables that influence the success of the group include support for communication, support for handling sharable resources and making them available, social-dynamic aspects of supporting the group (maintaining coherence), and integration of the project with the curriculum in which it is embedded.

Some instructional strategies that evolved from her project were quite similar to those currently being used to establish group coherence on The WEB Exchange:

Link student work, in partial or completed form, to the course site for use as examples and for comment and demonstration during class lectures;

Make sure that each member has clearly defined and separate contributions to the group;

Have a manager in each group responsible for reporting group progress and group reflection, and link this to the course web site so that a group archive is built from the communication;

Use tools with shared workspaces so that a common view of notes and partial products is available to all for self-, group-, peer-, and instructor-evaluation.

Collis found that because this process is intrinsically motivating, it is often difficult to keep a sense of overall cohesion and integration in the course. Part of this is administrative: students working at different times and places must have good access to the common set of materials relevant to the course. Another part of this is conceptual: students working on their own are likely to become so engrossed in their own project tasks and technology tools used to carry out those tasks, that the link between their activities and the course's intent may become very weak.

One of her proposed solutions to the problem is to use the Jigsaw Method: two "image specialists" edit the digital images, while moderators monitor the conversation, data managers make sure that all the data are uploaded and accessible, and so forth. However, since individual student projects on The WEB Exchange each have their own separate discussion thread, a Jigsaw approach may not be a viable strategy to use at this time.

The "learning curve" for technology-based conversation

Whereas Collis' studies dealt with creating and maintaining coherence among a community of learners, Wolcott & Robertson's (1997) study investigated the effects of collaborative work on individual student learning. Students carried out collaborative research in a post-secondary environment, but their basic process has been used by K-12 educators as well. Students worked on a narrowly focused project in a mediated environment for a period of ten weeks. Teams defined an area of research in the field of theatre history. They were given instruction in the use of the Internet and the process of creating hypertext documents. The design specifications for the finished document were intentionally left vague and evolved over time, driven by the research materials uncovered by the students. Final editing was done collaboratively by two editors selected by the group, working on the same file, from their respective (distant) locations.

Throughout this process, students progressed through three stages:

Dualistic learning (the intellectual frame of reference is in terms of right and wrong answers);

Early and late multiplicity (the learner begins to understand that not all knowledge is known, and that there are perhaps areas in human experience where answers are unknowable); and

Contextual relativism (the learner is comfortable examining evidence, drawing conclusions from evidence and from the circumstances in which it is found; and making moral and ethical decisions regarding how to act upon those conclusions.)

Although knowledge is built and meanings are negotiated by the group, the important outcome is an increase in each individual student's learning and performance. It is important to look for continuous growth in critical thinking as students progress from seeking simple answers to complex problems to more mature judgment and reflection. Examining conversation threads on The WEB Exchange, starting with the initial Violence In America discussions, and looking at the spring 1999 downloads from Taking A Stand, it is evident that students are beginning to develop an understanding of and an appreciation for multiple perspectives as they respond to text.

Managing mediated conversations

The final case study in this section deals with strategies for addressing the unique problems of online conversation. Donald Winiecki (1999) carried out research on conversations between students and instructors in an "asynchronous learning network" (ALN) at Boise State University. Through a study and discussion of the communications practices inherent in face-to-face communication, Winiecki illuminated the difficulties found in mediated communications and proposed some strategies that could make online communication a more conducive medium for learning.

In an ALN, interactions follow a many-to-many pattern: students and teachers send messages to the entire class and to individual students at the same time. In the case of The WEB Exchange, the interactions are even more complex -- teachers, mentors, musicians, graphic artists, community members, and other students can all comment on student work and participate in design conversations. Groups of students at various schools participate in discussions about the same texts within the same time frame. This may make the conversations hard to follow because of multiple, concurrent threads.

In normal face-to-face conversations, communication behaviors are based on a serialized, turn-taking system in which usually only one person talks at a time. In contrast, the many-to-many messages that characterize mediated conversations do not occur in a serialized, turn-taking manner. Thus, each message may potentially introduce or address several topics at once.

To alleviate this problem, Winiecki (1999) suggests using software (such as that used for The WEB Exchange) that displays a graphical map connecting messages in a "spanning tree". Additionally, he suggests using strategic "snipping" to simulate conversational overlaps, and using formulations and repairs to emulate normal conversational practice.

According to Winiecki, successful learning through knowledge-building conversations demands that the participants be able to follow the total interaction from beginning to end. In normal face-to-face interactions, there are several tacit verbal practices that permit participants to keep track, update, or even repair gaps in their understanding of the conversation. Besides turn-taking, these practices are repair, overlap, and formulations. Such practices enable each learner to remember what has been mentioned and elaborated upon during the conversation. These practices can be adapted for online conversations.

Repair overcomes participants' difficulty in remembering what has transpired in the conversation through the use of statements that refer back to things that were said earlier in the conversation, or in another conversation. A simple way to repair an electronic conversation is to reintroduce prior (and often forgotten) snippets of conversation into the discussion by strategically "snipping" (extracting and embedding) short, relevant sections of an old message into a new message.

Overlap occurs when a listener interjects a new thought, even when the present speaker intends to continue. In this case, it is appropriate for the present speaker to relinquish his/her turn to the listener. This works most efficiently when messages are kept short, which also tends to limit the number of new ideas per turn and keep the conversation focused.

In conversational interactions, individuals actively attempt to understand what is being said in terms of what has been said earlier and what they already know. Thus, they create new understandings by combining new and old information and knowledge. To test or expose his/her understanding, a student may offer it as a formulation. A formulation is a summary of earlier statements ("Who said what, to whom, and when") that serves two purposes:

It discloses the speaker's understanding; and

It keeps other participants aligned with this understanding.

It is important to note that the purpose of formulations in a message is to repackage the history of the discussion up to this point and provide a logical "lead in" to the point(s) being made in the current message. Failing to adequately package these items may leave a reader as confused as if no formulation were provided (Winiecki, 1999: 11).

Summary of key concepts in mediated conversations

The CSILE project has had a long period of success in enhancing students' comprehension of text by building a community of learners who work within a mediated learning environment. Bereiter noted that students in a CSILE environment developed new insights and created new knowledge in a manner that Pea would describe as transformative, (i.e., "pushing the bounds of knowledge"). Sherry and Myers (1998) articulated the process of transformative communication by a team that was attempting to create a well-designed product. By working on and taking ownership of an authentic task within a mediated learning/design environment, the team developed a common knowledge base, generated new questions and problems that required resolution, and engaged in conscious reflective practice. Crook (1994) described the results of this process in terms of articulation (self-talk leading to metacognition), conflict leading to cognitive restructuring, and co-construction of meaning. Applying these principles to an online literature class that was rather like The WEB Project's Taking A Stand online book discussions, Mehus (1995) found that students who used e-mail in their literature classroom produced more writing, developed a larger vocabulary, and attained a greater awareness of group processes.

These outcomes are not without unintended side effects, as cited by Collis (1997). As students become more engaged in their online dialogue and design conversations, there may be a lessened sense of group cohesion and clear integration of their activities into the course. Moreover, there is a learning curve that students must go through as they collaborate in research and design, especially when the design specifications for the finished product are vague and evolve over time. Teachers and mentors must remain aware of the intolerance for ambiguity that middle and high school students have to overcome as they progress from dualistic learning through early and late multiplicity, and finally, to contextual relativism and informed decision making.

What Outcomes Characterize Student Learning?

Learning Skills

Learning theorists have proposed typolologies that distinguish various types of learning according to differences in what is being learned. A comprehensive list of all these outcomes looks like the fields of biology or particle physics during their early days of development -- a plethora of classifications and taxonomies.

Cognitive skills: Bloom's Taxonomy of Cognitive Objectives

One of the earliest taxonomies was devised by Bloom and his colleagues. Bloom et al. (1956) defined six different kinds of learning objectives and identified them with standardized terminology. These are:

Knowledge: (recall of specifics and universals, patterns, and structures);

Comprehension: (translation or "making sense of the problem", interpretation or defining the problem, and extrapolation or relating it to other ideas or similar problems);

Application: (the use of abstractions in a concrete situation, such as how to apply a known process to accomplish their specific task);

Analysis: (breakdown of communication into its constituent parts so that the relationships among the elements or ideas expressed are made);

Synthesis: (or selective combination, which involves putting together relevant information - putting the parts together to form a whole, to develop a pattern or construct a product); and

Evaluation: (making judgments about the extent to which the products and processes satisfy given criteria).

Some theorists have provided criticisms of this work. They believe it mixes comprehension and application skills. Sternberg (1998), for example delineates the boundary and divides learning objectives into "knowing that" (declarative or factual) and "knowing how" (procedural). Under procedural knowledge, Sternberg adds procedural tacit knowledge, which involves knowing how the system functions in which one is operating. This is particularly important for students who are using technology to develop an electronic product such as a MIDI music composition, a multimedia presentation, or a piece of digital art. Second, Bloom's Taxonomy confuses the cognitive level of the skill with the notion of the difficulty level of a question intended to elicit a certain type of response. Wells (1996) noted that most traditional "teacher talk" is concentrated at the low levels of Bloom's Taxonomy. The WEB Exchange is intended to facilitate discourse that will develop the higher levels.

These higher levels -- analysis, synthesis, and evaluation are three of the four thinking skills referred to by Perry (1992). By adding Perry's fourth thinking skill -- research -- one comes up with a more complete taxonomy that describes the types of learning outcomes that can be observed in WEB Project participating schools. In Perry's expanded model of higher order cognitive skills, "research" comprises error analysis (seeing what is wrong and telling how to fix it), structural analysis (identifying a major idea or theme and seeing how other themes are related to it), and investigation (or, in student terms, "solving a mystery").

The Gagne and Briggs Typology

Addressing some of the limitations of Bloom's Taxonomy, Gagne and Briggs (1979) identified five major types of learning:

Attitudes: internal states that influence learners' personal action choices;

Motor skills: organized movements of the skeletal muscles that learners use to accomplish purposeful actions;

Information: facts and organized knowledge about the world stored in learners' memories;

Intellectual skills: skills that permit learners to carry out symbol-based procedures, including rules; and

Cognitive strategies: strategies that learners bring to bear on their own cognitive processing in order to control their thinking or learning or to originate novel solutions to problems.

Sternberg: The Developing Expertise model

Sternberg devised a Developing Expertise model that incorporates five elements of learning: motivation, metacognitive skills, learning skills, thinking skills, and knowledge, within a context that includes both teachers and learners. According to Sternberg's framework:

Motivation drives metacognitive skills, which in turn activate learning and thinking skills, which then provide feedback to the metacognitive skills, enabling one's level of expertise to increase. The declarative and procedural knowledge acquired through the extension of the thinking and learning skills also results in these skills being used more effectively in the future. (Sternberg, 1998: 17).

Motivation

Motivation is the first step to learning. Besides intrinsic and extrinsic motivation, there are other types of motivation such as achievement motivation and competence motivation. People who are high in achievement motivation are attracted to tasks that are neither very easy nor very hard, and are constantly striving to better themselves. Competence motivation is equivalent to Bandura's (1982) concept of self-efficacy, which refers to people's beliefs in their own ability to solve a problem at hand or deal with a difficult task.

Metacognitive skills

Metacognitive skills refer to students' understanding and control of their own cognition, both what steps are required to solve a problem and the ways in which these steps can be executed most effectively. Perry (1992) refers to metacognition as thinking about thinking -- reflective thinking, deep processing, and planning thinking. Schon (1987) describes thinking-in-action as reflective practice. Cognition is involved in doing, whereas metacognition is involved in choosing and planning what to do and monitoring what is being done. Sternberg (1998) identifies seven metacognitive skills:

Problem recognition;

Problem definition;

Problem representation;

Strategy formulation;

Resource allocation ;

Monitoring of problem solving; and

Evaluation of problem solving.

These are the types of skills that are generally involved in problem solving and are captured in design conversations.

Learning skills

These knowledge-acquisition skills are sometimes divided into explicit (intentional) and implicit (or incidental) learning. Among the learning skills are selective encoding, selective combination, and selective comparison. Selective comparison is important in the cognitive psychology literature; it involves relating new information to prior knowledge. Cognitive psychologists believe that information is organized in schemas. Thus, if a piece of new knowledge is to be accessed and retrieved in the future, it must be related to an existing schema that was developed from the student's prior experiences and understandings.

Thinking skills

Sternberg classifies three main kinds of thinking skills or performance components.

Critical (analytical) thinking skills include:

Analyzing;

Critiquing;

Judging;

Evaluating;

Comparing and contrasting; and

Assessing.

Creative thinking skills include:

Creating;

Discovering;

Inventing;

Imagining;

Supposing; and

hypothesizing.

Practical thinking skills translate thoughts into action and include:

Applying;

Using and utilizing; and

Practicing.

Knowledge

As previously mentioned, there are two main kinds of knowledge: declarative knowledge or knowing about facts, concepts, principles, and laws; and procedural knowledge, which comprises knowing how to apply specific procedures and strategies.

Sternberg's theory predicts the following:

Initial outcomes will be found in the area of student motivation, time on task, and engagement;

Intermediate outcomes will comprise metacognitive skills; and

Final outcomes will show an increase in declarative and procedural knowledge.

The results of the online survey of The WEB Project participants in the summer of 1998 validated this prediction, although there is no indication at present that the results will be stable over time.

Putting theory into practice

Bloom's Taxonomy, the Gagne and Briggs typology, Sternberg's Developing Expertise Model, and Perry's Thinking Skills all attempt to cast learning skills into various types of integrated frameworks. Moving from theory to practice, one must begin to think of an "information ecology" (Nardi & O'Day, 1999) or an "activity framework" (Engestrom, 1996) in which the following six elements are inextricably connected:

The students;

The technological tools they use;

The products they create;

The norms and conventions of the school and the classroom;

The roles and levels of expertise of peers, teachers, and mentors; and

The entire community of learners -- both in the classroom and online.

What Outcomes May We Expect from Student Dialogue Within an Authentic Context?

Rich Environments for Active Learning (REALs)

The combination of students working on authentic tasks on The WEB Project computers and the mediated environment, carrying out online conversations supported by The WEB Exchange, within a supportive classroom or laboratory environment, guided by participating teachers, mentors, resident artists, and members of the greater online community, together constitute a rich learning environment or information ecology. The WEB Project context shares many characteristics that are held in common by a group of learning environments that were identified by Grabinger (1996) as Rich Environments for Active Learning (REALs).

REALs are based on constructivist values and theories including collaboration, personal autonomy, generativity, reflectivity, active engagement, personal relevance, and pluralism. They engage students in a continuous collaborative process of knowledge construction in an environment that reflects the context in which that knowledge will be used. They are not delivery mechanisms, learning systems, nor Microworlds, which simulate or mirror authentic environments; they are authentic environments in and of themselves. They may be mediated, as in The WEB Exchange, CSILE, the CoVis Project, or the work of Collis, Crook, Mehus, and Wolcott and Robertson; or they may be used in traditional classrooms with face-to-face interactions such as reciprocal teaching, problem-based learning, or cognitive apprenticeship.

All REALs share a set of common characteristics. They:

Promote investigation situated within relevant, complex, information-rich contexts;

Encourage student responsibility and intentional learning;

Cultivate an atmosphere of cooperative learning and social negotiation of meaning;

Utilize dynamic, generative learning activities that promote high level thinking processes; and

Assess student progress through realistic tasks and performances.

Moreover, all REALs emphasize knowledge-building conversation among collaborative groups of students as one of the primary learning activities.

Conversations in a REAL Facilitate Developing Expertise

The role of conversation is crucial in all REALs, although it may take many forms in many different contexts. For example, conversation may:

Engage students in problem solving in authentic environments,

Utilize mediated environments to engage students in looking at problems From multiple perspectives,Encourage students to take ownership of their own learning and construction of knowledge within a community of learners, or

Introduce them into a community of learners consisting of novices and experts and scaffold them as they gain performance expertise within that community.

In The WEB Project, these activities are exemplified in Taking A Stand, Civic Discourse, Vermont History Day, and the design conversations on The WEB Exchange.

It is important to note the emphasis of the concept "community of learners" in all REALs. Communities of learners consist of experts and novices. If the goal is for students to carry out expert activities within a community of learners, then it is necessary to treat them as developing experts. Experts and novices approach learning activities in different ways. For example, novices in mathematical problem solving classify problems according to surface characteristics rather than identifying the general principles that underlie the solution to the problem (Chi & Bassok, 1989). Novices in instructional design tend to rely on a formal design procedure, whereas experts engage in constant redesign and re-evaluation as the product begins to take shape. (Nanard & Nanard, 1995).

In their research on collaborative design, Sherry and Myers (1998) noted that a common knowledge base emerged from active dialogue and participation among the online design team members as they sought to understand concepts and apply techniques to solve problems and create a polished, professional project. Throughout this process, expertise was irregularly distributed and developed at different rates for each individual in the community. Recalling Bereiter's (1994) argument, collaborative knowledge building did not happen according to a set of fixed rules. Instead, the participants began to recognize and work with emerging patterns and relationships that made sense to them. This process allowed relatively novice designers to take advantage of the distributed expertise of the team and function in a manner more analogous to expert designers. Because roles were blurred in the learning environment, novices were able to participate in all aspects of the design process.

Applying These Concepts to The WEB Project

A similar process occurred with The WEB Project conversations, both face-to-face in the classroom and on The WEB Exchange. Within each of the participating initiatives of The WEB Project, students shared similar goals: they were expected to address problem-solving standards from the Vermont Framework of Standards and Learning Opportunities. However, a recent survey of students in participating schools revealed that there were significant differences (p< .05) between schools and districts in students' attitude toward school, application of skills, inquiry learning, and perceptions of learning new skills. Differences in metacognitive skills approached, but did not actually attain, significance (p = .065). Expertise was irregularly distributed, and students were just starting out on the route from novice to expert.

In a recent paper (Kruse & Quinn, 1999), teachers whose classes participated in Taking A Stand, Civic Discourse, and Vermont History Day online conversations reported that these activities did not go smoothly at first, despite a two-week preparatory period in which students participated in small group discussions, formulating and responding to interpretive questions, referring to passages in the text to back up their opinions, and relating the experiences of the characters in the text to their own prior experiences. Initial online activities resulted in the types of user concerns predicted by Hall and Hord (1987): seeking more information about the technology, uncertainty about their ability to meet its demands, "finding a voice and having something to say" (Wilson et al., 1996), and getting caught up in a host of technology problems. The students did not yet see a clear payoff for their efforts.

However, as time progressed and as students went through a learning curve similar to that described by Wolcott and Robertson (1997), they began to see online conversation as more than a recreational activity. It became a way to gather and exchange information and ideas with students who had a multiplicity of perspectives. Technology was used as a tool for learning, not simply as an end in itself. In these online conversations, teachers and students alike experienced a variety of successes and frustrations in achieving the standards around which the curriculum was based. Students in a focus group reported that it was difficult to converse with peers who responded at a superficial level to the text being discussed because students at the distant school had not refined their face-to-face conversational skills in class prior to going online. They also stressed the importance of in-class practice in searching for patterns of behavior and relationships between ideas, and in identifying underlying concepts among a number of disparate pieces of literature.

Patterns of conversation captured in the following sections can illuminate the learning processes that occurred online, and may serve as exemplars for schools, teachers, and mentors who are new to the principles and practices of mediated discourse. Moreover, by analyzing snippets of online conversation, features of successful dialogue and design conversations may be identified. It is important to note that the skills used in these episodes of conversation span many of the learning elements in Sternberg's Developing Expertise model.

Exemplars of Online Conversations

This section presents three examples of threaded discussions that were captured from The WEB Exchange in spring 1999.

"Dress Code" is a dialogue among teachers and students from four participating schools. It addressed the controversial issue of appropriate student clothing from multiple perspectives. At times, it exhibited some features of a discussion, especially advocacy for one's own favored viewpoint.

"Cop Killer" is a comprehensive dialectic conversation with in-depth comments by artist-mentors who grappled with the sensitive issues of censorship and artistic intent.

"Night Owl" is a design conversation between a two students and an artist-mentor. It took the form of request for feedback by a student artist, with responses at levels 2 and 3 on a scale of 1 (low) to 3 (high).

"Dress Code": A Dialogue/Discussion

This conversation took place on an the student open forum: one of the many threaded discussions that were hosted on The WEB Exchange. Students and teachers were encouraged to discuss controversial issues that were not limited to academic topics. As the conversation evolved, it fluctuated between a dialogue (an exchange of ideas and opinions between two or more persons) and a discussion (the consideration of a question in open debate).

A student posed several topics for discussion and invited others to join the conversation. A teacher presented the "pro dress code" viewpoint, and then students and teachers continued the conversation:

As a teacher I often wish we had some kind of dress code. I am all for liberal policies that allow people to wear what they want, but it seems to be at an extreme. I see so many girls wearing, um, nothing! As an adult, I often want to ask why people are wearing just camisoles or tank tops with a bra that covers more than the tank top. If I ask students could they hitch up their pants, I get "the look" or "it's a free country" or "what are you looking at?" Some students act as if it is sexual harassment for a teacher to notice what they wear.

Her sentiments were echoed by another teacher who introduced the issue of uniforms. Another teacher put forth a few probes to extend the conversation:

If I comment about its inappropriateness, can that be considered sexual harassment? Do clothes affect your state of mind and work habits? Would the right clothing change students' attitudes towards school and their work ethic?

A student who had already discussed her school's hat policy with the Student Governance Committee took the floor. Her posting is a good example of a meaning-making response.

I think it's important for the school to try to guide students as to what is appropriate in society, but not force them...[in our school] we aren't allowed to wear hats (except on Friday afternoons now!), and most students don't know why. The teachers explained that it's a matter of politeness and that in our society hats aren't okay. I think we create our own norms. I don't think my dad is impolite, and he wears hats.

The student who initiated the conversation spoke out against uniforms, but suggested some guidelines for appropriate student dress. Another student introduced the issue of equity:

I believe that you have some good points, but it is a free country and you should be allowed to wear what you want. At Cabot I feel that our dress code is minimal, but certain students are told what to wear more than others, and this should change. If you are going to apply something, you must apply it to everyone equally.

At this point, the conversation began to degenerate into a discussion/debate rather than a dialogue, as students argued the pros and cons of dress codes:

Uniforms eliminate a student's worry about being fashionably correct, and this allows them to focus more on their school work than what they look like, and what others think of them. Uniforms also create an environment of order and strictness.

I was originally thinking of going to __ High School -- but --there is a very strict dress code and I wasn't about to follow it...whether a school requires a uniform or not more often than not determines whether that kid decides to go to that school or not.

I think that the style or type of clothing that we wear reflects our personality. It would be like someone telling you that because you are not exactly like them that you have to change so that they can feel more comfortable...We are all different in a lot of ways, so why should we be told to wear the same thing as everyone else?

I do understand how you feel about the lack of clothing which students are NOT wearing these days. But it is important to remember that many years ago it was considered impolite to show our ankles, never mind our belly buttons. I do understand that sometimes we go a little too far, yet I also understand that change, in this manner, may be a good thing.

If you don't like how it looks, don't look at it. That's what I was always told.

Another teacher shifted the focus of the conversation by replying to the first teacher's response. Following the Vermont Standard 6.2 -- uses of evidence and data -- he traced the origins of the conflict and presented some historical evidence to bring the conversation back to its original intent: the sharing of ideas. Citing a book on the history of sexuality, the teacher presented the view that "how we express our sexuality always remains in flux, torn between forces that desire change and those that desire 'sameness'." I think the same dynamic is present here. His posting showed how teachers could work from student interest to then insert educational objectives into the online conversation:

I don't feel the same resistance to change: "Clothes are never the man" to me. But I am intrigued by how we PROCESS this discourse. Usually, schools control these changes by codes and rules and discipline. The structure of our response is not to get at the underlying dynamics, but to treat the symptoms of that dynamic. If I were stopped by the police and asked to refrain from doing something ONLY because "society frowns on it". I'd ask some more serious questions.

The online conversation did not follow up on that comment. Rather, it ended as more of a discussion than a dialogue.

"Cop Killer": A Dialogue/Dialectic Conversation

The "Cop Killer" conversation took place on the adult discussion forum. In this dialogue, the dialectical tensions (the juxtaposition of contradictory ideas, seeking to resolve their conflict) were between censorship of art versus freedom of expression, and between intellectual honesty in critique versus making moral judgments about the images and remarks that students posted on The WEB Exchange.

It started when student posted a work of art titled "Cop Killer" on The WEB Exchange for critique and feedback. Many of the teachers considered the image to be objectionable because of its depiction of violence. They were also concerned about the tendency of the male students to create what one teacher termed "Mortal Kombat ripoffs" for some of their projects. At the beginning of the conversation, art teachers and mentors raised the issue of censorship. Here are some illustrative comments from several participants:

I asked my husband to look at the site and help me sort out my feelings regarding these images. It helped a little bit. I kept my comments to talking about the creation of the art and tried to steer away from moral judgments.

My basic conservatism causes the hairs on my back to bristle at the sight of this "art" work displayed here...this is just the "ammunition" the anti-NEA people like to get their hands on.

Acknowledging the advocacy comments of these and other participants, a teacher alluded to the student's unspoken intent in creating and posting the image:

Throughout art history it is the bizarre and inventive images that get the most attention. Recently, gross and disgusting are having their day too. There does seem to be a place for it in art...I do see by the amount of work he is putting up here that he is a serious artist. Perhaps he is unaware of what he is doing, but I have a feeling he is chuckling behind closed doors thinking he got away with this...I think we should ask this kid to refrain from putting this kind of offensive work up here.

Here is a good example of Senge's (1994:246-247) "Left-hand column exercise". Throughout the rest of the conversation, teachers and mentors tried to distinguish between their perception of what the student intended (based on their own tacit assumptions and preconceived notions) from what the student actually intended. To assist mentors with this process, the WEB Project co-director raised some provocative questions:

Should the mentors have to see or comment on images that they find disturbing?

What are the artistic goals for this student and how are they being met?

What type of tone do we wish to set in this online community?

Several mentors suggested that they engage students in a discussion to clarify their artistic goals. This could potentially shift the focus from the highly charged issue of censorship to one of appropriateness, as in the preceding "dress code" dialogue.

If I had this student in my class, I can imagine using the work to engage him or her, and others, in some very interesting discussion that could in fact lead both to deeper understandings of the aesthetic experience and also of the social issues that underlie this expression.

This should be brought into a structured discussion with students. All of this is really relevant with recent violence towards police in our own backyard and the "meek" teenager from Kentucky who destroyed the lives of his classmates.

When I encounter student work I find offensive. I try to raise questions about representation, about audience, and then reflect back on the original goals of the project. Often, without getting into too much content, students begin to understand the POWER of images and their effect (or lack thereof) on other people.

The student's teacher was also engaged in the online dialogue so she located the student who posted the "cop killer" image, told him that there was a discussion going on among the mentors and adults using the site, and asked him what his intent was in posting the image "so that I could understand where he was coming from with it." The boy's response surprised her:

He said that he'd been sitting in the doctor's office reading old magazines, and in a Time or Newsweek, came across an article about kids throwing bottle grenades at cops. There was a picture of a kid with a bottle in the air, ready to throw it, and in the article it mentioned that this kid was killed by the police moments after the picture was taken. The image stayed with [the boy] and he was haunted by it. He drew ["Cop Killer"] as a way of saying, "Look at the consequences of your actions" to other young people.

His teacher suggested that the online conversation be shared with her students. By so doing, she felt that the boy and other students who were uploading inappropriate comments and offensive images to the site would be made accountable for their actions.

I think [the creator of the image] will welcome the opportunity to explain his intent, and I would like to see [another student] realize that there are other ways to express one's need for power, one's feelings of powerlessness, etc., than through the genre of war games.

The other mentors responded positively. The supportive comments that were interspersed with their questions and replies reiterated Herrmann's (1995) findings that community-building messages are the "glue" that holds an online community together. Participants shifted the focus from censuring the student to examining their own tacit beliefs, values, and assumptions.

I found the description of your discussion with the student/artist fascinating. And it is such an incredible example of how the "label" contextualizes our understanding of an image.

It reinforces the simple rule: check it out ... if you are hurt or angry from a remark made online, go back and ask and find out from the source. Often, it seems we spin out a whole reality out of a remark (or image in this case) that takes on a powerful energy and "carries us away"...people need to know the impact of what they do, and often just that knowledge will change the scene.

There is a part of me that likes the fact that he landed here and we are dealing with this. If we push him down here, he will just pop out somewhere else. Hopefully all our responses won't encourage him to pursue this subject; but rather refocus on why so many of us responded this way.

The remainder of the dialogue is an example of Crook's (1994) process of working toward longitudinal continuity, in which participants negotiate and agree upon shared meanings. In the ensuing messages, many teachers and mentors continued to reflect upon their own perspectives. Here are some illustrative postings:

Moving out a little farther from the concrete image, I think there is a difference between personal offense and censorship, and it is a difficult line. [It is] one we are going to have to grapple with...which points to the purpose of the site and the exchanges that are going on here. Is it a learning site, and bound by all the rules and conventions that schools are? Is it a public exchange? Is it some difficult blend of the two? And where is the line?

Is it OK for us to react honestly to images, and tell the artist how we feel? The question of honesty seems to drive at the heart of a lot of this conversation. I have spent a good part of my teaching trying to recreate the spirit of critique that I found SO valuable in my first film class!

I found myself feeling shocked and excited at the same time with the discussion that is taking place concerning "cop killer". Excited that this online medium is allowing spontaneous communication without regard to physical distance and time, allowing us to connect and share important viewpoints in an incredibly short time. I am moved by the well reasoned and considerate comments this online community is expressing on topics like censorship and the power of images, how they are labeled, presented, and perceived.

Toward the end of the conversation, one of the mentors used the "repair" tactic to refocus the dialogue. She "snipped" the questions raised by The WEB Project co-director from her message, inserted them into her own posting, and requested:

As a mentoring participant in this project I would like to bring back to the forefront of the discussion something that [the WEB Project co-director] brought up and that is...

["Snip" questions from prior message]

I'd like to add another thing...

Are these kids being properly trained how to upload to this site? What should a kid put in this field? "Enter a description of your work below:"

Could we ask students to be a bit more literate here? Explain more about where their ideas come from? Or how they created the image?

I think what we are talking about is that difficult process of creating a language between US first, and the kids.

This is an issue that was also raised by Kruse and Quinn (1999) regarding the student online book discussions. As Wilson and his colleagues (1996) found, one of the barriers to student participation in online activities is "finding a voice and having something to say." As some of the mentors suggested in their ensuing messages, if the "description of work" field could include student questions about use of color, style, and what they might be trying to achieve; and if the art mentors could share the method of reflection and critique that worked successfully for the more experienced mentors who critiqued students' sound files; then all would benefit.

"Night Owl": A design conversation

A design conversation is goal-related and focuses on creating something new -- in this case, a work of art. "Night Owl" is a short but insightful design conversation that was initiated by a young artist who loves her work, but is not necessarily talented. She was interested in improving her pencil sketch and achieving her intent. Her description of work was terse:

This is an 8 x 11-inch pencil sketch that I turned in for my weekly homework sketch.

In contrast, her request for feedback was quite specific:

This was a free sketch that I chose because it is one of my favorites. If you have any ideas on how I might be able to make the background look more lifelike or make the owl stand out more, I would appreciate any comments.

The first response to "Night Owl" was from another student:

You wanted to know how to make the background less confusing, and how to make the owl stand out more. (I hope that's what you asked.)

Well, I think by changing one you will "kill two birds with one stone" (sorry if the quote is a little crude)

First, I think by removing the stars, or some of them, your background wouldn't interfere with the owl as much, and therefore make it stick out more.

This initial interchange is a good example of Senge's (1994:245) strategies for improving personal communication: reflection, advocacy, and inquiry. The artist was able to identify the problem and make her own thinking and reasoning apparent to the online community. In return, the responding student purposefully inquired into the artist's thinking and reasoning when responding. The artist replied positively to the other student's suggestion:

Thanks for the idea. I might just try it. When I do, I'll post another picture and see what you think.

The next response was from a mentor. As in the message from the student, the mentor addressed the artist's expressed intent directly, before offering suggestions for improving the piece. Even though the conversation was short, this "repair" strategy kept the message focused on solving the problem that the artist identified in her initial posting.

You want to make the background more lifelike or make the owl stand out more...hmmm...

Another strategy that was used by this mentor, and many of the other mentors, was to begin and end the response with supportive messages, and "sandwich" the critique in the middle.

The first thing I notice about this drawing is that you have done a great job with drafting the owl (the lines) but you have not added much shading. In order to achieve a realistic looking image you need to add shadows to create the illusion of contour in your owl...

The critique pointed out several other problem areas in the piece, with specific suggestions for improvement such as drawing the feathers to simulate the curve of the bird's body and darkening the night sky. She discussed larger issues such as contrast and shading, again interspersed with supportive comments and further inquiries into the student's thinking and reasoning:

When looking at a work of art, my eye naturally is drawn first to the part of the image that has the most contrast. Try this yourself by looking at a few pictures. Drawing is as much about looking as it is about applying the medium. When I look at our piece here I see the moon outline first. It has the sharpest contrast. Then I see the claws, the dark lines of the feathers, and the black eyes. Tell me, is that what you wanted to draw attention to? When you add detail in the form of dark contrasting lines it becomes an important element in the drawing. You are the creator of this drawing. You have a great start here. You could leave it alone or you could take it further by choosing an important area in which to create contrast...

Next, she offered several examples of other artists' work for the student to use as examples for improving her own piece. Since all of the threaded discussions are hosted by a server that is connected to the Internet, linking to online resources was straightforward:

I can't help but think about an artist I associate with drawing. He became a master during his lifetime. He lived almost exactly 500 years before you. He was born in 1471; his name is Albrecht Durer. Here is a drawing by Albrecht Durer when he was 13 years old.

http://metalab.unc.edu/cjackson/durer/p-durer52.htm

Below is an owl illustration I found on the net. I thought you might enjoy looking at it to see how another artist used line and shadow. This drawing looks like it might be pen and ink. The artist used all black lines but in the center of the chest there is definitely a lighter area that appears to give it form.

http://idt.net/~jsss/drawgal.html

To help the student extend the conversation, the mentor closed her message with some probes and additional supportive remarks:

I think you have a good eye and you have worked hard so far on this drawing, I can tell. I hope some of what I wrote here helps you. If it does (or even if it doesn't), would you mind dropping a note here in this thread so I can know too? I would appreciate hearing from you and hearing your feelings about this drawing and about my suggestions.

The student replied within the hour:

Thanks for your ideas. I'm going to put your ideas into consideration. I noticed after I posted the sketch the sky didn't come out very dark so I will darken the sky. I took some of the stars out of the sky already, and I'll try to add more shading, but I think you're right, I am afraid of shading. I will work on the feathers. And as soon as I redo that, I'll post the sketch for you. Thank you very much.

The spring term ended before the student had a chance to re-post her revised work.

Lessons Learned

Features of Successful Online Conversations

As with most qualitative analysis, the creative work follows data collection -- trying to categorize episodes of conversation and make sense out of them. The conversation is explored as it evolves, and the observer strives for explanations of the interchanges as they happen. In this study, dozens of conversations were considered. Since all of the conversations were previously posted as threads in the various online forums on The WEB Exchange, the issue of the observer affecting the conversation did not arise.

The overarching categories and themes for online conversations emerged from the review of relevant literature, namely: dialectic, discussion, dialogue, and design (Jenlink & Carr, 1996). Our task was to look on The WEB Exchange for exemplars of each of these four categories, specifically those that were goal-oriented and resulted in some negotiated understandings. Several excellent examples emerged from this initial exercise. This list was winnowed down via a series of e-mail messages between RMC Research Corporation and The WEB Project. Finally, at a face-to-face meeting in Basin Harbor Vermont, the three co-authors discussed and agreed upon three "best of the best" examples to analyze further.

A list of codes was then generated, comprising possible "moves" that were identified in the literature review and that might occur in the conversational episodes under investigation. These included but were not limited to: apprehending meaning (Bohm, 1990); questioning and clarifying (Palincsar & Brown, 1989); challenging, explaining, and expanding (Wells, 1996); repairing conversational gaps (Winiecki, 1999); reformulating (Wells, 1996; Winiecki, 1999); reflecting, advocating, and inquiring (Senge, 1994); negotiating shared meanings and creating a shared cognitive context (Crook, 1994); maintaining coherence (Collis et al., 1997); identifying a problem and suggesting strategies for solving it (Polya, 1957; Bransford & Stein, 1985); modeling, articulating the problem-solving process, and giving feedback (Collins, Brown, & Newman, 1989); and giving messages of support (Herrman, 1995). Finally, the conversational threads were analyzed in an effort to identify which snippets of conversation contained specific "moves" that kept the conversation moving in the direction of the individual's or group's intended goal.

Throughout this exercise, several features of successful conversations became apparent:

These were not simply casual conversations. There was a goal or reason for each conversation, whether it was a dialogue, a dialectic, or a design conversation. The goal was usually to identify, define, and understand problems, and to suggest strategies for solving them.

Articulation of reasoning, knowledge, or problem solving processes was crucial. In each of these conversations, participants tried to become more aware of their own thinking and reasoning; attempted to explicate it to others; and inquired into others' thinking and reasoning.

Multiple perspectives were respected and valued.

Especially in the "Cop Killer" conversation, peers who came into conflict were prompted to resolve arguments and engage in reflection, rather than deferring to authority or solidifying their stances.

Once participants were able to apprehend their own perspectives and preconceived notions, they were better able to make a conscious decision to drop them in favor of achieving group coherence.

A mix of probes and supportive comments helped to extend the conversations.

Whenever dialogue degenerated into discussion, a mentor stepped in and tried to refocus the conversation, whether through conscious reflection/explication of his/her own personal perspective; by citing historical evidence that deepened the issue under consideration; or by repairing the conversation.

In cases where the participant's intent was made clear, others were better able to make suggestions that would help the participant reach his/her intended goal.

Critiques were interspersed with supportive comments.

Critiques that addressed specific features of a participant's request for feedback were taken seriously; the author generally tried to incorporate them, where appropriate.

The body of knowledge expanded and was transformed by this open dialogue. Teachers, mentors, and students all learned from the online experience.

Implications for Practice

Based on the success factors of these three conversations, together with the synthesis of research on conversation presented in this paper, the following recommendations are made:

Have a clear goal -- Each conversation, whether online or face-to-face, should have a goal. The goal must be made explicit to teachers, mentors, and students alike. In the case of the discussions on The WEB Exchange, Civic Discourse, Vermont History Day, and Taking A Stand, the goal was to address specific standards from the Vermont Frameworks.

State your intent -- In a design conversation, it is important for the author or designer (here, a student) to explicate his/her intended goal in creating and posting a product. Requests for critique/feedback should be specific, and should match the author's stated intent. Otherwise, teachers and mentors may only respond with general comments that do not suggest specific potential solutions to identified problems. Furthermore, if the author does not state his/her intent, serious misunderstandings can result, as evidenced in the "Cop Killer" episode.

Publish and follow guidelines -- Each conversation should have guidelines that all participants are expected to follow. The WEB Project staff explicated these guidelines in two documents that were shared to all participating teachers and mentors. A Guide to Online Critique (Tavalin, 1998) presented heuristics for design conversations that were intended to help students improve their works of art and music, based on the experiences of Vermont teachers and students who had engaged in online critique in previous years. A Guide to Online Discussion (Tavalin & Boke, 1998) presented guidelines for conversation, specific connections to the Vermont Frameworks, and assessment rubrics for student online activities that were part of Vermont History Day, Taking A Stand, and Civic Discourse.

Communicate using a cycle of request/respond/reply -- Simple question/answer interactions do not constitute a dialogue. Back and forth communication is the heart of online dialogue. It begins when teachers, mentors, and other students extend the request/respond interaction into a cycle of request/respond/reply, using a mix of probes and supportive comments. Tavalin (1998) offers some tips for responding to one another's postings: acknowledge the last entry when appropriate, "snip" ideas from previous messages into the current message to repair gaps in the conversation, and pose open-ended questions for others to respond to.

Respect multiple perspectives -- In the focus group, students stated that they valued multiple perspectives. However, it is important for conversation moderators to be on guard for instances in which conflicting perspectives could cause the dialogue to degenerate into a discussion in which various participants advocated for their own viewpoints. At such times, it is important for the moderator to encourage the students to engage in reflection, critical evaluation, and synthesis of opposing viewpoints, rather than solidifying their stances, as in the "Dress Code" episode. In a best-case scenario, conflict can lead to cognitive restructuring, as observed by Crook (1994).

Articulate thought processes -- Articulation of reasoning, knowledge, and problem-solving processes are crucial to the success of a conversation, especially when participants have never met one another face-to-face. Examples of such articulation occur when students make connections between features of a text and their own experiences in and out of school, when they cite evidence to support a point of view, when they differentiate between decisions based on fact and those based on opinions, when they can explain the effects of actions and effects from different points of view, and so forth.

Align class work with online conversations and vice-versa -- The online component of instruction should be closely aligned with what is happening in the classroom. Students in the focus group mentioned that it was important for them to discuss various approaches to conversation in class before going online. Similarly, the "Cop Killer" misunderstanding was resolved in face-to-face conversation with the offending student, resulting in a "teachable moment".

Co-construct meaning both in class and online -- Mehus suggested that the amount of messages on the network should be reduced by spending more time on collective work and limiting the size of individual responses. On The WEB Exchange, there were examples of messages that presented the consensus of a group of students regarding a specific topic, or, in some cases, a summary of the conflicting views and rationales of the students in that group. This practice keeps students from being overwhelmed from a proliferation of messages and helps them relate their postings more closely to the discussion topic. It also alleviates the problem of access, especially when students are not able to read and post messages on a daily basis.

Match schools -- Some problems arose when the level of experience with Great Books discussions and the accessibility of the network were very different among pairs of schools. Mehus' (1995) recommendation to match schools by academic level and motivation is an appropriate way to deal with these problems.

Use the conferencing system appropriately -- In Winiecki's (1999) research with asynchronous learning networks, as well as in The WEB Project staff's experiences with The WEB Exchange, they found that following the conferencing system's structure (i.e., threaded discussion) tuned participants into whether they were continuing with an idea that was previously posted or starting a new message about the thread topic.

Summary

Over the past few years, Vermont teachers have been using dialogue and design conversation to help students improve their work. The WEB Project has built on knowledge gained from local artists, writers, musicians, graphics designers, teachers, parents, and students go develop an online system for improving these processes at a distance. A review of relevant literature indicates that the problems and successes of this innovative project are shared by many other educators around the world and across time. A thorough study of the nature and purpose of conversation -- particularly dialogue and design conversation -- can help practitioners to understand just how these conversations are structured, and how they can be used to improve student work. An analysis of episodes of online conversations can help to explain how and why the interchanges took place, and to identify factors that exemplify "good" online conversation.

Success factors identified in this analysis such as having a goal for each conversation, creating and publishing guidelines for online conversations, articulating thinking process, valuing multiple perspectives while resolving conflicts, having students clearly state their intent in creating a product, using a mix of probes and supportive comments to extent conversations beyond a simple question/answer interaction, and using conferencing software effectively, can inform future directions of other educational projects that intend to use online conversations to enhance teaching and learning.

References

Bereiter, C. (1994). Implications of postmodernism for science, or, science as progressive discourse. Educational Psychologist, 29(1), 312.

Berge, Z. (1997). Computer conferencing and the online classroom. International Journal of Educational Telecommunications, 3(1), 321.

Bloom, B., Englehart, M., Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive Domain. New York: Longmans Green.

Bohm, D. (1990). On dialogue. Ojai, California: David Bohm Seminars.

Bransford, J., & Stein, B. (1985). The IDEAL problem solver. San Francisco: Freeman.

Brown, A.L. (1994). The advancement of learning. Educational Researcher, 23(8), 4-12.

Brown, A.L., & Palincsar, A.S. (1989). Guided, cooperative learning and individual knowledge acquisition. In L.B. Resnick (Ed.), Knowing, learning, and instruction, Essays in honor of Robert Glaser (pp. 393451). Hillsdale, New Jersey: Erlbaum.

Chi, M.T.H., & Bassok, M. Learning from examples via self-explanations. In L.B. Resnick (Ed.), Knowing, learning, and instruction, Essays in honor of Robert Glaser (pp. 251282). Hillsdale, New Jersey: Erlbaum.

Collis, B., Andernach, T., & VanDiepen, N. (1997). Web environments for group-based project work in higher education. International Journal of Educational Telecommunications, 3(2/3), pp. 109130.

Collins, A., Brown, J.S., & Newman, S.E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L.B. Resnick (Ed.), Knowing, learning, and instruction, Essays in honor of Robert Glaser (pp. 453494). Hillsdale, New Jersey: Erlbaum.

Crook, C. (1994). Computers and the collaborative experience of learning. London: Routledge.

Deming, W.E. (1994). The New Economics. Cambridge, Massachusetts: Massachusetts Institute of Technology Center for Advanced Educational Services.

Engestrom, Y. (1996). Interobjectivity, ideality, and dialectics. Mind, Culture, and Activity, 3(4), 259265.

Fabos, B., & Young, M.D. (1999). Telecommunications in the classroom: Rhetoric versus reality. Review of Educational Research, 69(3), 217-259.

Gagne, R., & Briggs, L. (1979). Principles of Instructional Design (2nd Ed.). New York: Holt, Rinehart and Winston.

Grabinger, R.S. (1996). Rich environments for active learning. In D.H. Jonassen, ed. Handbook of

Research for Educational Communications and Technology, 665692. New York: Simon &

Schuster Macmillan.

Hall, G.E., & Hord, S.M. (1987). Change in schools: Facilitating the process. NY: State University of New York Press.

Herrmann, F. (1995). Listserver communication: The discourse of community-building. Indianapolis: Proceedings of the CSCL'95 Conference [Online]. Available: http://wwwcscl95.indiana.edu/cscl95/herrmann.html

Isaacs, W.N. (1996, January/February). The process and potential of dialogue in social change. Educational Technology, 2030.

Jenlink, P., & Carr, A.A. (1996, January/February). Conversation as a medium for change in education. Educational Technology, 3138.

Kintsch, W. (1989). Learning from text. In L.B. Resnick (Ed.), Knowing, learning, and instruction, Essays in honor of Robert Glaser (pp. 25-45). Hillsdale, New Jersey: Erlbaum.

Kruse, H., & Quinn, R. (1999). Online book discussion: Taking a Stand. Available: Holly Kruse, Cabot School, Box 98, Main Street, Cabot, VT 05647.

Mehus, P. (1995). An inter-cultural approach to literature: E-mail in the literature classroom. International Journal of Educational Telecommunications, 1(1), 85-92.

Nanard, J., & Nanard, M. (1995). Hypertext design environments and the hypertext design process. Communications of the ACM, 38(8), 4956.

Nardi, B.A., & O'Day, V.L. (1999). Information Ecologies: Using technology with heart. Cambridge MA: MIT Press.

Pea, R.D. (1994). Seeing what we build together: Distributed multimedia learning environments for transformative communications. The Journal of the Learning Sciences, 13(3), 285299.

Perry, S.M. (1992). Building better thinking skills. Available: Aurora Public Schools, 1085 Peoria Street, Aurora CO 80011.

Polya, G. (1957). How to solve it (2nd Edition). Princeton: Princeton University Press.

Schon, D.A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass.

Senge, P., Kleiner, A., Roberts, C., Ross, R.B., & Smith, B.J. (1994). The Fifth Discipline Fieldbook. New York: Doubleday.

Sherry, L., Myers, & K.M. (1998). The dynamics of collaborative design. IEEE Transactions on Professional Communication, 41(2), 123139.

Sinclair, J. McH., & Coulthard, R.M. (1975). Towards an analysis of discourse: the English used by teachers and pupils. Oxford: Oxford University Press.

Sternberg, R.J. (1998, April). Abilities are forms of developing expertise. Educational Researcher, 27(3), 11-20.

Tavalin, F. (1998). A Guide to Online Critique. Montpelier, VT: The WEB Project. Available: The WEB Project, 58 Barre Street, Montpelier VT 05602.

Tavalin, F., & Boke, N. (1998). A Guide to Online Discussion. Montpelier, VT: The WEB Project. Available: The WEB Project, 58 Barre Street, Montpelier VT 05602.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, Massachusetts: Harvard University Press.

Wang, Y. (1996, June). E-mail dialogue journaling in an ESL reading and writing classroom. Paper presented at the Association for the Advancement of Computing in Education (AACE) World Conference on Educational Telecommunications, Boston, MA.

Wells, G. (1996). Using the toolkit of discourse in the activity of learning and teaching. Mind, Culture, and Activity, 3 (2), 74101.

Wilson, B., Ryder, M., McCahan, J., & Sherry, L. (1996). Cultural assimilation of the Internet: A case study. In M. Simonson (Ed.), Proceedings of selected research and development presentations. Washington D. C.: Association for Educational Communications and Technology, in press.

Winiecki, D.J. (1999, March 2). Keeping the thread: Adapting conversational practice to help distance students and instructors manage discussions in an asynchronous learning network. DEOSNEWS, 9 (2), 1-14. Available: Post to LISTSERV@LISTS.PSU.EDU the following command: GET DEOSNEWS skip a space and then type 99-00002

Wolcott, J.R., & Robertson, J.E. (1997). The World Wide Web as an environment for collaborative research: An experiment in graduate education. International Journal of Educational Telecommunications, 3(2/3), 237-260.