4. Findings

4.1 Introduction

In Chapter 3, I presented ten research questions based on the six themes that emerged from my empirical studies and my review of relevant research. I also presented the data collection activities that were used to address each of these questions. (See Table 3.2.) Since I am using Yin’s case study methodology, I will answer each of these ten questions using all applicable data from the data collection activities. I will also find out whether the data did or did not support my initial propositions.

Since this is a descriptive study rather than one which is intended to test a preselected set of hypotheses, care must be taken to generalize these findings beyond the SOE. I will start by presenting the demographics of the samples selected from this population; then I will address each of the research questions in turn. These findings will be organized by theme, and within each theme, by research question.

4.2 Demographics

4.2.1 1995 Demographic Information

January 1995 marked the beginning of a period of change within the Instructional Technology (IT) program. More advanced doctoral students were following the old program, which contained an instructional design component as one of its five requirements. Others elected to follow the new program, a schoolwide program in Educational Leadership and Innovation that contained a Curriculum, Learning, and Technology (CLT) thread. IT classes and seminars contained a mix of ILT master’s and doctoral students. To add to the confusion, the IT master’s program was in the process of evolving into a new, broader, Information, Learning, and Technology (ILT) program housed in the Division of Technology and Special Services (TSS). Thus, when the 1995 e-mail survey was given to a convenience sample of 73 IT students and faculty in the IT classes and seminars, there was no distinction between the IT doctoral students, the CLT doctoral students, and the IT master’s degree students. Figure 4.1 presents the demographics of the 1995 survey sample.

Table 4.1

4.2.2 1997 Demographic Information

Two additional survey questions in the 1997 instrument addressed gender and program. The question regarding a home computer and modem was dropped. The survey was administered to a stratified random sample of 278 faculty, staff, and graduate students representing all the programs and divisions throughout the SOE. Table 4.2 presents the demographics of the 1997 survey sample.

Table 4.2

A summary of the responses to the question, "What program are you affiliated with", was presented in Table 3.3. To allow for a comparison with the 1995 IT sample, a proportionately larger number (N=112) of ILT and CLT students were included in the 1997 sample (107 ILT students and 5 CLT students). Since the 1995 sample consisted of a mix of IT, ILT, and CLT students, ILT and CLT respondents were not separated in the 1997 survey analysis. To maintain consistency across the two-year period separating the two surveys, I will label the 1997 ILT/CLT sample as "IT". Any sampling error introduced by combining CLT students who do not have an IT emphasis into the total IT sample should be less than 5%.

A summary of the responses to the question, "Do you have an e-mail account?", will be presented below in the section on extent of Internet use.

4.3 Theme 1: User Characteristics and Perceptions

User characteristics and perceptions comprise extent of use, reasons for use, and challenges to use of Internet tools. The extent of use (Research Question 1A) is the outcome variable that is affected by the factors that were explored using the various data collection activities. If one thinks of the SOE as an activity system, then, like Jonassen and Murphy (1998), one would characterize its members as goal-directed human beings who carry out purposeful actions through conscious intentions (see Research Question 1B, reasons for use). In using tools of the activity system, they face certain barriers and challenges (see Research Question 1C, challenges to use). Moreover, if one considers this set of Internet tools as an innovation, then the speed at which it is adopted depends on the users’ perceptions (Rogers, 1995) and their personal, task, and impact concerns (Hall & Hord, 1987).

4.3.1 Research Question 1A

To what extent is the Internet used by the SOE?

A comparison of Tables 4.1 and 4.2 shows that the percentage of respondents who reported that they did use e-mail (about 86%) remained the same over the two-year span. Extent of use, however, goes beyond a simple "Do you use e-mail?" inquiry. It involves the frequency of use, the types of tools or information technologies used, and the expertise or level of use.

4.3.1.1 1995 Frequency of Use

The 1995 survey only asked about the frequency of e-mail usage; it did distinguish between the varieties of e-mail accounts available commercially or through the university. This was because CEO was not universally available; Netscape only came out in late fall 1994; and CINS had just installed its WWW server in January 1995. All registered students could create their own accounts on Ouray or have their advisor create one for them on Carbon (the WWW server).

Table 4.3 presents the responses to the question, "Where and how often do you access your e-mail account?". Figures are given in percentages.

Table 4.3

Note that for those who read their e-mail most days, nearly one-half accessed their accounts from home, and about one-third from work. One-fifth of the respondents accessed their accounts from the SOE lab on a weekly basis. That might correspond to the night they attended class, since many IT classes were held in the SOE lab.

4.3.1.2 1997 Frequency of Use

The spring 1997 survey was distributed to a sample of SOE students, faculty, and staff. A question on Web usage was added to the original question about e-mail usage. Free accounts were still available on Ouray or Carbon for all registered students. By 1997, all faculty and staff were assigned CEO accounts. The use of free student accounts on CEO was being encouraged in classes, seminars, and labs. However, home access to the WWW was not covered under student fees until March 1998.

Table 4.4 presents the relative percentages of responses to the question, "Where and how often do you access your e-mail account?".

Table 4.4

Table 4.5 presents the relative percentages of responses to the question, "Where and how often do you access the World-Wide Web?".

Table 4.5

For frequent users, frequency of access increased from home and work and decreased from the SOE lab from 1995 to 1997. Frequent users tended to access the WWW primarily from home or work. In contrast, weekly access to either e-mail or the WWW in the SOE lab exceeded daily use in all cases, perhaps because the SOE lab had direct connectivity to all the university servers and perhaps because respondents were usually on campus once a week for class. These trends are what one would expect from a commuter campus. Interviewed students report that they find it more convenient to connect from home or work on a regular basis rather than traveling to campus and paying for parking so they could access the Internet from the SOE lab.

4.3.1.3 Types of Tools Used

In 1995, participants primarily used Carbon or Ouray for their e-mail accounts, except for the few who used commercial or corporate accounts. By 1997, the variety of available tools had increased. The percentage of survey respondents who used e-mail or the WWW and the types of accounts they used are presented in Table 4.6. The percentages do not sum to 100% because many respondents reported that they have multiple accounts.

Table 4.6

Most respondents used accounts with the newer, graphical interfaces rather than the older, text-based accounts. Interestingly, over three-quarters of the respondents reported that they used the WWW, but CEO (the primary e-mail account used) did not support WWW use at the time that these data were collected. This may be because the SOE lab computers are directly connected to the university’s WWW server, and also because many school computer networks also have WWW access.

4.3.1.4 Proficiency or Level of Use

The surveys did not address levels of use; this information was explored in the interviews and focus group. The focus group consisted primarily of novice users. The twelve interviewees reported a large spread in their expertise with computers, CMC tools, electronic conferencing, searching on-line databases, on-line publishing, managing research information, and creating simple Web pages. This was understandable, because the people who were interviewed were purposefully selected to cover the spectrum from early adopters to majority, to late adopters. The details of the adopter categorization process are presented in Appendix D.

All interviewees reported using CEO e-mail on a regular basis. The other tools they used and the frequency with which they used them are presented in Table 4.7. The identification codes for the twelve interviewees are the same as those presented in Table 3.4. Participants who were identified as early or majority adopters for the purposeful sample reported using a wider variety of tools and applications than did the late adopters, with the possible exceptions of the late adopters with identification codes 9 and 11. Their interview responses were qualitatively different as well.

Table 4.7

The focus group of SPSY interns was a more coherent group of fairly new users, as shown by the summary of skill levels (1 = low, 5 = high) in Table 4.8.

Table 4.8

The SPSY participants were fairly skilled at using computers, but had little to no experience in advanced uses of the Internet. Though the mean level of expertise with CEO e-mail was low to moderate, there was a wide spread in the expertise of the individual students in the cohort. One participant had no home access and had never used CEO. The SPSY cohort was in the process of creating an on-line conference for their professional and academic use. The participants were eager to learn about the possibilities of class-related conferences to enhance teaching and learning.

In exchange for their participation in this study, I gave their professor of record a copy of Nancy Chism’s (1998) handbook for instructors on the use of electronic class discussions. The professor later told me that the guidebook was very valuable. This might be a useful job aid to share with the rest of the SOE.

As I shifted my focus from novices to early adopters, I found that the students and faculty who had their own personal Web pages were for the most part expert users. A few faculty members still retained the basic pages that were created for them in 1995, but three of them expanded their personal Web pages to include on-line publications, class work, and scholarly products produced by their doctoral laboratories. Some students had created pages for their school, corporation, or professional society. Some pages included advanced features such as movies, image maps, and scripting. In total, there were nineteen personally authored Web pages linked to the SOE "Faculty, Staff, Students" page.

Another set of electronic artifacts was the "UCD Scholarly Publications" page. As of February 12, 1998, there were 54 publications on-line, in ten categories, published by thirteen first authors in collaboration with twenty-two co-authors. The types of on-line documents comprised journal articles, magazine articles, proceedings papers, presentations, synthesis papers, annotated bibliographies, needs assessments, evaluations, and research reports by doctoral laboratories. When this page was created in 1995, the only contributors were a handful of IT professors and doctoral students. In the past year, contributions from other programs and divisions have increased in number. Doctoral students are encouraged to post portfolio products on-line and link them to this ever-growing directory of on-line scholarly works.

Please note: there was no proposition for Research Question 1A.

4.3.2 Research Question 1B

For what reasons is the Internet used by the SOE?

4.3.2.1 Broad Patterns of Activity from the Survey Responses

A factor analysis of the responses to the survey questions that dealt with reasons for using the Internet was performed for three subsets of data: (a) 1995 IT, (b) 1997 IT, and (c) 1997 Non-IT. These revealed the general trends and changes in reasons for use of Internet tool over two years. They also highlighted the differences between the IT and the Non-IT respondents.

In 1995, the primary e-mail tool was PINE on the Carbon or Ouray servers. Internet access was for the most part text based, using Lynx on the university servers. A few individuals had commercial accounts. Thus, users generally used a command line interface to access the disparate Internet tools (e.g. telnet, ftp, Usenet newsgroups, gopher, and a non-intuitive mail system).

HTML coding was primarily the bailiwick of the innovators and early adopters, such as the Internet Task Force. No Web authoring classes were offered yet.

4.3.2.1.1 1995 IT Survey

Question 7 of the 1995 survey (see Appendix B) asked respondents how useful fourteen different activities were, all of which involved using e-mail and various other Internet tools. These activities were assigned letters from A through N, and were measured on a 5-point Likert scale (1 = not useful, 5 = extremely useful). Since the 1995 survey was only distributed to IT students and faculty, it is important to compare these results with a similar sample from the 1997 survey (i.e., IT) to illustrate the evolution of their reasons for use. This is the reason why I collected more data from IT in 1997 than from any other program.

In 1995, the principal components analysis for N=73 respondents extracted four factors with eigenvalues of 1.0 or higher. The Varimax rotation converged in seven iterations to produce four factors that, together, contributed 69.7% of the total variance. Item I ("transfer information between your home computer and your university account") did not reach the cutoff point which was set to 0.5 loading on any factor. Factor 4 contained only one item ("consult with your advisor") which contributed 7.8% of the total variance. Factors that consist of just one item do not constitute reliable "factors". Factor 4 did not reappear in the 1997 analysis.

Factor 1 contained six items. It was named "communicate and share information", and combined local access and communication items together with information sharing and dissemination items. Factor 2 contained four items and was named "find information". Factor 3 contained only two items and was named "collaborate". A full comparison of the 1995 IT responses, the 1997 IT responses, and the 1997 Non-IT responses will be presented in Table 4-9.

4.3.2.1.2 1997 IT Survey

By 1997, CEO was free for all registered students. Signup sheets were handed out in class; disks were available in the SOE lab or CINS; and CINS provided both an on-line helpdesk and technical support. CINS also gave out sets of trifold brochures and conducted free workshops in the use of Internet tools. Internet use had spread widely throughout the global on-line community via electronic journals, on-line forums, and listservs. Newsgroups were becoming a thing of the past. E-mail was still available via PINE from the mainframes using the old text based interface. However, CEO now featured a more intuitive, graphical interface that was similar to the Macintosh or Microsoft Windows desktop metaphor.

CEO still did not support Web pages, but HTML authoring courses were being taught as part of the IT curriculum, using Ouray or Carbon. The SOE lab computers had Netscape and Internet Explorer loaded on them. The university’s gopher server was replaced by a host of Web pages residing on Carbon and Ouray. Since Web page authoring spontaneously arose among early adopters in various colleges, there were no design standards. In late 1997, the Chancellor mandated a uniform look for all official pages, though students could design their personal Web pages as they wished.

Question 9 of the 1997 survey was identical to Question 7 of the 1995 instrument, allowing for a direct comparison of the two IT samples. In 1997, the principal components analysis for N=112 IT respondents extracted four factors with eigenvalues of 1.0 or higher. Factors 3 and 4 had eigenvalues very close to 1.0. The Varimax rotation converged in seven iterations to produce four factors which, together, contributed 69.7% of the total variance—the same cumulative percent of variance as the four 1995 factors.

Factor 1 contained four items and was named "share information". Interestingly, these four items contributed 44.0% to the total variance, whereas in 1995 the six items in Factor 1 only contributed 41.5%. Factor 2 also contained four items and was named "communicate". Factor 2 contained the items from the 1995 Factor 1 that dealt with communication, plus item E, "consult with your advisor", which did not load on any factor in 1995. Factor 3 contained three items and was named "collaborate", as it was in 1995. Factor 4 was named "find information" and contained three of the items that were originally found in the 1995 Factor 2.

4.3.2.1.3 1997 Non-IT Survey

A principal components analysis for N=166 non-IT respondents extracted three factors with eigenvalues of 1.0 or higher. The Varimax rotation converged in six iterations to produce three factors that contributed 68.8% of the total variance—roughly the same variance as the four factors for the 1995 IT analysis (69.7%) and the three factors for the 1997 analysis (69.7%).

Even after the rotation, Item K ("author electronic articles with hypertext links to remote sources") loaded on both Factor 1 and Factor 2. For the purposes of this analysis, I will include it in Factor 2 because its loading on Factor 2 (0.723) was in the same general range as the loading of the other three items in Factor 2, whereas its loading on Factor 1 was only 0.505.

Factor 1 contained six items which combined the four IT Factor 2 items (which made up "find information") with the two IT Factor 3 items (which made up "collaborate"). This single factor contributed 50.4% of the total variance. Factor 2 contributed 10.6% of the total variance was named "share information". It contained four items, three of which were found in the 1995 IT Factor 1 and the 1997 IT Factor 1. Factor 3 was named "communicate". It contains the items from the 1995 Factor 1 and the 1997 IT Factor 2 that deal with communication.

Tables 4.9 and 4.10 summarize all the information from the three factor analyses on reasons for use.

Table 4.9

Table 4.10

4.3.2.2 Electronic Artifacts

The SOE Web page has two areas that are rich in electronic artifacts. These are (a) the "UCD Scholarly Publications" which contains an assortment of journal articles, proceedings papers, and other electronic documents; and (b) the "Faculty, Staff, Students" page which links to personal Web pages. These can all be considered objects of activity within an activity theory framework. According to Jonassen (1998), "the object of activity can be anything so long as it can be shared for manipulation and transformation by participants" (p. 8).

Why would a student or faculty member want to create a personal Web page? Of the nineteen personal Web pages linked to the SOE, eleven authors answered that question via e-mail. Often, the final product evolved into a very different form than the creator originally envisioned. Here are the reasons given by two different students:

Personal Web pages were created for a large variety of reasons. They can serve as part of a course requirement; a research management product; an electronic portfolio; an information source for friends, family, and professional colleagues; a way to make the author’s writing more easily available to a global audience; a way to make connections with people who are interested in similar research issues; a dynamic resume during a job search; a way to showcase the author’s work in interactive multimedia and distance learning; and a resource for students and classes.

Beyond that, personal Web pages are seen as a way to present the creator’s personality and values to family and friends—a new way of carving out a public identity. A faculty member sent in the following eloquent response:

Though personal Web pages are very individualized, most of the scholarly publications were jointly authored. Often, the collaborative writing process involved e-mailing drafts back and forth from co-author to co-author, as described by participants in the interviews. Papers were posted for review by colleagues as pre-print documents for submission to professional journals, as products to be incorporated into electronic portfolios, or as drafts for presentations at professional conferences. Students commented that CEO was a convenient means for attaching electronic copies of papers to e-mail messages and sending them to their instructors for review.

4.3.2.3 Interview and Focus Group Responses

Since the interviewees were purposefully selected to represent early, majority, and late adopters, their reasons for use were varied, as expected. The breakdown of their reasons for use is presented in Table 4.11. The interviewee identification codes (ID) refer back to Table 3.4.

Table 4.11

Clearly, these reasons are very different from those given by the authors of electronic artifacts. Their primary reason for use was to access and find information, followed by communication with faculty, colleagues, and students. The omissions were as important as the stated reasons—only one interviewee had authored a web page, and she considered it a rather stressful endeavor. None had published any articles on-line, though one faculty member mentioned that a student in a doctoral laboratory had created a page for the group and had put a collaboratively authored paper on-line.

There was some interest in using the Internet for distance and distributed learning, especially for cohorts in Durango, Colorado Springs, and other distant cities. Just as students considered CEO to be a more efficient means of communication with other members of the SOE than phone calls, faculty members predicted that using CEO for distributed learning might be a good alternative to long-distance travel on a regular basis for class meetings.

Focus group participants used the Internet primarily for e-mail communication and for locating specific information to address basic research questions for term papers. They discovered that there were some useful databases of legal information that were easier to search than textbooks. They also used on-line databases to find quick answers to questions about disabilities, medical problems, and associated medications.

4.3.2.4 Proposition for Research Question 1B

It is evident that my proposition for Research Question 1B was only partly true for IT: that the primary use of the Internet was for information sharing. Communication, in contrast, was far more important than I had thought. Changes in reasons for use have definitely occurred over the past two years, with finding information slipping to Factor 4.

The importance of sharing information, communicating, and collaborating hints at the use of the Internet for forming a learning community—a trend that I explored in the interviews. The 1997 survey results do not show what Berge (1997) and Fishman (1997) found, namely, that written communication apprehension and the use of CMC to publish and share information—especially with a public audience—may constitute a serious problem for some new users. The interviews, in contrast, did show that some participants experienced problems composing messages for class conferences and had some apprehensions about being misinterpreted.

My proposition also does not hold true for Non-IT since neither sharing nor dissemination appear in Factor 1. Finding and organizing information join together with collaboration to form Factor 1. This is an important factor that contributes over half of the total variance. Thus, there are clear differences between the IT community and the rest of the SOE.

One important note is in order here. Recall that "IT" comprises both ILT and CLT respondents to the 1997 survey. CLT is a schoolwide program. In contrast, ILT is housed within a division. Since there were only five CLT members among the 112 total respondents to the 1997 survey, their contribution to this overall trend analysis is minimal.

A similar trend showed up in the interviews. Finding information and communicating with students, faculty, colleagues, and other people with similar interests were given more frequently as reasons for using the Internet than information sharing and dissemination. The majority of interviewees used CEO as their primary tool and perceived the Internet as an efficient means of communication.

Some respondents who went beyond simple messaging and participated in on-line class conferences had reservations about them, as I will show in the next section. In contrast, those who used a variety of tools also considered that it was a good means for information access, sharing, and dissemination as well as for communication.

4.3.3 Research Question 1C

What challenges to the use of the Internet are perceived as most important?

4.3.3.1 Broad Patterns of Activity from the Survey Responses

A factor analysis of the responses to the survey questions that dealt with reasons for users’ perceptions of barriers or challenges to the use of the Internet was performed for three subsets of data: (a) 1995 IT, (b) 1997 IT, and (c) 1997 non-IT. A full comparison of these three factor analyses will be presented in Table 4-11.

4.3.3.1.1 1995 IT Survey

Question 8 of the 1995 survey asked respondents to agree-disagree with 11 different questions relating to the challenges people feel when they first learn to use the Internet. These were assigned the letters A through K and were measured on a 5-point Likert scale from "strongly agree" (SA) to "strongly disagree" (SD). (See Appendix B.) Questions A,B,D,G,H,I, and J were worded negatively; questions C,E,F, and K were worded positively. In the SPSS analysis, the responses were given a numeric code and then recoded so that each question was polarized in the same direction (1 = least positive; 5 = most positive response). The comparison of these responses with the 1997 responses for IT shows how users’ perceptions of challenges or barriers to use evolved over the two-year period.

For the 1995 data, the principal components analysis for N=73 respondents extracted three factors with eigenvalues of 1.0 or higher. The Varimax rotation converged in seven iterations to produce three factors that, together, contributed 60.1% of the total variance. Item A ("I don’t have time to learn the Internet") and Item D ("getting access from home is too expensive") did not reach the cutoff point of 0.5 loading on any factor.

Factor 1 contained five items and contributed 32.5% of the total variance. It was originally named "clear payoff and value" but I later renamed it "clear benefit and value" to avoid jargon. It is equivalent to Bandura’s (1982) "value" and Rogers’ (1995) "obvious benefit". Factor 3 contained three items and was originally called "technophobia and technophatigue" by Wilson and Ryder. I renamed it "self-efficacy" to retain positive polarization and to associate it with Bandura’s equivalent construct. Clearly, we are seeing the efficacy x value pairing that Bandura identified in his own empirical research. Factor 3 contained a single item: Item B ("I have a hard time expressing my thoughts in writing"), a factor identified by Berge (1997) and Fishman (1997). It was named "mediated writing proficiency" to retain positive polarization.

Recall that factors that contain but a single item are not reliable. Moreover, if a factor does not persist over time, I do not consider it to be reliable. In the same way that the Factor 4 disappeared in the 1997 factor analysis for Research Question 1B, Factor 3 disappeared in the 1997 factor analysis for Research Question 1C. However, if these factors were ignored, then important contributions (about 8 to 10%) to the total variance by these one-item factors would not be accounted for.

4.3.3.1.2 1997 IT Survey

Question 10 of the 1997 survey was identical to Question 8 of the 1995 survey. The principal components analysis for N=112 IT respondents extracted four factors with eigenvalues of 1.0 or higher. The Varimax rotation for the 1997 data converged in seven iterations to produce four cleanly separated factors that contained all the items and, together, contributed 66.8% of the total variance.

Note that in 1995 there were three factors with eigenvalues of 1.0 or higher, which contributed only 60.1% of the total variance. Factors 2 and 4 in the 1997 analysis contain only two items each. Though they may not be particularly reliable, they cannot be ignored.

Factor 1 contained four items and was named "clear benefit and value". It contained four of the five items in the 1995 Factor 1 and contributed just about the same amount of the total variance (32.4%), as did the five items in 1995. Item H ("I can accomplish the same thing with mail and phone calls") moved down to the new Factor 3. Factor 2 consisted of Item A ("I don’t have time to learn the Internet") and Item D ("Getting access from home is too expensive"). This factor, named "time and access", contributed 14.8% of the total variance.

Here, we see a major shift from 1995 to 1997. In 1995, items A and D (i.e., time and access) did not load on any factor, whereas in 1997 the combination of these two items became the second most important factor. Note that the older Internet tools with command-line interfaces (PINE and Lynx) were part of the free Ouray and Carbon accounts. In contrast, the newer graphical browsers (Netscape and Internet Explorer) required TCP/IP connectivity, which, at the time the surveys were conducted, necessitated paying monthly fees to an Internet Service Provider (ISP) for students who wished to use these tools at home. Also, interviewees commented that they simply did not have the time to go to the SOE lab and pay for parking. They found it more efficient to connect from home or work.

Factor 3 was named "self-efficacy". It contained two of the three items (Items G and J) found in the 1995 self-efficacy factor, with Item H ("I feel I can accomplish the same thing with mail and phone calls") now replacing Item I ("I’m the type that needs a lot of hand holding"). Item I, together with Item B ("I have a hard time expressing my thoughts in writing") now formed Factor 4, which appeared to be another aspect of self-efficacy.

Together, Factors 3 and 4—which share conceptual similarity—accounted for 19.5% of the variance. The contribution of these two different self-efficacy factors to the total variance is about the same magnitude as that of the single 1995 IT self-efficacy factor (17.2% of total variance) and the 1997 non-ILT/CLT self-efficacy factor (16.4% of total variance).

4.3.3.1.3 1997 Non-IT Survey

A principal components analysis for N=166 Non-IT respondents extracted only two factors with eigenvalues of 1.0 or higher. The Varimax rotation converged in three iterations to produce two factors. Together, these contributed 54.7% of the variance. Item D ("getting access from home is too expensive") did not load above 0.5 on either factor.

Factor 1 was identical to the 1995 Factor 1, and was named "clear benefit and value". It contributed 38.4% of the total variance. Factor 2 also contained five items and was named "self-efficacy". It is a mix that not only contains the same self-efficacy items as the 1995 Factor 2 but also adds the time item from the 1995 Factor 2 and the mediated writing proficiency item from both the 1995 Factor 3 and the 1997 Factor 4.

Tables 4.12 and 4.13 summarize all the information from these factor analyses.

Table 4.12

Table 4.13

4.3.3.2 Open-Ended Comments from the 1997 Survey

The few open-ended comments from the 1995 survey were presented in Chapter Two, and were primarily concerned with potentially useful interventions. The 84 comments gathered from the 1997 survey were more varied in nature, so I clustered them into fourteen naturally occurring categories, grouped by theme. These data are presented in Table 4.14.

Table 4.14

Two of these clusters of comments—"need basics" and "no time"—deal with user characteristics and perceptions, the theme addressed by Research Question 1C. Though cost/access and usability of the Internet tools are perceived as challenges to new users, they rightfully belong in Theme 3. However, in the factor analysis, time and access were paired, so cost/access cuts across the boundaries of the six themes that I identified. The other comments, even if they deal with barriers, will be addressed in subsequent sections that discuss the other major themes. Note that there were no comments that dealt with group learning, adoption, and conceptual change.

Of the 84 respondents who replied to the open-ended final question ("We’d like to make it easier for you to use the Internet effectively. What other suggestions do you have for us?"), six felt that they did not have sufficient basic knowledge to use the Internet. Here are two typical comments:

Three respondents mentioned time as a barrier:

Four respondents pointed to cost/access as a major challenge:

Beginning in March 1998, the University finally began to offer full TCP/IP access through student accounts on Ouray and through faculty accounts on Carbon, so the issue of getting home access to the Internet is finally being addressed.

4.3.3.3 Interview and Focus Group Responses

Research Question 1C addressed not only the challenges that Internet users face but also their concerns, frustrations, the things they hated about it, their horror stories, and issues they faced that were not covered in the surveys. In the focus group, the respondents were relatively new users. Their primary concerns were technical, as seen in the following comments:

One focus group participant clearly distinguished between clear benefit and value and lack of access:

Since the interview participants represented a wider range of expertise and levels of adoption, their responses showed greater depth and variation. Table 4.15 summarizes their challenges and concerns, ordered by frequency.

Table 4.15

There was no single set of concerns that stood out from the others. As Hall and Hord (1987) note, there are many types of concerns ranging from personal to task to impact. That is evident from the interview comments.

First, there were personal concerns about ethical issues—confidentiality, privacy, and the implications about putting one’s research on-line. One student’s story was particularly poignant:

In contrast, another student cautioned us about the need to balance security with one’s right to free expression:

A faculty member clearly referred to the discomfort and ambiguity that participants face in presenting their thoughts to a public forum and interpreting other participants’ written messages:

Second, there were the technical problems and task concerns that novices faced, such as disliking reading text on-line, the primitive interface on PINE, the inability to download and read one another’s attachments, insufficient computer capacity, and computer crashes. These were magnified by a lack of guidance and technical support.

Progressing to the third level—impact concerns—respondents mentioned that participation in class conferences was often inequitable. Certain students dominated the conversations; postings from other students often went unanswered; and side conversations off the designated topic were distracting. Often, replies to deep questions were superficial.

One faculty member noted:

A second faculty member specifically addressed the impact issue by asking, "What is the impact of the technology on people that are using it? And how is it changing?" Regarding the new impetus to start putting course work and lab work on-line, a third faculty member who was concerned about distance learning responded with another question:

4.3.3.4 Proposition for Research Question 1C

In the surveys, there was no distinction between early and late adopters, so the pattern that emerged from the factor analysis—namely, the pairing of efficacy (about one-third of the variance) x value (about one sixth of the variance) that reflects Bandura’s findings—was seen across the entire sample, for both years. In contrast, the mediated writing proficiency factor in 1995 was unreliable. It consisted of a single item that did not reappear in the 1997 survey analysis. Interestingly, reservations about writing to a public audience did appear among the interview statements.

Referring back to Table 4-15, one can see that two faculty members identified reservations about writing to a public audience as one of their concerns. Moreover, two students suggested that some off-topic conference messages should be sent as "reply to sender" rather than to the entire group, and two students indicated that they preferred to read other people’s messages rather than respond to the conference themselves. Although mediated writing proficiency did not appear in the survey results as a reliable, major trend across programs, it was definitely a concern expressed by interview participants. And although the mediated writing proficiency factor from the 1995 factor analysis was unreliable and conflicted with the importance that it was given by the interviewees, it must be addressed if future on-line conferences are to be successful.

The emergence of a time and access factor in 1997 for IT but not for the rest of the SOE was interesting, even if the factor only consisted of two items. Time and access did show up in seven of the 84 comments and suggestions collected from the final, open-ended 1997 survey question. A focus group participant who did not master the tools pointed to lack of convenient access as a major problem. I reiterate: the unreliability of a time and access factor across the surveys does not mean it can be ignored—especially since it was mentioned in the open-ended survey comments; was pointed out by a focus group member; and was identified as a serious barrier by one of the members of the seminar in which the CEO class conference took place.

Problems with the usability of the SOE’s network and software were also identified, and several students did air their frustrations in using them. These will be dealt with in Theme 3. Though participants in the various data collection activities aired their frustrations regarding the tools, few actually went as far as to give up using them. All of the new users and late adopters who used CEO felt it was worthwhile. Those who really expressed negative sentiments primarily referred to problems with PINE on Ouray or Carbon.

One majority and one late adopter student stated that they preferred to read the messages of others rather than posting to electronic conferences or class lists themselves. In the interviews, faculty members discussed the issues about writing to a public audience without the social cues that one normally finds in face-to-face communication. Thus, there is some support for the second part of the proposition.

4.4 Theme 2: Cultural and Organizational Issues, Norms of Use,

Cultural and organizational issues revolve around the SOE as a cultural entity. Generally cultures share social meanings via a common language and a mutually agreed-upon set of signs and symbols. They share common beliefs, values, assumptions, norms and conventions, and experiences. Within the cultural context, the members perceive that certain types of activities are rewarded or considered legitimate or appropriate.

To Jonassen and Murphy, "the community negotiates and mediates the rules and customs that describes how the community functions, what it believes, and the ways that it supports different activities" (1998, p. 9). Continuing the loose parallel with an activity system framework, Theme 2 addresses the SOE’s norms, conventions, and rules.

What constitutes "legitimate" or "appropriate" activity? To Lave and Wenger (1991), it is a way of describing the relations between newcomers and "old-timers" within the cultural community, and the activities, identities, and artifacts they share. As newcomers gradually become part of a community of practice, their intentions to learn are engaged and supported, and their meanings are formed by carrying out intentional, legitimate actions within that community (Lave & Wenger, 1991, p. 29).

Within this framework, the SOE has (or does not have) an incentive structure for using the Internet, together with a set of appropriate activities that may (or may not be) consonant with its norms and conventions. Moreover, if neither an embedded incentive structure nor a commonly agreed upon set of legitimate or appropriate uses of Internet tools exist, then can the SOE be thought of as a single, cohesive culture? These are two important areas that I will explore in this theme.

4.4.1 Research Question 2A

How does the incentive structure of the SOE influence the types and levels of use of the Internet?

4.4.1 1 Interview Responses

Some of the probes used in the interviews dealt with the extent to which Internet use was culturally or personally compatible with respondents’ lifestyles; clear benefit, value, or rewards for using it; perceptions of persuasion or coercion to use it; and the balance between any payoff versus the time spent in learning how to use it to support teaching and learning.

The early adopters were not particularly concerned with any extrinsic incentives or motivation. Internet use was already an integral part of their lives, irrespective of the School’s norms. The following comments characterized the early adopters’ views. They reflect Rogers’ (1995) definition of personal compatibility—the degree to which users perceive an innovation to be consistent (or inconsistent) with their existing values, past experiences, and ongoing needs:

Other interviewees stated that extrinsic pressure was exerted to mandate or persuade them to use the Internet:

A staff member expressed personal interest in using the Internet. However, her comments expressed a sense of personal incompatibility because learning new skills might add new, unrewarded responsibilities to her current job:

Two other interviewees explored the relationship between an incentive structure and the use of the Internet to support teaching and learning. A staff member saw how making a clear relationship between Internet use and the School’s goals could be construed as an incentive:

In contrast, a faculty member who regularly used CEO for communication had reservations about how the Internet might support instruction:

Another faculty member elaborated on the subtle difference between an incentive system and an impetus to adopt Internet tools:

4.4.1.2 Focus Group Responses

In contrast to the widely varying interview responses, the SPSY focus group had reached consensus that there was a clear benefit and value in using CEO, though they had not used other Internet tools to any great degree. This does not reflect any incentive system or form of persuasion inherent in the SOE’s culture. Rather, clear benefit and value relates most closely to the Factor 1 that was identified in Research Question 1C. It is the compelling need for students to engage in the discomfort attending the learning of new technologies—the same factor that Wilson and Ryder (1995) found in the earlier IT focus group.

SPSY cohort members were also quite aware that if they weren’t connected, they might just be left behind. Here are some typical responses:

4.4.1.3 Proposition for Research Question 2A

Interestingly, seven of the twelve interviewees—including all of the early adopters—perceived that there was indeed a lack of incentive structure to use the Internet, yet they voluntarily used it, and used it regularly. Although early adopters do not necessarily pay attention to incentives, they clearly benefit from them. In contrast, all of the majority interviewees felt that there was some form of external pressure to conform, whether through persuasion, coercion, or a clear message from a professor that the use of Internet tools was expected within a specific class. The late adopters were mixed: two felt there was indeed an incentive structure, and two did not. The focus group participants all described their perception of incentives as a clear payoff (convenient, efficient communication with faculty) or a desire not to be left behind, rather than an embedded part of the School’s culture.

Table 4.16 summarizes the views of the twelve interviewees. Faculty responses are shown in boldface.

Table 4.16

Despite my impressions from conversations and informal meetings with faculty, three of the four faculty members and one of the two staff members stated that though there was no specific incentive structure, they used CEO regularly for communication, sharing papers on-line, or advising students. The early adopter faculty member and policy maker were also actively exploring advanced uses of the Internet to support distributed learning and class conferencing because they saw value in it, based on their past experiences and future predictions. Thus, for this sample, those faculty who saw a clear payoff and value for using the Internet used it voluntarily, whether or not they perceived any extrinsic rewards or persuasion.

Additionally, the scattering of interview responses indicates that there may yet be no coherent belief regarding any set of incentives that spans the entire SOE, whether intrinsic or extrinsic. This stands in sharp contrast with the focus group participants who expressed consensus and a shared value system. Thus, if a culture can be construed as a group of people with a common set of beliefs, values, and assumptions, then the SPSY cohort reflects an emerging sub-culture within the SOE.

4.4.2 Research Question 2B

The SOE is a system. The nature of any system is to hold itself together against any disintegrating forces that operate within its component subsystems (Schein, 1996a). As we have just seen, the SPSY cohort represents one of these subsystems that may, in fact, share different aims and goals as a culture from those of the School. As Deming (1994) states, every system must have an overarching aim that is clear to every member of the system. In many educational organizations, this is presented in a terse mission or vision statement that reflects its systemic aims and goals.

The SOE’s vision statement is stated on its Web page as "Providing leadership for learning in Denver’s diverse and thriving metropolitan communities". Exactly what does leadership consist of? Leadership is people: people who plan, organize, motivate, and support learning, innovation, and change within learning organizations. Leadership is new knowledge: a scholarship of practice that links all members in a common cause for improved learning. Leadership is involvement: creating the types of learning environments that these members would like schools at all levels to espouse.

Thus we see, within the bounds of a traditional educational system, aspirations for fostering change and for creating innovative environments to support teaching and learning. These aspirations are meant to be the goals or intentions that drive the actions taking place within the School as an activity system. Its informal goals, however, are less clear, possibly less innovative, and encourage the creation of traditionally structured scholarly products as exemplified in the 1994-97 doctoral portfolio requirements.

Next, we move to the level of objectives. These are clearly stated in the doctoral handbook as disciplined inquiry, professional engagement, and professional leadership and commitment. Evaluation of one’s progress in reaching these objectives takes place through review of scholarly products that are prepared for the purpose of fulfilling the School’s academic requirements. If the electronic artifacts that are posted and publicly shared by faculty and students match these objectives, and if they were intentionally created to do so, then they are in consonance with the administration’s vision. Similarly, if the on-line conferences and other uses of Internet tools support these objectives, they, too, are in consonance with that vision.

As we have seen in Research Question 1B, the reasons for use stated by the interviewees were to access and find information, followed by communication with faculty, colleagues, and students. Only two early adopters mentioned disseminating information and furnishing leadership and support for teaching and learning. The SPSY cohort and the majority-to-late adopters among the interviewees saw the Internet as a convenient means of research on-line and CEO as another means of communication rather than an aid to teaching and learning. As one faculty member pointed out, "It’s easier, often, than the telephone, although that’s equally good. And better than voice mail, which, I find, a very clumsy device".

The Web page authors, in contrast, described their intentions in terms of creating a place to manage information including synopses of papers, class notes, research tied to topic foci, and their publications; to showcase Web-related graduate work and portfolio products; to provide their students with class resources, news, assignments, and a place to share products; and to serve as a point of contact for people interested in their work or their program. When asked why he created a personal web page, one faculty member stated, "Because it is critical in my field. People look for information on the Web first. You are not considered topical if you don’t have a Web page". This statement and the other reasons enumerated under Research Question 1B all reflected the School’s vision statement and the objectives of the doctoral program in Leadership and Innovation.

Why, we may ask, are only a handful of faculty and students using the Internet to disseminate their scholarly products, to exhibit leadership and involvement in the educational community, to serve as models for other educators, and to engage in activities that are ostensibly sanctioned by the administration? First, let us explore the problems and the associated suggestions for improvement that were mentioned in the open-ended comments on the 1997 survey. Next, let us investigate the issues that were discussed in the interviews. Finally, let us look in more detail at the on-line publications and scholarly products.

4.4.2.1 Open-Ended Comments from the 1997 Surveys

Referring back to Table 4-14 under Research Question 1C, we note that 17 out of the 84 respondents offered suggestions that relate to the norms and conventions of initial Internet use within the School as an organization. Comments under the "class require", "scheduling", and "hands-on" keywords indicated that they would like on-line activities and hands-on exploration in their introductory classes so they could learn the basics of the Internet; more flexible scheduling of seminars and workshops; labs available when needed for practice; and more opportunities for hands-on practice. Here are some typical comments that reflect these concerns:

These open-ended survey responses bring up policy considerations that ought to be explored but that are beyond the scope of this study. Can we ask professors to require their students to use the Internet? Is it appropriate for Internet basics to replace one week of an introductory graduate course? Should students be explicitly steered to CINS and asked to take a free introductory workshop whenever they can fit it into their busy schedule? One student’s comment regarding Internet use for class-related activities was insightful:

4.4.2.2 Interview Responses

In answer to the question, "What do you consider to be appropriate or legitimate uses of Internet tools, within the SOE, and why?", one interviewee began with the statement,

A faculty member’s response may be construed as an answer to this question:

Thus, we must distinguish between what is appropriate per se, and what is appropriate within the norms and conventions of the SOE. This study deals with the expressed perceptions of SOE members, not with the general issue of "legitimacy" or "appropriateness"—a construct that one can only speak of within a specific cultural context and community of practice.

Here are some of the uses that the interviewees considered appropriate or legitimate, based on their own experiences. Perhaps these activities might be considered appropriate in other institutions of higher learning; perhaps not.

1. On-line conferences, discussions, and chats for doctoral labs;

2. On-line conferences and committee meetings for faculty;

3. Collaborative sharing of research papers;

4. Drafts of papers and articles on Web pages posted on-line for review;

5. In-class modeling of technology tips and ideas from other students;

6. Suggestions for some professional listservs that students may join;

7. On-line communication with outside experts in your research area;

8. Learning how to search for on-line articles and resources;

9. Distance learning with instructor-as-guide rather than technology to support traditional lectures and rote learning; and

10. Course information, lectures, and tips on doing class assignments posted on the Web.

When I probed a policy maker about policy implications in one of the interviews, he suggested that we take a broader viewpoint regarding appropriateness:

A staff member who also served as a student advisor gave this pithy advice:

4.4.2.3 UCD Scholarly Publications

Currently, there are 54 on-line publications on the "UCD Scholarly Publications" web page, under ten major headings. One notes immediately that 34 of these publications—more than half—were jointly authored. Roles reversed from time to time as co-authors became lead authors, and vice versa. Moreover, authors often crossed over from one research area to another.

As Jonassen and Murphy stated, within an activity system, very little activity is accomplished individually. "Individuals involved in a particular activity are simultaneously members of other activity dynamics going on concurrently...each component of an activity is the result of other activities which produced it" (Jonassen & Murphy, 1998, p. 12).

This process is clearly evident in the contributions of the UCD faculty and students to the SOE Web site. Some of the publications are scholarly products such as the research reports in the High Achieving Classrooms for Minority Students (HACMS), an ongoing doctoral laboratory jointly sponsored by the Educational Psychology and Language, Literacy, and Culture programs. Others are publications written by students and/or faculty, reporting on needs assessments, case studies, and evaluations of new instructional strategies at the K-12 level. Still others are theoretical papers that deal with innovative learning environments, new models of teaching, learning, evaluation, assessment, technology adoption and integration, school reform, distance learning, and communities of learners. Many of these papers reference one another, in a continuous effort to construct knowledge collaboratively.

Scholarly publications are not posted capriciously. To qualify as a scholarly publication, the product must be a proceedings paper, a journal article (or at least a final draft), a magazine article, a research report or synthesis paper approved for a student portfolio, a scholarly product generated by a graduate seminar, or a work of equivalent quality. These represent significant contributions to the research literature, and, as such, represent the culmination of the School’s activities in fulfilling its stated objectives.

Considering that there are about 100 faculty members and thousands of students in the SOE, why are there only 54 on-line publications? Part of this has to do with designing text for on-screen presentation—an issue that is more germane to Theme 3, tools, design, and impersonal supports. An interviewed student stated that reading on-line text was not visually appealing:

She alluded to new forms of literacy that need to be explored and developed—visual literacy, media literacy, and literacy in terms of the Internet and e-mail—other forms of literacy that are appropriate if the School really wants to be a community of learning. A professor suggested that it takes additional cognitive skills to deal with on-line text:

Another student agreed that reading papers on-line was an appropriate use of the Internet because it increases both one’s cognitive processes and one’s writing ability.

Reading on-line text, however, is a very different goal from publishing on-line documents. Not one of the interviewees expressed the same views that the authors of personal Web pages did—that disseminating their writings to a public audience could contribute to educational leadership and innovation by presenting valuable information and helping to shape the opinions of their readers. Clearly, on-line publishing is not among the commonly accepted "legitimate" activities at this time.

Another important issue is that of confidentiality—a topic that was raised in Research Question 1C, challenges to use. A faculty member stated:

Interestingly, none of the interviewees mentioned copyright issues, the misuse of proprietary information, plagiarism, or other concerns that are mentioned in the current literature, as reasons for not publishing on-line.

4.4.2.4 Proposition for Research Question 2B

Currently, some students are producing electronic portfolios, and Web pages as research management products; and they are definitely contributing to the "UCD Scholarly Publications" Web page. However, are they really ahead of the professors in publishing on-line documents? Eleven of the Web authors did use their personal pages for disseminating full-text versions of their publications—three faculty members and eight students. In examining the scholarly publications page, which espouses a clearly stated set of high standards for publication, I found that the contributors consisted of nine faculty members, twenty students, and seven unaffiliated second authors. Of the nine faculty members, three were students within the SOE when they first conducted the research that formed their contribution to the papers that are currently posted on the scholarly publications page.

Thus, faculty do lag behind students in publishing on-line documents—in number, but not in quantity of contributions. Six faculty members served as either lead authors or co-authors in at least two of the scholarly publications. The first two statements in the proposition are supported by the data. Students are producing on-line research management products—an activity that is fully consonant with the requirements of the graduate programs. They do, in fact, publish more papers than the faculty.

It is not clear whether or not the lag in publication by the faculty might be explained by a fear of compromising intellectual property rights, at least not by the data that were collected in the interviews. Though not specifically mentioned in the interviews, copyright issues for on-line documents are becoming very important for two reasons. First, classroom teachers are apprehensive of copyright infringement. Second, both the American Psychological Association (APA) and the editor of Educational Technology Publications have a stated policy that if a document is published on-line, it is considered published and, therefore, will not be accepted as a contribution to a professional journal or book. This, however, is not true of many other professional societies, notably the Institute of Electrical and Electronics Engineers (IEEE), The Association for Computing (ACM) and the Association for Advancement of Computing in Education (AACE).

4.5 Theme 3: Tools, Design, and Impersonal Supports

At this point, I have dealt with the individual users and the norms and conventions of the community to which they belong. Continuing with the parallel between my conceptual framework and activity theory, the third theme addresses the tools or mediating artifacts of an activity system—specifically, the tools, design, and impersonal supports that are germane to the university’s networking system. As Tikhomirov (1981) mentions, these tools transform human activity. For the SOE, Internet tools are integral parts of the telecommunications system in which communication, access, and sharing of information take place. In this sense, Internet tools are the artifacts that are used to carry out goal-directed activities such as developing skills, building knowledge, and transforming information into products or outcomes that are considered appropriate or legitimate by members of this learning community.

To what extent do these tools mediate such activities? As Horton (1994) indicates: to the extent that they reduce the effort required by the recipient to decode and interpret whatever information is being communicated. And, in Allen and Otto’s (1996) framework: to the extent that they decrease the individual’s biological cost to perceive, acquire, and retrieve information from the environment. Thus, if Internet tools require more effort to use than the time and energy that the individual might spend in using different tools or an alternate procedure, they will most likely not be used.

4.5.1 Research Question 3A

What improvements to the University and the SOE’s human-computer interface (HCI) design and available Internet tools are suggested by new and continuing users?

4.5.1.1 Open-Ended Comments from the 1997 Survey

In the 1997 survey, eight respondents considered usability to be an important challenge. Several of the comments under the "usability" keyword dealt with the PINE e-mail interface on Carbon and Ouray. Here are some typical comments:

With the availability of free CEO accounts that use a more intuitive e-mail interface, these concerns may now be alleviated. However, students still complain about the difficulty of setting up their modems at home.

A second class of problems identified in the 1997 survey was the fact that the tools available in the lab may not necessarily have the functionality needed to support the activities that students are expected to carry out. For example, there has been a heated debate within the Internet Task Force since 1994 concerning whether to use templates and HTML authoring tools or whether to train users in basic HTML coding. One survey respondent stated simply, "Buy authoring tools". New users would like to make this choice for themselves.

Along with usability and functionality comes the issue of impersonal supports—brochures, booklets, on-line tutorials, and other forms of print-based or electronic performance support that do not require one-on-one interaction with a graduate assistant, fellow student, faculty member, or staff member. Under the keyword "manuals", there were seven comments from comments who requested various types of impersonal aids and supports such as "how to do" sheets, quick reference guides, step-by-step tutorials, and booklets to refer to often. Comments included:

As we shall see in Theme 4, these respondents represented the minority opinion rather than the majority.

4.5.1.2 Interview and Focus Group Responses

In answer to the question, "What improvements would you suggest for the design, interface, Internet tools, and usability of the computers in the UCD labs?", half of the interviewees stated that they rarely—if ever—use the SOE lab, primarily because of parking hassles and long commutes. One interviewee explained:

Similarly, the consensus of the focus group was summed up by two of the participants:

For those interviewees who did use the labs, several problems surfaced. For example, one interviewee repeated the oft-used mantra—"More RAM":

Interviewees were also asked if they might suggest any design or interface improvements for computers at home or at work. One student referred to some ergonomic considerations:

Another student described her mechanical troubles and problems with usability when she tried to use CEO at home:

A third student referred to the problem of transferring files between PC’s and Macintoshes, and even within the same platform, if the programs were incompatible. A policy maker described the origin of this problem:

Other interviewees referred to the difficulty of reading text on screen, an issue that was already discussed under Research Question 1C. Whether this problem is due to the reduced resolution of whatever is printed on a computer screen vs. the resolution of ink and paper; the difference in illumination methods (books are illuminated by an outside source while computer screens provide their own illumination); the fact that paper allows for scanning and rescanning of difficult text; or the relative position of the reader’s eyes to the position of the presented text—the relative importance of these contributing factors is mere speculation at the moment. However, it is clear that many members of the SOE prefer to print any documents that are sent to them, and then work from the paper copy.

A student and two faculty members also referred to usability and functionality from a more cognitive point of view:

One student who has been teaching courses in the SOE as an adjunct instructor made an interesting observation about the differences between new and continuing users:

4.5.1.3 Proposition for Research Question 3A

Based on the survey, interview, and focus group responses, the proposition holds. New users definitely prefer CEO over the UNIX systems for communication, even though there are still ongoing problems with modem settings, software installation, and reliable connections for home computers. Since all of the interview and focus group participants regularly used CEO, the more user-friendly interface on CEO may be an important factor in increasing its usage over that of Carbon or Ouray.

As I have already mentioned under Research Question 1C, respondents suggested that ISP cost could become a problem. In fact, one interviewee commented:

With TCP/IP connectivity now offered through Ouray, ISP cost will no longer be an issue. Moreover, the new version of CEO supports many of the advanced e-mail features of Eudora, so we may expect to see some changes in usage in the foreseeable future. However, these two software upgrades are so new that it may take a while for students to learn about them and adopt them.

An unintended consequence of providing full TCP/IP connectivity through student accounts is this: increased Web usage requires increased speed, bandwidth, and system capacity. Although acquiring and keeping a reliable connection to the WWW through Ouray was easy when the upgrade was first introduced, the ease of access and reliability of home connections may decrease as more new users come on-line.

4.6 Theme 4: Social Issues: Scaffolding, Mentoring, Communication

Within any activity system, there is always a division of labor. Jonassen and Murphy (1998) describe the division of labor as the task specialization that is prescribed by and for individual members of groups that exist within a learning community. I interpret "division of labor" to refer to a person’s role within a community of practice. From a situated learning perspective, Lave and Wenger (1991) refer to the authority of masters or old-timers versus the legitimate peripheral participation of newcomers within a community of practice. Collins, Brown, and Newman (1989) refer to the differences in meaningful activity carried out by skilled practitioners and novices. These differences embed the learning of cognitive skills and knowledge in a social and functional context that supports a cognitive apprenticeship model that incorporates personal scaffolding, coaching, and mentoring—a model that is wholly different from the traditional, decontextualized methods used in formal schooling.

From a systems perspective, Deming suggests that improved practice comes from encouraging communication within an organization:

Regardless of the perspective, it is clear that a learning community consists of: (a) novices who are seeking information and building knowledge; (b) old-timers or experts who can model appropriate behavior, who can coach and scaffold novices through the learning process; and (c) channels of communication and dialogue that join all members of the organization.

4.6.1 Research Question 4A

What changes to the University and the SOE’s communication and support structure are thought to be most helpful to overcome barriers and support Internet use?

4.6.1.1 1995 Survey

In the 1995 survey, 69 of the 73 respondents ranked a set of eight supports for training and performance using e-mail and the Web, with 1 = most useful support and 8 = least useful support. The eight aids and supports from which respondents were to choose, as shown in Appendix B, were (a) formal classes, (b) brochures, (c) booklets, (d) on-line tutorials, (e) paper-based tutorials, (f) interactive computer demos, (g) individual attention by graduate assistants, and (h) one-or two-hour workshops.

As I have already mentioned in Chapter 3, some survey respondents confused "ranking" with "rating"—some ranked, and some rated. This is a problem that I attempted to alleviate in the 1997 survey by making the instructions more explicit.

Table 4.17 shows the 1995 percentages of the rankings of these preferred supports, adjusted by SPSS for missing data. Initially, I tried to present this table as a three-dimensional chart, but it was nearly impossible to read. Instead, I highlighted the relative maxima in each column of the table. For example, 32% of respondents ranked formal classes as their first choice, whereas 13% of respondents ranked formal classes as their eighth choice out of the 8 options presented. 17% of respondents ranked individual assistance by graduate students as their first choice, and another 17% ranked this option as their third choice.

Table 4.17

Scanning this table for the maxima in each category, one can see that the supports ranked highest by the survey respondents were formal classes (ranked #1 by 32% of respondents) and workshops (ranked #2 by 30% of respondents). Booklets were least popular, with over 20% of the respondents ranking it #6 or #8. The other types of impersonal supports were ranked somewhere in the middle, with brochures (ranked #3 by 17% of respondents) and interactive computer demos (ranked #3 by 19% of respondents) ranking higher than on-line or paper-based tutorials.

The frequency distribution for individual attention by graduate assistants was bipolar: 22% of the respondents ranked it #8, whereas 17% of the respondents ranked it either #1 or #3. This bipolarity was worthy of further investigation.

Recall that, for the data analysis, the eleven challenge items in the survey were all recoded (1 = least positive; 5 = most positive response). Recall, too, that 1 is a "high" (most positive) ranking for a preferred support and 8 is the "lowest" (most negative) ranking for that support. Thus, a positive correlation would occur when there was a low value for a particular challenge factor and a high value for the relative ranking of a particular type of support.

Using the above scales, I found significant positive Spearman correlations (p < .05) between the 1995 self-efficacy factor and the relative rankings of both formal classes (+0.26) and individual attention by graduate students (+0.29). I also found a significant negative correlation between the self-efficacy factor and the relative ranking of booklets (-0.25). These correlations led me to the conclusion that respondents who were low in self-efficacy (least positive response) considered personal supports and scaffolding to be relatively useful (as shown by high rankings) as compared with impersonal supports such as informative booklets (as shown by a low ranking).

I also found a significant negative Spearman correlation (p < .05) between on-line tutorials and mediated writing proficiency (-0.27). From this, I concluded that those who encountered difficulty in finding a voice and having something to say (least positive response) may not have considered impersonal supports such as on-line tutorials to be particularly useful.

Interestingly, by 1997, the mediated writing proficiency factor had disappeared. As I have stated in my discussion of Research Question 1C, any factor that contains a single item is not usually very reliable. Evidently this was true of the mediated writing proficiency factor.

4.6.1.2 1997 Survey

Were there any major changes in these relative rankings two years later? Table 4-18 shows the patterns that emerged from the SPSS frequency distribution of rankings of aids and supports from the 1997 survey. Again, relative maxima are highlighted.

Table 4-18

As in 1995, formal classes (ranked #1 by 27% of respondents) and workshops (ranked #1 by 32% of respondents) topped the list of preferred supports. Booklets (ranked #8 by 24% of respondents) were still ranked lowest. The changes in the overall patterns, however, were noticeable. Whereas brochures dropped in overall rank (ranked #7 and #8 by over 17% of respondents), interactive computer demos (ranked #1 and #3 by 19% of respondents) rose in overall rank. Only paper-based tutorials remained in the middle (ranked #5 by 17% of respondents). The bipolarity in the ranking for individual attention by graduate assistants had disappeared, with 25% of respondents ranking this personal type of support as #1. In contrast, on-line tutorials were now bipolar (ranked #2 by 18% and #7 by 19% of respondents).

These rankings indicate that the respondents preferred personal scaffolding to impersonal supports—with the possible exception of "interactive computer demos". This surprised me because other lines of evidence did not support the apparent interest in interactive computer demonstrations.

In searching for an explanation for this conflict, looked carefully at the transcripts of the interviews and the focus group. I soon realized that the words "interactive computer demo" had different meanings for different people. Though I interpreted this type of job aid as a form of computer-aided instruction (CAI), two of the interviewees interpreted it as "a demonstration of CEO, in the lab, by a skilled mentor, with interactive participation by attendees". Thus, they considered it to be a form of personal, rather than impersonal, scaffolding.

If the survey instrument were to be used again, this ambiguous point would definitely need to be clarified. However, because both the high ranking of interactive computer demonstrations and the bipolarity of the ranking of on-line tutorials did show up as interesting survey findings, I will include them among my recommendations. Those who might wish to continue to explore these two forms of scaffolding can deliberate this point among themselves.

A second interesting finding was this: there were no significant correlations between either the clear benefit and value factor or the self-efficacy factor and any of the suggested types of scaffolding. Whereas in 1995, respondents who scored low in self-efficacy or mediated writing proficiency preferred personal scaffolding to impersonal scaffolding, that was no longer true in 1997. I needed to find the root of this unreliability. Why had the earlier correlations disappeared?

Examining the interview responses, I found that personal supports seemed to be valued across adopter categories, but there were individual preferences in the types of scaffolding that the interviewees actually did find, or might find, useful. Among the interviewed students who sought help with Internet tools, an early adopter preferred "show me" mentoring sessions; a majority student preferred explanations from "real people", for example, help via telephone by a CEO expert; and a late adopter wanted formal classes and workshops but didn’t know where to find them. All faculty and the policy maker, regardless of adopter category, requested "show me" personal help and mentoring. The early adopter staff member requested both written job aids and formal demonstrations of CEO, while the late adopter said she would like to see a demonstration of CEO in the SOE lab on a regular basis.

Another confusing issue surfaced as I examined Table 4.18. Why were so many types of aids and supports ranked #1, I asked? Handwritten notes on the collected 1995 survey forms had indicated that there was some confusion on the part of the respondents regarding the ranking scheme. Therefore, to clarify my intentions in the 1997 survey, I reworded Question 11 to read [boldface retained]:

Respondents, however, still did not read the directions carefully. Handwritten notes by respondents in the margins of the 1997 survey forms attested to this. Moreover, many respondents employed a rating system rather than a ranking system, using the same number more than once. Thus, while the trends are apparent, the actual data were not clean—they represented a collection of responses that include both ratings and rankings. Were I to use this survey instrument again, I would use ratings rather than ratings, and I would reword Question 11 so that it was absolutely clear.

4.6.1.3 Open-ended Comments from the 1997 Survey

The open-ended comments from survey respondents shed light on their impressions about the SOE communication channels as well as their preferred types of support and scaffolding. In fact, the relevant comments (21 in all) were more numerous in Theme 4 than in any of the other themes.

In the analysis of the open-ended comments, there were two three keywords that had eight or more comments (about 10% of the total comments): (a) "class-require", (b) "usability", and (c) "communicate". Respondents’ suggestions to require training in Internet tools and basics as part of the required, introductory classes were dealt with in Theme 2. Improvements to usability and requests for written materials were treated in Theme 3. "Communicate", along with the keywords "GA/modeling" and "‘live help", rightfully belongs in Theme 4 and will be discussed here.

Communication—or a lack thereof within an educational system—has many aspects. First of all, lack of communication can reflect an unfamiliarity with the existence of resources that are already available, as seen in the following comments:

Part of this unfamiliarity may be due to the fact that the SOE Web Page brochure is available right outside the SOE lab, but the other CINS brochures are found on the second floor. Few, if any, SOE students ever frequent the CINS facilities on the second floor, judging from the survey responses.

Second, lack of communication can indicate that students are aware of resources, but that they feel these resources do not communicate sufficient information to meet their needs:

We cannot judge from these comments whether or not these students were aware of the free CINS workshops, the on-line tutorials produced by CINS and the SOE, the on-line helpdesk on CEO, and the "Getting Internet Help" section on the SOE web page, or whether they found them personally incompatible with their needs and wants.

Third, lack of communication can indicate that students are aware of the resources, that they often find them helpful, but that they would like more or better information communicated to them on a regular basis:

Regarding personal scaffolding, two respondents indicated that they would like "live help" available either on-line or by telephone:

Currently, CEO has an on-line helpdesk. I asked the CEO contact person in the SOE how many members of the School actually used it. "Not many, but many individual users out in the districts and at large do", was the reply. Did they find it helpful? The answer was clear, but the interpretation left much to be desired:

I posed the same question to the CEO contact person in CINS and received the following reply:

These statements back up the statements of the survey respondents who said that they want a "live person" at the other end of the line, whom they know and respect, to answer their individual questions and respond to their specific problems, rather than an impersonal, on-line helpdesk.

Seven respondents specifically asked for graduate assistants to help them with activities in the lab, especially during hands-on work following a demonstration. Here are some typical comments:

Another student suggested peer-to-peer mentoring as an alternative:

On the one hand, individual attention and mentoring by graduate students is a good example of the type of scaffolding suggested by Collins, Brown, and Newman (1989). It is also consonant with Everett Rogers’ (1995) notion of client-change agent empathy. But is it a practical solution?

Judging from the experiences and statements of the focus group members who have served as graduate assistants, it may not be possible to "hire GA’s who know this stuff", as the above respondent requested. For example, can we really expect a graduate assistant in the lab to have expertise in SPSS, HTML coding, and all the rapidly changing types of software that the lab supports?

4.6.1.4 Interview Responses

What type of support and scaffolding are people really looking for? The importance of personal "show and tell" sessions and mentoring by colleagues was underscored by one of the interviewees who was both a graduate student and an instructor in the SOE:

A faculty member described her "wish list" quite clearly:

Another faculty member had his "wish list" fulfilled in the person of a colleague from another program:

Regarding the communication and support structure of the School as a whole, the interviewees gave mixed responses. First of all, a direct visit to CINS solved one student’s problem:

In contrast, a faculty member’s question to the CEO on-line helpdesk went unanswered:

Lack of awareness of workshops and incompatibility with needs and wants—as we have already seen in the survey comments—were two issues brought up by several interviewees, as illustrated in the following comments:

A third issue, that of getting some information and support but needing more, was also common to both the survey comments and the interview responses:

Under the survey comments, I encountered the question: is it really practical to provide GA assistance on a regular basis? A staff member recounted her struggle with this issue in the SOE lab:

In contrast, another staff member would be quite content with a written job aid that included frequently asked questions. She had also experimented with optional CEO demonstrations in class, but was dissatisfied with the outcome and suggested a more workable approach:

Since CEO is the most frequently used type of Internet communication in the SOE, a flexible schedule of demonstrations with optional student attendance might be a good way to provide direct instruction, hands-on practice, and a way for individual students to pose their questions directly to the experts. A policy maker concluded with this statement:

4.6.1.5 Focus Group Responses

Of the six focus group participants, three had served as graduate assistants and had experiences trying to model and coach other students. Thus, their enthusiasm for individual mentoring was a bit more tempered. They suggested a variety of approaches:

The dialogue with the focus group was really interesting because it showed that the participants had developed multiple perspectives on the issue of technical support. On the one hand, they were in favor of individual modeling and coaching, but on the other hand, their direct experience showed them that students have a variety of learning styles, and the greater the variety of learning styles, the greater should be the range of supports to accommodate them.

4.6.1.6 Proposition for Research Question 4A

Both of these statements are supported by the data. Granted, there are workshops available for the faculty. However, once the initial training is over and the professor is alone in his/her office with a computer and new, unfamiliar tools, problems are bound to arise that require one-on-one mentoring or technical support at a level that is currently unavailable. Faculty members who have found mentors among their fellow professors are most appreciative for this collegial help, but not all are in this lucky position.

Students, too, find themselves in the same situation in the lab, and especially at home. There have been many requests for individual help from the graduate assistants in the SOE lab. However, those who served previously as graduate assistants realized that the demands on their knowledge and time were simply too overwhelming to allow them to mentor students at the level that they would like and still carry out their regular duties.

4.6.2 Research Question 4B

How does the way that SOE members are joined to communication channels and other individuals influence their use of the Internet?

As I have mentioned before, activities within a learning community are socially and contextually bound. Thus, I posit that there is a social influence factor that affects individuals’ use of Internet tools as much as technical support, coaching and mentoring, and communication. Fishman (1997) states that "the extent to which someone will use electronic mail (or other CMC tools) could be predicted by the use of those tools by relevant co-workers or supervisors" (p. 19).

To explore the social influence factor, I broke the eighth question in the interviews and focus group into two parts. The first part of the question asked participants how the way in which they are joined to the School’s or their program’s channels influence the way in which they use the Internet. The second part asked participants to name three people that they talked to (or communicated with) most frequently about how they use the Internet for teaching and learning. Most participants preferred to identify these three people by their roles rather than their proper names.

Another, indirect way of exploring the social influence factor was to examine the electronic artifacts—especially those that grouped according to specific research areas or topic foci.

4.6.2.1 Focus Group Responses

From introductory statements and responses to Interview Question 8 by the SPSY focus group members, it was evident that these participants were joined to one another and to their professors by communication channels.

In fact, the focus group participants planned to begin their own SPSY Cohort electronic conference on CEO at the time the focus group was conducted. However, when asked whom they communicated with most frequently about using the Internet and technology, no tight linking was apparent, as indicated in Table 4.19.

Table 4.19

It was also evident from the following comments that were unaware of, or did not use, the total range of aids and supports that are currently offered by CINS and the SOE.

4.6.2.2 Interview Responses

Whereas the focus group respondents were primarily novice users and were all in the same program, the interviewees represented a broad range of expertise and several programs. Their social connections and communication channels were also quite varied. Here are some typical comments from the early adopters:

And here are some comments from the majority and late adopters:

When asked whom they communicated with most frequently about using the Internet and technology, Table 4.20 shows that the social circles of the early and late adopters were quite different. The early adopters communicated with more students and classmates, and more frequently with colleagues outside the SOE.

Table 4.20

Little mention was made about connection to communication channels with CINS and people within the SOE who could furnish support and scaffolding. Some interviewees—irrespective of rate of adoption—had developed their own support system:

For their own reasons, others avoided reading technical support notices of publications altogether, or were apparently unaware of them.

One student identified an inconsistency, not only in Internet use per se, but also in communicating the vision about how it was to be used:

One characteristic of a self-sustaining system is that a consistent vision is communicated to all stakeholders. Thus, we may ask, are we dealing with a single system in the SOE, or are we dealing with a set of loosely connected sub-cultures and sub-groups, each with its own version of appropriate use, formal and informal support structure, and communication channels?

4.6.2.3.Electronic Artifacts

Looking at the UCD Scholarly Publications web page, one can immediately identify the subgroups that have contributed to this collection of full-text, on-line publications. First, there is the HACMS lab—a coherent group of faculty and students who have worked on the same grant for several years. All of their papers have the same set of first author and co-authors. Next, there is the IT cohort, a fairly cohesive group of master’s students, doctoral students, recent graduates, and colleagues outside the university who are centered around the Instructional Design doctoral labs, the Instructional Technology seminars, and the Internet Task Force. Many names of cohort members appear repeatedly on individually or jointly authored papers. Finally, there is a disparate group of master’s and doctoral students with varied interests that do not necessarily involve educational technology, from which students have submitted one, or at most three, products in fulfillment of the requirements for various classes or seminars.

There is a single publication from a member of the Educational Policy and Administration (EPA) doctoral thread—a publication with a focus on instructional strategies for distributed learning. Absent are any publications from the Integrated Family Services (IFS) doctoral thread.

Though trifold brochures were distributed to all members of the SOE this past year, documenting the contents of the SOE Web page and including the UCD Scholarly Products, some faculty members may not know about these collected works.

A similar scenario can be found in the student and faculty Web pages. As mentioned under Research Question 1B, these pages were created by IT members for the intent of (a) managing their electronic portfolio and other research associated with their studies; (b) communicating information to friends, family, and people with a common interest in instructional technology; (c) showcasing scholarly products to serve as models for other students; (d) disseminating ideas, opinions, and values to a wide audience; (e) serving as resources for their students and classes; and (f) providing prospective employers with up-to-date vitae and resumes. In all cases, the sphere of influence of these Web authors is global, rather than local.

4.6.2.4 Proposition for Research Question 4B

Both of these statements are supported. Based on the number of electronic artifacts that represent information dissemination mechanisms, the CLT cohort—especially those who are involved with instructional design and technologies—have adopted the Internet earlier than the rest of the SOE.

Similarly, the SPSY cohort has decided as a group to explore the possibilities of using an electronic conference to support group communication and learning. To wit, the one focus group participant who was not currently an Internet user made the following comment:

Regarding awareness of workshops and training, there were two schools of thought. There were some interviewees and focus group participants who genuinely didn’t know what the CINS publications or the SOE’s technical support resources were. Then, there were those who were aware of them but did not choose to avail themselves of them, either because they thought that "Internet basics" training was not an appropriate way to spend class time at the graduate level, or because they had already formed an alternative support structure through collegial mentoring and coaching.

Perhaps publicizing the existing aids and supports through current or planned communication channels might reach the first group, but it will probably fall on deaf ears for those who choose not to use them for clearly elucidated reasons of their own.

4.7 Theme 5: Individual Learning, Adoption, and Conceptual Change

Returning to the parallel between the SOE and an activity framework, we note that every activity system has an outcome or goal of the activity. Theme 5 represents this goal or outcome: individual learning, adoption, and conceptual change.

Jonassen and Murphy state that "activity theory believes that conscious learning emerges from activity (performance), not as a precursor to it" (Jonassen & Murphy, 1998, p. 3). In short, the individual (or subject) uses tools (here, Internet tools) to intentionally act on an object (e.g., read a message, access an on-line database, revise a Web page, or otherwise transform an electronic artifact of some sort) to produce an outcome (intentional learning). The tasks, operations, and actions that transform the object produce the intended outcome. Thus, in an activity system, the tools or mediating artifacts help the individual to transform something. In this regard, Jonassen and Murphy make their strongest statement: "From this perspective, traditional classrooms fail because learners are not engaged in activity; they are not transforming anything" (Jonassen, 1998, p. 10).

Moreover, the continued practice of using tools within a community of practice results in a reciprocal relationship between (a) the tools; (b) the individuals who are members of that community, and who reflect its norms and conventions, and its roles or division of labor; and (c) the objects that they transform using those tools. In effect, the tools themselves evolve or are customized to meet the needs of the individuals. This leads directly to Rogers’ concept of re-invention. Rather than using an innovation as it was initially intended, or as it has been used elsewhere, users tend to re-invent an innovation when they apply it to different types of problems. This puts a completely different focus on the original, linear concept of adoption:

It is also entirely consonant with Vygotsky’s concept of learning as the social construction of knowledge—thus making a clear connection between learning and adoption. Though it is impossible within a social framework to separate individual learning, adoption, and conceptual change from that of other members of the learning community, for the purposes of this study it is easier to break them out and study them separately. Thus, I will save group learning, adoption, and conceptual change for the final theme.

4.7.1 Research Question 5A

How do activities involving the use of Internet tools impact individual learning, adoption, and conceptual change?

4.7.1.1 Interview Responses

I have already discussed the reasons for use, individual user concerns, and barriers or challenges to the use of Internet tools in Theme 1; and the usability and design issues in Theme 3. As I moved on to Theme 5, I was interested in finding out how individuals participated in knowledge-building conversations with other members of the learning community, and how these interactions affected their own individual learning. The ninth interview question specifically addressed how Internet activities such as electronic conferencing and e-mail messaging impacted the interviewees’ own process of learning, conceptual change, adoption, and use of Internet tools.

Netiquette was an issue that plagued all categories of adopters. Though the issue of cultural norms was addressed earlier in Theme 2, strong objections to perceived violations of netiquette did not surface until the issue of relating Internet use to individual learning was addressed. Are there cultural norms of mediated conversation? (See Hafner and Lyon, 1996.) Or does the learning community invent these cultural norms as the conversation progresses? The following two comments were made by a late adopter and an early adopter, respectively. To them, the conversational norms were pre-existing cultural conventions:

Regarding the impact of Internet tools use upon learning, adoption, and conceptual change, responses were varied and seemed to depend on the adoption category of the interviewee. The late-adopter interviewees occasionally participated in electronic conferences but preferred to lurk rather than actively engage in dialogue. Here are some comments from the late adopters:

Moving on to the majority adopters, I found that they were aware of the anonymity afforded by electronic conferencing. Rather than expressing the ways in which CMC influenced their own learning and conceptual change, they concentrated on the ways in which mediated communication differs from face-to-face communication:

In contrast, the early adopters were stimulated and excited by on-line dialogue, and perceived that CMC facilitated generative learning experiences. Here are some typical comments:

4.7.1.2 Focus Group Responses

The SPSY focus group participants felt comfortable with electronic messaging, but had not yet participated in any electronic conferences. They described their experiences with e-mail more in terms of comfort and convenience. However, they had not seen any major impact on their own learning, conceptual change, or adoption at the time. Here are some of their comments:

4.7.1.3 Analysis of Electronic Conference Messages

Wilson et al. (1994), Ryder and Wilson (1996), Sherry and the UCD Internet Task Force (1996), and Wilson et al. (1996) all studied electronic messaging and conferencing within the SOE. This study is an extension of that ongoing work.

I wanted to find some examples of the use of CMC that really impacted individual learning and change. The interview and focus group participants appeared to concentrate on individual motivation and comfort rather than any real cognitive changes. Thus, I found it necessary to explore the dialogue within a successful class conference as an alternative data collection method.

For this analysis, I chose the spring 1997 Advanced Quantitative Methods seminar. The class membership consisted of the instructor and ten students who completed the course. The course was highly structured, with both a midterm and a final project presentation—a situation that struck terror into the hearts of many of the students, since they had not taken exams in many years. Participation in the class electronic conference on CEO was purely voluntary.

After the seminar was over, I saved all the messages. Then I sent a closure questionnaire to the instructor and all ten students. (See Appendix F.) Three weeks later, I sent a reminder letter to the non-respondents. All but one of the class members completed and returned the questionnaire.

Seven of the eleven class members had participated in the conference and posted a total of 107 messages. Three posted infrequently (1 to 8 messages); three posted frequently (15 to 19 messages); and one posted very frequently (41 messages). The thirteen conversation threads are shown in Table 4.21.

Table 4.21

The 107 messages clustered into thirteen response types that were associated with four of the six major themes, as shown in Table 4.22.

Table 4.22

Since Brown and Palincsar (1989) measured reading comprehension by the number of responses from each student in four response types: questioning, clarifying, summarizing, and predicting, I measured both individual and group learning, adoption, and conceptual change by the number of responses that reflected my two themes. I felt that the last three response types actually reflected group learning, since the postings were made to the group rather than simply to an individual. Moreover, these messages often built upon the answers, clarifications, and feedback given by other class members earlier in the conversation. It was at this point that Themes 5 and 6 began to merge. The data could not easily be disaggregated.

First, I examined the number of messages by theme. The majority of messages sent by members of the class reflected individual (44 messages) and group (24 messages) learning, adoption, and conceptual change. The remaining messages fell into two other themes, namely: legitimate activities supported by the CEO conference and the norms of use of the class (19 messages), and social issues that dealt with scaffolding, mentoring, and communication (20 messages). Surprisingly, there were no messages dealing with individual user characteristics and perceptions such as concerns about use of CMC or self-efficacy, nor with the design or functionality of the conferencing tools.

Next, I examined the messages by response type. Since there were 107 messages and thirteen different response types, I considered that any response type that appeared more than ten times (about 10% of the total messages) was important. These important response types were: "report progress", "support/share", "ask question", "I understand!", and "give clarification".

Again, there was an interesting parallel between these response types and those of Brown and Palincsar’s (1989) students:

1. "Report progress" is somewhat like Brown and Palincsar’s summarizing. It was the most frequent response type in Theme 2.

2. "Ask question" is the same as Brown and Palincsar’s questioning.

3. "Give clarification" is the same as Brown and Palincsar’s clarifying.

4. "I understand!" is not quite the same as Brown and Palincsar’s predicting, but it is the sign that people really did undergo an individual conceptual change—i.e., they learned, and they were able to express what they learned.

5. Brown and Palincsar did not mention "support/share", but it is very important for the development of a learning community. It shows that individuals were willing to give both moral and intellectual support to one another and to share their resources with other members of the class.

Note that Herrmann (1995) found that listserv communications tended to fall into three types: administrative, academic (or information transmission), and supportive. It was those "support/share" messages that were the "glue" that kept the Advanced Quantitative Methods conference together.

Finally, I examined the messages by sender identification code. Table 4.22 shows the percentage of responses by response type and by sender identification code.

Table 4.22

Here are some important themes that emerged from the data analysis:

1. Other than the single message which accounts for 100% of one student’s responses (a fairly comprehensive evaluation of one of the required reading articles), the response types seem to scatter fairly evenly among the seven conference participants.

2. Five of the seven respondents posted at least one "I understand!" message—a clear sign of individual conceptual change.

3. With the exception of the person who posted the single reading evaluation message, all respondents, including the instructor, contributed at least one "support/share" message, a sign that the class was functioning as a social group.

4. Though only two respondents specifically asked for feedback from the group on their final projects, four participants gave feedback to them.

5. Purely social interactions that were not associated with the syllabus or class topics were sparse. Proliferation of off-topic messages is a problem that has been seen in other electronic conferences; it did not happen here.

4.7.1.4 Closure Questionnaires

What happened to the other four individuals who never participated in the conference? One student reported that she had no access to e-mail at home or work. A second student used his individual account from work, so although he occasionally participated in the electronic dialogue, his messages were not captured on the CEO conference. A third, the class "lurker", stated,

The fourth student did not reply to the questionnaire or to the reminder.

It is at this point that the closure questionnaires filled in the missing information. Since the questionnaire responses showed such a tight linkage between individual and group learning, adoption, and conceptual change, I will save the analysis of these open-ended comments for the sixth and final theme.

4.7.1.5 Proposition for Research Question 5A

Whereas the early adopter interviewees thought that mediated communication sped up learning, adoption, and conceptual change, the majority and late adopters were not so convinced. The focus group responses shed no light on this issue because none of participants had ventured beyond ordinary e-mail use into the full range of CMC. In the electronic conference, the active participants were practicing higher order thinking skills and generative learning activities, to wit, posting critical reviews of required reading articles, participating in collaborative problem-solving of difficult examples in advanced quantitative methods, and providing constructive feedback on other students’ final projects for revision and refinement.

A fuller answer to this question will be found in Theme 6. For now, I will say that the CEO conference did have some impact on individual learning, adoption, and conceptual change. Even those who participated less frequently by posting messages to the group from work, did, in fact, have about the same percentage (approximately 13%) of "I understand!" messages as did the more active participants. Theme 6 will also present the surprisingly astute and positive comments from the class "lurker", as well as some rather insightful comments from the infrequent participants.

4.8 Theme 6: Group Learning, Adoption, and Conceptual Change

In this closing theme, I explored the relationship between individual and group learning. In any activity system, all elements are interrelated. According to Schein (1996a), the community functions as a learning organization in which not only the individual members, but the organization as a whole, undergoes generative learning or a perspective transformation. I will present the findings from the Advanced Quantitative Methods closure questionnaire first so that I can maintain continuity with the previous theme.

4.8.1 Research Question 6A

How does individual learning, adoption, and conceptual change influence the other members of the community to which these individuals are culturally linked?

4.8.1.1 Closure Questionnaire

The first open-ended question in the closure questionnaire asked participants how participation in the electronic conference may have enhanced or detracted from their own experience of learning. Table 4.23 summarizes their answers.

Table 4.23

These responses shed light on the issue of "lurking". Though respondents with identification codes 7, 8, and 10 posted zero or one message each, they did read the responses of other class members and felt the conference was a helpful method of information exchange, discussion of the required readings, modeling, and feedback from the instructor. In contrast, the respondents with identification codes 1 through 4, who did participate actively, reported that it helped them construct knowledge independently, outside of the usual class sessions. Moreover, for them, individual learning, adoption, and conceptual change were definitely enhanced by the collaborative problem-solving approach. One respondent told a story that clearly shows the interaction between her own individual learning and that of the group:

The next two questions dealt with any form of anxiety, apprehension, or unintended limitations or negative aspects that the participants might have experienced when dealing with the electronic conference. As expected, the student who did not have access from work stated that she had no computer anxiety; her problem was simply lack of access. Others encountered initial difficulty with the CEO settings and said they could have used more technical support, but they were able to resolve their technical problems early in the semester.

Two of the active participants said that they were afraid that they might intimidate other users and hog the conversation, but since the affective feel of the class meetings was one of camaraderie, their apprehensions were most likely unwarranted. One mentioned that she didn’t want to upset other students when they were wrong and she knew it; she was concerned about how her criticism may have been interpreted and whether or not her responses were appropriate.

The instructor stated that she felt no anxiety, but rather, some pressure to check the conference daily and to respond quickly to questions. Other students reported the same type of pressure to post their responses in a timely fashion for the benefit of the group. Several respondents asked why there wasn’t more participation from the rest of the group, since lack of participation limited the feedback they were getting. Here are some insightful comments from two of the infrequent participants:

Interestingly, these reasons were not related to actual lack of knowledge or skills in using the network, written communication apprehension, perceived lack of self-efficacy, nor any reasons that I originally expected. The "lurker" had no problems with the technology; access; or multiple, concurrent topics and discussion threads. She clearly identified the cause of her lack of response to messages as personal incompatibility:

The fourth question asked respondents to suggest how an electronic conference might complement in-class discussions. Interestingly, one of the most positive responses came from the class "lurker":

Another insightful response came from one of the most active participants. She alluded to a "shared memory" idea—one that was also mentioned by the policy maker and by Crook (1994) in his discussion of longitudinal continuity.

Table 4.24 summarizes the questionnaire responses.

Table 4.24

In the fifth question, class members were asked what preparation and support they felt were necessary in order for an electronic conference to work successfully. Table 4.25 summarizes their comments.

Table 4.25

The class "lurker" alluded to many of the barriers that have already been discussed in Theme 1. Her commentary is a nice summary of the salient factors that can pose challenges to the success of electronic conferences:

There was general consensus on the usefulness of a short CEO training session, provided it was not mandatory. However, there was no consensus on home access. Interestingly, students were not fond of the idea of circulating printouts of on-line messages for those who did not have access, because they felt that could detract from the participatory nature of the conference.

The netiquette issue, which first surfaced in Theme 5, was also discussed here. In contrast to the above quote, in which the student was generally positive about posting discussion guidelines and expectations, one of the active participants was not. To her, it was not simply a question of knowledge, skills, and access to Internet tools but a matter of developing the air of camaraderie that she had described previously—a collegial, respectful tone that supports inquiry and validates all sorts of responses:

When asked whether they would like to use e-mail in future courses, all respondents either agreed or strongly agreed that they would—with the following reservations. The student who did not have home access stated:

One of the active participants stated:

The next section of the questionnaire used Cooperrider and Srivastva’s (1987) appreciative inquiry approach to delve into the success factors for the electronic conference. Class members described their most memorable experiences in a multitude of ways:

1. the richness of the questions/answers/ discussion among the participants;

2. the discussion of articles and debate about questions;

3. the flexibility for a diversity of individual expression;

4. the instructor’s help with all the problems about EMS;

5. help with the repeated measures error terms;

6. preparing for exams and trying to get the EMS concept down;

7. getting into the discussion and having the constructive criticism as well as the compliments for my ideas;

8. a classmate’s posting of crib sheets; and

9. having other students give you input on our final projects.

What caused that experience to be so memorable? Other than the one student who tried to use someone else’s crib sheets and didn’t find them very valuable, there was relative consensus among the other conference participants. They felt that that the sharing of ideas; the ability to have problems diagnosed on-line; hearing multiple perspectives on common issues of concern; and receiving feedback on final projects, all contributed to a worthwhile learning experience.

Here are some of the success factors that the class members identified in the questionnaire:

1. The frequent responses of many class members;

2. This is my image of what computer-supported collaborative learning is all about. It’s nice to see it in practice, and to be a part of it.

3. Partially because you could see this "teachable moment", where everyone in the class was totally out to lunch, but we were still trying and the professor had the opportunity to diagnose the holes in our learning.

4. Amount of discussion back and forth plus people trying to show different ways of how to figure it out. It helped to see different ways of doing it and how people thought about it.

5. Having everyone in class and the teacher share my ideas.

6. I thought they would be of great help, invaluable as the final approached, but it’s difficult to use someone else’s crib sheets and it ended up not being very helpful for me.

7. I always appreciate comments from my cohort.

The closure question asked class members how they envisioned themselves using electronic conferencing or any other Internet tools to support their own growth and learning in the future. Their responses are presented in Table 4.26.

Table 4.26

I will now return to the second part of the proposition for Research Question 5A, which states that:

Respondents with identification codes 1 through 4 were active participants in the class conference; those with codes 5 through 8 were infrequent participants; and respondents 9 and 10 did not post at all. Other than the student with identification code 5, all the rest of the comments indicate that the collaborative learning process was valuable and meaningful to them, and that they planned to use Internet tools to support their learning and growth in the future. Examining the rest of student 5’s responses, it is important to note that she felt the preparation for the exams, the assistance with the difficult concept of EMS, and the multiple approaches of different students toward the same problem definitely contributed to her learning and growth in the course. Therefore, I do not feel that her response was anomalous.

In closing, I would like to return to the interview and focus group responses to see if (or how) their perspectives differed from the electronic conference participants regarding group learning, adoption, and conceptual change.

4.8.1.2 Interview and Focus Group Responses

Since the SPSY cohort had never used e-mail for anything except personal communication, they really had little to add. Here are a few representative comments:

Though the responses from the members of the class conference shed more light on the connection between the use of Internet tools and collaborative learning, the interview responses allowed participants to examine their own self-image, their teaching and learning styles, and their relationship to the SOE learning community.

Are the early adopters and the technology-savvy people really perceived as radicals? Are they trying to change the culture of the SOE? Judging from these comments—yes.

Do they encourage group dialogue with their colleagues that leads to adoption and conceptual change? Sometimes—yes…

…and sometimes—no:

Have they seen learning, adoption, and changes in perspective occur through the use of electronic conferencing and the use of Internet tools? Is it worth the extra effort? In their own words—yes. Connections are made, meaning is negotiated, knowledge is constructed, and problems are solved collaboratively.

Can the same thing be said for the majority and late adopters? While several have not yet made a connection between the use of Internet tools and collaborative learning, two students were able to voice their own reservations about using CMC to enhance distributed cognition. These comments reflect aspects of the personal/cultural compatibility factor identified by Wilson and his colleagues (1996):

How can the use of e-mail messaging or class conferencing be improved? A majority adopter student suggested an on-line variation of the customary manner of breaking a large class into small discussion groups:

The policy maker suggested that capturing the text-based comments on an interactive chat session could contribute to a "collective memory"—again, the same concept as has been suggested by Crook (1994a) as one of the important benefits of mediated conversation:

Since faculty members who had actually used Internet tools in their classes made most of the really insightful comments in this theme, it is appropriate to close this chapter by addressing the final proposition.

4.8.1.3 Proposition for Research Question 6A

Both parts of this proposition are supported by the comments made by the interviewees. Whereas the early adopter faculty members have incorporated technology into their teaching and learning, make effective use of it, and are able to express exactly how they are making effective use of it, they tend to perceive themselves as radicals, or at times, iconoclasts.

Recalling Rogers’ (1995) emphasis on client-change agent empathy, it is clear that these faculty members are often fighting an uphill battle in attempting to influence their peers to do likewise, especially colleagues like the following professor who expressed his reluctance quite candidly:

4.9 Summary

In January 1995, when this study unofficially began, only a handful of IT "techie" faculty and students used the Internet for information access and sharing. They used either commercial accounts or the old PINE e-mail interface on UCD accounts for communication. Marc Andreesson had just introduced Netscape in October 1994, and the university made its new WWW server available for faculty and student use three months later. A great deal has changed since then. In fact, it has changed so rapidly that it is very difficult for researchers to conduct the large-scale, controlled, longitudinal studies that can lead to reliable findings.

Table 4.27 summarizes the growth, diffusion, and changes that have taken place between the winter of 1995 and the spring of 1998. The rows in this matrix represent short summaries of the answers to the ten research questions posed in this chapter.

Chapter 5 will expand on this table to explore potential future diffusion of Internet usage within the SOE.

Table 4.27

Table 4.27 (Cont’d.)

Table 4.27 (Cont’d.)

Table 4.27 (Cont’d.)