Sherry, L., Lawyer-Brook, D., & Black, L. (1997). Evaluation of the Boulder Valley Internet Project: A Theory-Based Approach to Evaluation Design. Journal of Interactive Learning Research, 8 (2), 199-234.
Copyright © 1997 AACE All Rights Reserved
EVALUATION OF THE BOULDER VALLEY INTERNET PROJECT: A Theory-Based Approach to Evaluation Design Abstract This paper summarizes a two-year evaluation of the Boulder Valley Internet Project (BVIP). A wide variety of evalulation methods were used, most notably the construction of a theoretical model called the "Integrated Technology Adoption and Diffusion Model" That guided data collection, analysis, interpretation, and reporting. The results were interpreted in relation to several theoretical models of the innovation adoption process within complex educational environments. _______________________________________________________________________ Overview The Boulder Valley Internet Project (BVIP) was established to introduce teachers in the Boulder Valley School District (BVSD), Colorado, to the use of the Internet. The premise of this five-year, NSF-sponsored initiative was that the Internet can provide a rich variety of tools and resources that may be used to enhance instruction and communication by students, teachers, and administrators. Five research questions were aligned with the project's objectives. These focused on the effectiveness of the training component; the change in the participants use of the technology; the influence of the project on curriculum and instruction; the project's impact on the school and district as a whole; and possible uses of the model by other districts. Data addressing the research questions were collected from surveys, focus groups, interviews, a work group, an embedded case study, and an examination of BVIP system logs. Though participants' use of telecommunications has increased due to a successful peer training program and expanded district-wide connectivity, integrating Internet-based activities into the classroom is encountering barriers in terms of time lags and lack of classroom access. Moreover, the technology and its associated tools changed significantly throughout the duration of the project. The data collected from this case study yielded an enormous amount of information and revealed the complexity of the issues. It became necessary to create a model that could identify key explanatory factors and organize them into a coherent framework. The data both supported and greatly expanded previous models of technological barriers, individual user perceptions, and technology adoption and diffusion that were identified in the review of relevant literature. As a result, the evaluation not only addressed the five research questions but also organized findings into a comprehensive model. The Integrated Technology Adoption and Diffusion Model has four strands with multiple sets of variables that impact the effectiveness of the technology adoption/diffusion process: technological, individual, organizational, and teaching and learning factors. Findings were interpreted using nationwide surveys of technology usage, Rogers (1995) Diffusion of Innovations, and Farquhar and Surry's (1994) Adoption Analysis Tool, and Jones et al.'s (1995) Engaged Learning model. Generalizing these findings to other settings will depend on the district's degree of site-based management, its level of support for telecommunications, and the level of involvement of all parties comprising the its ecological culture. History The Boulder Valley Internet Project (BVIP) was a collaborative venture between the University of Colorado at Boulder and the Boulder Valley School District, primarily funded by the National Science Foundation. Activities and funding were also coordinated with the Annenberg/CPB Math and Science Project. The BVIP was designed to deploy Internet connectivity to the BVSD, enhance curricula, analyze the effect of the network technology on the educational process, and integrate the activities into the workings of the district and surrounding community. The project was intended to pioneer a comprehensive approach to K-12 use of networking and the Internet, where planning and implementation involve district, teacher, university, community, and regional participation. The BVSD contributed funds, staff, release time, and the testbed itself. The University provided funding, facilities, staffing, assessment, network expertise, and curricular guidance. In the initial phase, Colorado Supernet (the Internet service provider for the State of Colorado) and Westnet (an NSF-funded regional network that provides Internet connectivity to a six-state region) furnished connectivity, network platforms, and operational support. The project was housed in the district office. The project leader, Libby Black, was hired by the collaborative project team to lead the project because of her background in K-12 education. After the second year of the project, she was supported by an administrative assistant who coordinated the Internet accounts throughout the district. In 1995, the BVSD funded a half-time position for a webmaster to coordinate the BVSD home page. At the close of the project in the fall of 1996, both the project leader and the administrative assistant were hired by the BVSD as full-time employees to sustain the project's momentum. The project leader, with support by the University of Colorado, trained a cohort of 26 participating teachers from the BVSD in the summer of 1992. These teachers, in turn, became instructors and resource personnel for their fellow teachers. The objectives of the initial intensive training for the core group of peer trainers, and for later generations of participating teachers, were to become proficient in the use of e-mail, to become comfortable investigating Internet resources, and to become experienced at exploring ways to integrate the use of the Internet into their curricula. During the training period, the peer trainers spent an average of five to seven hours per week on the Internet, with most of that time in electronic mail (e-mail). Both the teachers who were already sophisticated users of technology and the self-described "non-techies" reported that they learned a tremendous amount about electronic networks and the Internet, and that they were very enthusiastic about the Internet's potential for promoting teaching and learning. The overall project, from deployment to curriculum, was subject to ongoing assessment, with active dissemination of results. At the beginning of the project, Dr. Kenneth Klingenstein of the University of Colorado was the principal investigator for the grant. A year later he was replaced by Dr. Kenneth Wolf, also of the University of Colorado. Together with the project leader, Dr. Wolf carried out a formative evaluation at the close of the first year of the project in 1993. In 1994, RMC Research Corporation, Denver Office, submitted a proposal to perform subsequent formative and summative evaluations of the project as an external evaluator. RMC was awarded the evaluation contract by the BVIP because of its expertise with instructional uses of technology. The evaluation team consisted of Dr. Dianna Lawyer-Brook and Lorraine Sherry, both of whom had worked extensively with educational telecommunications. Over the five year span of the BVIP, both the project and the evaluation design evolved significantly as connectivity increased, Internet tools became more user-friendly, teachers and students acquired Internet knowledge and skills, and telecommunications began to garner popular support. In 1991, however, when the project designers first created their expansive vision for the BVIP, little was known about effective integration of the Internet into a school district. Even potential benefits and barriers were a matter of conjecture. The project designers believed, from information available at that time, that the Internet could be of benefit to the educational community. Thus, they developed a plan that included the development and delivery of comprehensive district-wide training over a three-year period beginning in July 1992 for a critical mass of teachers who wished to use the Internet for instructional purposes, and the establishment of an initial foundation for curriculum-related resources and activities. The value that could be added to the district's curriculum and instruction by the use of Internet-based activities and resources, however, was an unknown quantity at the time, and is just beginning to be explored at present. The initial cadre of 26 teachers used a "trainer of trainers" model to diffuse their expertise throughout the BVSD via formal classes, school presentations, individual tutoring, troubleshooting, and technical support. The project personnel also developed a set of ancillary materials and resources, a series of structured classes, and a set of open, unstructured laboratory workshops. They elicited feedback from new trainees to continually update and redesign the training program. Later, they created a home page for the BVSD to support the training program, to render assistance to new users, and to disseminate information about the project throughout the district. Formal classes were offered for professional development credit or sometimes for pay at the convenience of BVSD teachers, primarily during the summertime. Structured classes included a summer Internet seminar series, a technical support series, a Technology in Learning Institute, a writing center support series, and a Learning Partners Program Workshop series that extended into the fall/winter of 1996. The content spanned introductory classes in Macintosh computers for absolute beginners, maintenance and repair workshops, troubleshooting printers, ideas for improving technical support in your school, using Eudora, using Usenet News, using ftp, "Everything you always wanted to know about UNIX but were afraid to ask", designing a curriculum project utilizing the Internet, creating your own Web pages, using PPP with the BVSD modem pool, the World Wide Web, and the like. Unstructured, open labs were offered approximately twice a month at four different schools from 3:15 to 5 PM, during which time teachers could drop in and practice their Internet skills, network with other BVSD teachers using the Internet, and get help from a knowledgeable BVSD teacher. Additionally, if a teacher wished someone to come to his/her home and help install PPP and get it up and running, the project leader compiled a list of students, staff, and community members who would be willing to provide this service for a small, negotiable fee. Since there were few books on the Internet in 1992, ancillary materials in the form of handouts for the training sessions were all designed and produced by the project leader, based on the best on-line documentation at the time. They included information on e-mail and the various Internet tools such as gopher, Archie, Veronica, ftp, telnet, and the like; a sample Internet curriculum unit; a staff contract agreement and application for a BVSD Internet account, together with an Acceptable Use Policy; troubleshooting lists; links to exemplary Web sites, listservs, and other on-line resources; and other instructional materials. As the Internet evolved, these materials were continually updated, based on feedback from participating teachers. Later, the project team designed a notebook that organized all of these print resources and gave a copy to every trainer and every school throughout the district. Although the grant period has ended, the BVIP is continuing at a rapid pace. All but three schools were connected by January 1997. The project is not an isolated effort in communications, but rather, a system wide effort that both impacts and is impacted by the numerous factors in the educational system. Throughout the two years of the current case study, the Internet itself evolved. So did the scope of the BVIP, and along with it, the model that was used to gather and report data from the evaluation. Teachers moved through various phases of growth, starting from exploration and professional development, through communication and generation of on-line learning communities, to eventual use of Internet activities in the classroom. The hopes of the project organizers for integration of telecommunications-based activities into curriculum and instruction have not been fully met due to the inhibiting characteristics of significant educational reform itself, as well as the slow pace of connecting all 53 of the district's schools, paucity of incentives such as release time or extra pay for teachers who have taken on new duties as a result of their Internet expertise, and ongoing cutbacks in FTE and technical support at the building level. Instead of simply being viewed as a teaching tool, the use of the Internet becomes entwined within program improvement, a much more complex undertaking. This study describes the complex system that has evolved and is continuing to evolve. Purpose and Scope of Study According to Patton (1987), evaluation research, particularly at the local program level, has been largely non theoretical. In contrast, the RMC Research Corporation evaluation team took a case study approach that was inductive, pragmatic, and highly concrete. In addition to building a sound theoretical base, the evaluators incorporated specific data that were relevant to the project, focusing on issues that were helpful in monitoring or fine-tuning program operations. The scope of the evaluation began as a global inquiry using a survey and interviews; then shifted to a detailed examination with a case study and focus groups. The role of the independent evaluation team was threefold. They provided: 1. Formative evaluation: ongoing information to the district about what was working, what was not working, and how the current system could be improved; 2. Summative evaluation: feedback via a comprehensive report at the end of the project on the degree of success in meeting its objectives and its impact on users; and 3. An expanded theory base and model: data that were used to gradually build a theoretical model throughout the two-year data collection phase, grounded in observations of program activities, impacts, and the relationship between treatments and outcomes. After the completion of the first year of the project, Wolf and Black (1993) conducted a formative evaluation using a survey and interviews. The data suggested that the initial cohort of 26 peer trainers were successfully trained; they felt that they were more knowledgeable about the use of Internet as a resource in teaching and learning; and they were beginning to apply their skills and knowledge in their classrooms through Internet-based activities and projects. Wolf and Black also identified five barriers that directly impacted teachers' use of the Internet and that have continued to affect the level of Internet usage throughout the duration of the project: (a) access, (b) time, (c) training, (d) resources, and (e) usability. In 1994, RMC Research Corporation was engaged to expand upon that formative evaluation. The evaluation team examined the training component to ascertain its value, both in terms of whether the training accomplished the short-term goals of being clear, useful, and engaging, and to determine whether the teachers were actually implementing in their classrooms the information and skills they gained from the training. The team also investigated the influence of the project on the development of new curriculum and teaching strategies and the benefits of participation for the entire school and district. Toward the end of 1996, they evaluated the BVIP as a whole to determine whether it may be easily replicable or transportable to other districts, and how it may be improved. The Integrated Technology Adoption and Diffusion Model that emerged from the BVIP evaluation is currently being restructured and will be used to both design and evaluate technology implementation plans in other states. Review of Relevant Research and Development of the Model In order to build a sound theoretical base for the case study, the evaluation team reviewed current literature that dealt with barriers to technological innovations and the factors that either enhanced or inhibited the change process by the adopting educational organization. Three approaches emerged. One looks at technology adoption from the perspective of the technology, including access, cost, type and age of available computers and hardware, the physical aspects of the school network, reliability, and user interface or system ease of use. The second considers the viewpoint of the user, encompassing both user characteristics and users' perceptions of the technology. The third focuses on the complex needs of the educational institution itself - a school or district situated within an organizational environment, which, in turn, is situated within the district and local culture as a whole. There was precious little information concerning how teachers were using the Internet to assist them in their teaching, or whether they thought that using the Internet in the classroom was a good idea or not. In fact, most evaluations of the effectiveness of technology focused on the technology itself; they did not examine the effectiveness of telecommunications as a tool for learning. In evaluating the BVIP, the RMC Research team utilized the Engaged Learning model of Jones, Valdez, Nowakowski, and Rasmussen (1995). These authors stated that engaged learners are responsible for their own learning, are energized by learning, actively develop their own repertoire of thinking and learning strategies, and develop new ideas and understanding through conversations and collaborative work with others. Technological Factors Based on Wolf and Black's (1993) initial findings, the evaluation team began by examining recent studies of school districts that were building telecommunications infrastructures to connect their schools. Honey and Henriquez (1993), Heaviside and colleagues, (U.S. Department of Education, 1996), and Levin (1995) described a number of technological barriers to successful adoption and diffusion of Internet-based classroom activities including limited funding for advanced telecommunications hookups, lack of equipment or poor equipment, and too few access points in the school building. Results from a comprehensive national survey (U.S. Department of Education, 1996) indicated that in about half the schools that were currently connected to a wide area network, district and regional administrators played a large role in developing the school's telecommunications program, while only about a third reported that teachers and other staff took the lead. Thus, the BVIP was in a rather unique situation in that the staff development program was funded by the National Science Foundation, whereas the technology was primarily supported by the district and its collaborative partner, the University of Colorado at Boulder. BVIP Project directors worked closely with district teachers and technical support staff at a grassroots level to create and disseminate its telecommunications program. Despite the BVIP training program and technical support that was provided through the grant, many teachers were not able to use the Internet to its fullest potential because they could not gain regular and dependable access to the information superhighway. Wolf and Black identified two important technological barriers to Internet use: access and usability. Data gathered from the current evaluation of the BVIP revealed that the most important technological factors affecting Internet adoption were access to the wide area network from home or school, access to the local area network within the school, presence and capacities of servers, modems, and computers, usability, network response time, and network reliability. Though Internet drops to individual schools were gradually provided by district bond funds, and though the BVIP provided funds for ongoing network upgrades and technical support, full, district-wide connectivity was not possible until January 1997. Moreover, because of the site- based decision making process of the district, the local area network configuration within each building was under the jurisdiction of each individual school. Individual Factors The second approach, namely, looking at the technology from the point of view of the end user, is exemplified in Rogers' (1995) Diffusion of Innovations model. In the case of the BVIP, the users include district administrators, support staff, teachers, and students. Individual factors include both the personal characteristics that these users bring to the technology and their perceptions of the value that it adds to the instructional process. Based on 30 years of research on the diffusion of innovations within organizations, the Rogers model discusses the social process by which people adopt a new idea or technology. An organization (such as a school district) is envisioned as a social system in which people work to achieve a common goal through established practices and processes. The novelty of a technological innovation creates uneasiness and uncertainty among members of the social system that can adversely affect the rate of activities that take place as the new technology is adopted by the organization. This uncertainty, in turn, creates a curiosity/anxiety conflict in new users, that is described in detail by Spielberger and Starr (1994). In the Rogers model, "Uncertainty implies a lack of predictability, of structure, of information. In fact, information is a means of reducing uncertainty." (Rogers, 1995, p. 6). Some people are naturally cautious; others are more curious, more willing to explore and innovate. Spielberger and Starr's (1994) model of epistemic curiosity describes a dual process consisting of anxiety and curiosity. Individuals possess normal, random variations in both of these characteristics, not only as a personality trait, but also in relation to their state of relaxation or agitation at any given time, which, in turn, depends on the context and situation in which they find themselves. The lower the comfort level of the new users of an innovation, the less willing they are to experiment with it. Similarly, Hall and Hord (1987) describe the stages of concern that an individual goes through when adopting a change or innovation: 0. Awareness (little concern about or involvement with the innovation); 1. Informational (interest in learning more details about it); 2. Personal (concerns about its demands and their adequacy in meeting them); 3. Management (processes and tasks of using the innovation); 4. Consequence (impact of the innovation on student outcomes); 5. Collaboration (coordination/cooperation with other users); and 6. Refocusing (altering or replacing the innovation). Each of these stages is characterized by specific questions, concerns, or anxieties about the innovation. In the initial, information-gathering stage, a potential user has some general awareness of the innovation and is interested in learning more about it. However, uncertainty and self-concern enters the picture while the individual focuses on his/her adequacy to meet the changing demands within the existing environment. At the task management stage, the individual focuses attention on the processes and tasks related to the use of the innovation. Later, this focus shifts to reflections about how the innovation impacts students and the community; its consequences for others in the environment; the cooperative use of the innovation in coordination with other teachers and students; and what benefits might arise from altering or replacing the innovation. A critical element of the Rogers model is the concept of classifying adopters and potential adopters on the basis of their innovativeness, in other words, how soon they adopt an innovation. Potential adopters vary in socioeconomic status, personality values, and communication behavior. The five categories of adopters are (a) innovators, who are the first to adopt; (b) early adopters, who are often the opinion leaders in the group; (c) early majority and (d) late majority adopters, who form the bulk of the adopter group; and (e) laggards, who are often the last to adopt. According to Rogers, the "S-shaped adopter distributions closely approach normality" (Rogers, 1995, p. 261.) Though the Integrated Technology Adoption and Diffusion Model drew heavily from the Rogers model, the initial data that was collected revealed that one cannot simply characterize early adopters as "techies" and late adopters as "technophobic". Such simplistic labeling of the adopting population cannot fully describe the complex relationship between the technology and the human element. Identified user characteristics such as motivation, need for control, and comfort level, coupled with expertise, reasons for and patterns of use, gender, and special needs definitely influenced both the content and the process of acquiring expertise in telecommunications. Knowledge, comfort level, and usage levels were closely related, as those with greater experience took on leadership roles. In a recent study targeting users' perceptions, Wilson and colleagues (Wilson, Ryder, McCahan, & Sherry, 1996) reported that some individuals who resisted using a new schoolwide Internet system felt intimidated by the experts within the school whom they labeled as "techno-gurus". They felt that these communications experts were speaking a new, different, technological language, filled with mystical expertise, jargon, and commands that were simply not worth learning. Rossett (1991) suggests that, when dealing with "new stuff", especially new technologies that are being implemented, the feelings of the users become the critical issue. In contrast to treating the situation as a performance problem in which there is a gap between optimal and actual performance, when potential users are adopting a new technology, there is a gap between optimals and feelings. This is an important distinction. Rogers himself states that "[p]ersonality variables associated with innovativeness have not yet received full research attention, in part because of difficulties in measuring personality dimensions in field interviews" (Rogers, 1995, p. 272). Based in part on the Rogers model, Farquhar and Surry (1994) developed an Adoption Analysis Tool, which takes both the characteristics and the perceptions of the users into account, since their levels of concern and uncertainty regarding the innovation are echoed in their perceptions of the new technology. According to Farquhar and Surry, user characteristics affecting adoption are (a) motivation, (b) anxiety, (c) knowledge base, (d) prior experience, and (e) skill level. As mentioned above, these characteristics, among others, were evident in the initial survey and interview responses from the BVIP evaluation. The user perceptions enumerated by Farquhar and Surry are the same as those of Rogers, namely, (a) relative advantage of the innovation over the existing system; (b) observability of the innovation's consequences; (c) compatibility with users' values and needs; (d) complexity vs. simplicity; and (e) trialability - in other words, can the innovation be tried out on a limited basis? Their Adoption Analysis model states that the more positively new users perceive an innovation with regard to these five characteristics, the greater the likelihood that the innovation will be adopted. Wolf and Black's (1993) initial evaluation also described similar user perceptions, as did the data collected from the present case study. Some teachers reported that the Internet can be very difficult to navigate, and that many of its avenues are open only to those who are quite knowledgeable about the system. Hence, even if the network is fully implemented and is functioning at full capacity, simply gaining access will not solve all of the problems that teachers may encounter as they try to find useful resources and integrate them into their classrooms. Organizational Factors Besides individual factors, the organization as a whole must be considered. In the third approach, organizational factors are concerned with the complex needs of an institution. These deal with the overlap between what the district brings to the diffusion process and the impact of the innovation. In Farquhar and Surry's (1994) model, organizational factors involve both the physical environment and the support environment in which Internet-based classroom activities are to be used. An example of the difference between an organizational factor concerning the physical environment and a technological factor concerning availability of technology is the use of a library modem. The modem may be working and in good condition (a technological factor), but if it is constantly used by the librarian and is not open for use by any of the teachers, that is an organizational factor. The physical environment includes the availability of computers and Internet connections in classrooms or other contexts in which learning will take place, availability of other important resources, and existing organizational arrangements such as scheduling practices. Key physical environment factors identified in the BVIP evaluation included classroom connectivity, network capacity, scheduling of laboratories and computers, availability of equipment, and availability and proper use of supplies such as diskettes, toner cartridges, and paper that were necessary for storing and printing out student-generated products. The support environment, in Farquhar and Surry's Adoption Analysis model, includes (a) production, (b) storage, (c) delivery, (d) dissemination, and (e) support services. These factors were not entirely appropriate for the BVIP, in which the organizational support environment includes district policies such as planning time and grading policies, incentives for staff development, administrative and technical support, problem-solving mechanisms, training, maintenance, and information resources. Of the five key elements identified by Wolf and Black that affected BVSD's use of the Internet, three were organizational factors - time, training, and resources. Throughout the current evaluation, however, it became evident that these three factors were multi-faceted, and that several related issues were being confounded in an attempt to simplify and limit the number of organizational factors that impacted the BVIP. Specific support environment factors identified in the BVIP case study were administrative vision and support, district policies such as site-based decision making, planning time, and grading policies, acceptable use policies and handling of student accounts, availability of communication channels, mechanisms for identifying and solving problems, availability and timeliness of training workshops, availability of resource and technical support personnel, availability of documentation, troubleshooting lists, and job aids, availability of on-line support, maintenance of the network and associated equipment, lack of release time to pursue training, lack of incentives for teachers who take on new duties as a result of their Internet expertise, lack of a shared knowledge base of curriculum implementation resources, non-district and global support via the Internet, and support from and cooperation with other grants. Beyond providing support through making the physical technology available, a district as an organization needs to be sure that the new effort is aligned with the district-wide vision and policies, as well as integrated into the established communication system. As problems arise, the administration needs to have mechanisms in place that can address new challenges and improve upon the responses until they are seamless and integrated into the whole. Hoffman (1996) identified eight success factors for teachers who are trying to adopt technology in their classrooms and to build the support they need for using technology: (a) administrative support; (b) staff development and technical support; (c) availability of technology; (d) a technology use plan; (e) a district computer or technology coordinator; (f) facilities and maintenance personnel who are included in planning for technology integration; (g) assessment and evaluation methods which reflect new educational approaches; and (h) broad participation by individual teachers, groups and committees of teachers, the school-site computer coordinator, the principal and other administrators, and district-level coordinators or administrators, superintendents, and school boards. In contrast to the physical/support environment factors that were identified by Farquhar and Surry, and Hoffman; Gross, Giacquinta, and Bernstein (1970) focused on organizational factors from a management point of view. Their factors included the following: (a) a clear vision of the innovation provided by administrators to all teachers, stakeholders, and users; (b) the staff's skills and capacity to implement the innovation; (c) the availability of required tools and resources; (d) the compatibility of the school's organizational arrangements (such as the grading system and scheduling of classes) with the new innovation; (e) taking account the difficulties to which teachers may be exposed as they begin to implement the innovation; (f) providing mechanisms to identify and cope with unanticipated problems that may emerge during the period of implementation; and (g) the wholehearted support of the administration for the innovation and for the teachers who are implementing it. Recent researchers have begun to integrate the technological, individual, and organizational factors in a more systemic fashion. Stockdill and Morehouse (1992) identified five critical factors that affect the successful adoption of new technology: (a) educational need, (b) user characteristics, (c) content characteristics, (d) technology considerations, and (e) organizational capacity. Farquhar and Surry (1994) also built on Stockdill and Morehouse's model in developing their Adoption Analysis tool. Successful implementation requires not only that adopters buy in to the use and application of the innovation, but also that the adopting organization provide a worthy environment in which to use the new technology and all of the resources and services needed to install and maintain it. Studies of failed innovations such as those of Gross and his colleagues (1970) and Teasley (1996) often provided insights into the organizational factors that affected the diffusion process. These researchers reported that, though the initial attitude of the school administrators and teachers was positive and that they were willing to expend the time and effort to implement the new technology, they ran into difficulties along the way. Teachers began to express frustration with the project, and administrators failed to give it their full support. Moreover, the administrators had no mechanisms in place for dealing with unforeseen problems that might arise during the implementation process. Communications broke down and teachers continued to run into problems with which they simply could not cope. Like any other complex system, school districts depend on good internal communications if they are to work effectively. A complex system, however, often embodies information that is hard to get (Ruelle, 1991). In other words, communication problems may arise between administrators and teachers or between teachers themselves. Carlson (1970) suggested that when the key change agents are dispersed (in his case, the district superintendents), then those who have clear communication channels with each other tend to work closely together. Those who do not have good communication links with others tend to remain isolated from the change process. Peled et al. (1994) and Egan (1985) viewed a school district not simply as an educational organization, but as a cultural entity with a common cultural blueprint that sets the pattern for the structures and processes that occur within and across the systems (i.e., the classroom, the school, the community, and the regional educational policy-making institutions). Morison (1984) suggested that opposition to innovations is cultural in nature and stems from the normal instinct to protect one's self and one's way of life. People will withhold judgment or even express disbelief about the dramatic claims of the new innovation. They will try to protect the existing system with which they identify themselves as well as the existing society of which they consider themselves a part. Lewis and Romiszowski (1996) studied learning organizations - educational institutions which constantly update the skills of their members and in the process, are themselves transformed. They used a five-element model that is similar in structure to the BVIP technology adoption model presented here, consisting of (a) technology-related, (b) organizational, (c) personal, (d) pedagogic, and (e) cost and resource-related success factors. They found that, from an organizational learning perspective, there is a considerable organizational learning curve to be traversed before organizations not only assimilate a new technology into their culture, but also accommodate their culture to be able to profit from the technology to the extent that is theoretically possible. They suggest that familiarity with the new technology and the ability to apply it in one context do not necessarily guarantee either the ability or the motivation to use it successfully in novel contexts or to integrate it in a major way into a new teaching program. When networking begins to be used seriously, the nature of the teaching and learning environment is inevitably changed. They recommend that further research efforts are needed to investigate the optimum mix of networking and other learning activities to enhance the learning environment for students and teachers, and to explore the decision making factors that influence the level of support that is given to new instructional innovations. The interaction of learning and adoption that took place in the BVIP will be reported in a separate paper (Sherry, 1997). Though a complex system tends to maintain a stable state, it may begin to change itself, "provided it begins by defining for itself its grand object, and see[ing] to it that [the] grand object is communicated to every member of the group" (Morison, 1984, p. 142). Peters (1988) remarked, What matters is that everyone who works for and with you observe you embracing the topic with both arms and your calendar. What they need to observe is your obvious, visible and dramatic, determination to batter down all barriers to understanding and implementation. (p. 501) In summary, the adoption and diffusion process can viewed in three ways: as a social process (the Rogers model), as a cultural process (Peled and Egan), and as a systems process (Ruelle and Morison). These overlap, since systems are cultures, and cultures have social processes. Within the system there are barriers to diffusion. Some of these are sociocultural, such as the individual characteristics of the end users; others are involved with maintaining the stability of the system as it presently exists. Moreover, the system itself imposes constraints on the adoption and diffusion process. To be successful, a new technological innovation requires a worthy environment that can support it; resources to install, implement, and maintain it; readily available, two-way communications channels; and an overall administrative vision that is clearly communicated to all members and key stakeholders. Teaching and Learning Issues Teaching and learning issues center around the impact of technologies and technology-enhanced educational programs on students engaged learning. As Lewis and Romiszowski (1996) state, an educational system must be studied as a learning organization in which all members are actively involved in both planning and participating in learning programs adapted to the specific requirements of the changing work or social environments in which they find themselves. The use of new technologies, especially telecommunications, enables teachers and students alike to focus on developing meta-skills that can be transferred to real-world situations, rather than simply concentrating on developing specific areas of content knowledge or subskills. In the Center for Applied Technology's (1996) national study, a comparison of the work of 500 students in fourth-grade and sixth-grade classes in seven urban school districts showed that students with on-line access achieved significantly higher scores on measurements of information management, communication, and presentation of ideas. Student learning was measured by evaluating students' final products based on nine learning measures in which students were required to demonstrate their knowledge, apply their skills, and illustrate their thinking processes. The authors state that "using the Internet can help students become independent, critical thinkers, able to find information, organize and evaluate it, and then effectively express their new knowledge and ideas in compelling ways" (p.1). This leads to a host of new instructional methods that are just beginning to be tried out in the classroom. One such method is a variation of the Socratic Method, described by Wason (1996). Instead of lecture-based instruction followed by testing, students can learn by a system of carefully developed questions that guide them through the learning process, rather than using questions as vehicles to test the learning product. This model of teaching combines an effective system of resource access with content-relevant navigation and landmarks; sequences of questions used as the underlying instructional tool that enable students to discover knowledge relationships and acquire learning skills; and a technology integrated model of teaching whereby teachers select questions matched to the needs of the specific students, who then work asynchronously, either alone or in self-selected groups. As a result, the educational process becomes student centered and the learning process takes precedent over the learning product. Such innovative models are fraught with difficulties, especially in a district like the BVSD, where the current emphasis is now shifting toward instruction based on model content standards and where the type of assessment used by the district is for the most part still norm-referenced. Though an instructional model similar to Wason's has been used successfully in multi-age, multidisciplinary classrooms such as the science class at Nederland Elementary School, it is not necessarily applicable for typical classes that were observed in the BVIP case study. Initially, the Engaged Learning model of Jones and his colleagues (1995) was incorporated into the Integrated Technology Adoption and Diffusion Model. Engaged learning issues comprise the various learning styles and roles of students in the classroom, authentic and relevant tasks, multidisciplinary curriculum, interactive and generative activities, and a learning context that emphasizes collaborative knowledge building. Jones and his colleagues identified eight variables that are related to a set of indicators of engaged learning: (a) the teacher's vision of learning; (b) relevant, challenging tasks; (c) ongoing, authentic, performance-based assessment; (d) a constructivist instructional model responsive to student needs; (e) the concept of students as part of a learning community incorporating multiple perspectives; (f) collaborative learning; (g) the role of the teacher as a facilitator, guide, and co/learner/co-investigator; and (h) the roles of students as cognitive apprentices, peer mentors, and producers of products that are of real use to themselves and others. In Boulder Valley, the teacher's vision of learning is closely related to his/her role in the classroom and his/her perception of the relationship of the classroom curriculum to state and district standards, whether the existing curriculum is to be enriched, enhanced, or replaced, and the precise role of Internet-based instructional activities in the classroom. The teacher's role also involves using students as assistants, collaboration in sharing ideas, resources and lessons with colleagues, sifting through and organizing relevant classroom activities from the massive volume of information available on the Internet, generating meaningful lesson plans, ideas, activities, and resources for classroom use, and tailoring existing Internet resources to meet the needs of the classroom. In the Integrated Technology Adoption and Diffusion Model, teaching and learning issues were divided into curriculum and engaged learning factors. Curriculum factors comprised changes and enhancement in content, the volume of content to be explored, planning and preparation, the relationship of lessons and standards, the use and sharing of lesson plans, and the evaluation and categorization of Internet activities. The engaged learning factors were essentially those of Jones and his colleagues, as mentioned above. Though the initial plan of the BVIP included both the development and delivery of comprehensive district-wide Internet training for all interested teachers and the establishment of a sharable resource of curriculum-related resources and activities, only the former has been fully implemented. The first steps toward establishing a resource access with content relevant navigation and landmarks as described in the Wason model were undertaken by the BVIP, in cooperation with the Annenberg/NSF Creating Connections project, when they created the Science and Math Initiatives (SAMI) database. They also assisted district teachers who built several school home pages which were linked to student activities and resources that could be shared by teachers. As part of the current evaluation, a work group of BVSD teachers also created a classification scheme for Internet-related resources, titled CLUE, which will be described later. This product will assist them in forming the foundation for curriculum-related ideas, activities, lesson plans, and resources that can be accessed via the Internet. The Integrated Technology Adoption and Diffusion Model After compiling and reviewing the research on the four strands of variables that impact the adoption and diffusion of a new educational innovation by an organization, and after collecting the initial data via surveys and interviews, RMC Research created a model that targeted the most important variables and applied it to the BVIP. As the data collection proceeded, and as the information that was gathered was coded and sorted, the model was expanded with additional information. Table 1 presents a summary of all of the factors that influenced the adoption and diffusion of the BVIP throughout the Boulder Valley School District.
Integrated Technology Adoption and Diffusion Model
Access to WAN from home or schol
Access to LAN within school
Available hardware and software
Network response time, search time
Reasons for use
Need for control
Patterns of use
Complexity (vs. ease of use)
Compatibility with needs and wants
Scheduling of labs and computers
Availability of equipment
Availability and proper use of supplies
Administrative vision and support
District policies (decisionmaking, grading, scheduling, etc.)
Acceptable use and student accounts
Mechanisms for solving problems
Availability of resource people
Availability of documentation, lists, and job aids
Availability of online support
Incentives for staff development
Non-district and global support
Support from other grants
Teaching and Learning Issues
Change in content
Volume of content
Planning and preparation
Relationship of lessons and standards
Use and sharing of lesson plans
Evaluation/categorization of Internet activities
Vision of learning
Teacher roles: facilitator vs. didactic
Collaboration in sharing ideas and resources
Student roles: explorers, experts, mentors
Interactive, generative learning
Authentic, relevant tasks
New types of assessment
Study Methods and Data Sources
Because the purpose of the study was to build an expanded theory base and
an Integrated Technology Adoption and Diffusion Model as well as to
provide formative and summative evaluation to the funding agency, the
evaluation was conducted as a case study. According to Yin (1994), case
studies are generalizable to theoretical propositions, and not to
populations or universes. The applicability of the BVIP model to other
school districts will depend in large part upon the administrative vision
and support and the structure of the decision making process of each
individual district, whether site-based or district-wide.
The Boulder Valley School District (BVSD) is a large, dispersed, partially
mountainous district comprising 53 elementary, middle, and high schools,
situated to the northwest of Denver. The City of Boulder is home to the
University of Colorado.
An Individual School (an embedded case study)
Nederland Elementary is an isolated, rural K-5 school that is situated in
a small mountain town within the BVSD. It is a high-end user school that
made good use of the Internet to expand the learning environment beyond
the walls of the classrooms and beyond the small community of which it is
a part. For several years, it was also one of the few schools in the
district with schoolwide connectivity.
After carefully reviewing the relevant research literature and discussing
the project with the BVIP project director, five central research
questions were formulated that underlie the investigations of this case
1. How effective was the training component of the project?
2. How did the project specifically affect the participants use of the
3. What was the impact of the project on curriculum and instruction?
4. What was the impact of the project on the schools as a whole? The
district as a whole?
5. What are the possible future uses of this model?
Because the BVSD was a complex system, multiple measures were used in
order to produce converging lines of inquiry. This approach involves both
qualitative and quantitative methods, as seen in Table 2. The five
research questions were examined and matched with the most appropriate
data collection instruments. The data collection instruments included (a)
an e-mail survey, (b) indepth interviews, (c) a curriculum focus group,
(d) an Internet activity classification work group, (e) an embedded case
study, and (f) documentation analysis.
Initial information on the effectiveness of the training component was
gathered from a district-wide e-mail survey of the 435 teachers who were
identified as regular Internet users via the system logs. The quantitative
data were supplemented and enriched by in-depth interviews with the BVIP
project leaders and eight teachers who were using the Internet in their
classrooms. More detailed data were obtained through the curriculum focus
group, the Internet activity classification work group, and the embedded
case study of Nederland Elementary School. The documentation analysis
involved examining the BVSD system logs, previous project documents and
evaluations, newspaper articles, white papers, and other artifacts.
Data Collection Strategy Matrix
BVIP System Logs
How effective was the training component of the project?
How did the project specifically affect the participants' use
of the technology?
What was the impact of the project on curriculum and instruction?
What was the impact of the project on the schools as a whole?
the district as a whole?
What are the possible future uses of this model?
Methods of Data Collection and Analysis
A 42-item survey was e-mailed to the 435 district teachers who had e-mail
addresses. A follow-up reminder was sent out one month later. 142
responses were received, a response rate of about 33%. Responses to
forced-choice questions were reported as percentages or in chart form.
Responses to open-ended questions were reported verbatim and were
clustered into naturally emerging categories, which were then checked by a
second researcher to enhance reliability.
Indepth Interviews with Project Personnel and Teachers
Indepth interviews lasting approximately two hours were conducted with the
project leader, the project administrative assistant, and eight teachers
who were active users of technology and had used some Internet-based
activities in the classroom. Tapes were transcribed, shown to
participants for their approval, and then coded.
Ten educators from various schools throughout the district were convened
in a focus group that addressed strategies, benefits, problems, and
content of instruction that incorporates use of the Internet. All focus
group participants had worked with Internet-based curriculum development.
Internet Activity Classification Work Group
The work group took place after the focus group. It comprised a group of
teachers who intended to create an assessment product that could be used
for evaluating curriculum resources that utilized the Internet. All work
group members were experienced users of the Internet in the classroom.
What eventually evolved from the group's efforts was a classification
system that would aid the district in organizing resources, lesson plans,
thematic information, and other helpful resources in a centralized
location. The resulting classification scheme, CLUE, is presented in Table
The BVIP Classification of Teaching and Learning Units (CLUE)
that use the Internet Effectively
Content Area: (circle one)
Art Language Arts Math PE Science Social Studies
Integrated (more than one area) Other: __________________
Specific Topic: _______________________________________
Grade Level: (circle one) K-3 4-5 6-8 9-12
Duration: (circle closest time frame)
1 period more than 1 day weeks 1 or more semesters
Degree of Collaboration: (circle one)
within the classroom within the district outside the district
Internet Connectivity: __________________________________
Date of Entering on Clue: __/__/__
BVIP system log information that covered the period from August 1995
through January 1996 was made available for all schools in the district.
The numbers of log-ins were tabulated for 1995 and 1996.
Embedded Case Study
As part of the BVIP evaluation, RMC Research carried out an embedded case
study at Nederland Elementary School (See Yin, 1994). This enabled the
evaluators to gain some insight into the school's organizational change
process, and to cross-check the self-reported data that were gathered in
the e-mail survey and the indepth interviews. The site was purposefully
selected because of several important factors, namely:
1. The trust established between the evaluators and the school staff;
2. The high participation of the school staff in the district's peer
3. The high degree of connectivity in the school; and
4. The high percentage of educators in the school who were regular
Internet users as demonstrated by the BVIP system logs.
The Nederland Elementary School case study used three data collection
instruments: (a) classroom observations and focus groups with students
from a fifth grade class at nederland Elementary School and a multi-age
science class at Nederland Middle/high School who were currently using
Internet-based activities; (b) a brief survey of teachers,
paraprofessionals, school administrators and support staff; and (c) short,
focused interviews of selected teachers, the principal, and BVIP project
leaders. In the embedded case study, RMC Research concentrated on
organizational and individual factors that were not addressed by either
the e-mail survey or the indepth interviews. Attention was also devoted to
the degree of systemic change.
The Integrated Technology Adoption and Diffusion Model proved effective
for collecting, organizing, and reporting the findings from all phases of
the evaluation. The information that was gathered was then used to answer
the five research questions.
1. How effective was the training component?
Overall, the BVIP was successful in meeting the goals of its training
component and raising the level of expertise in the use of the Internet
throughout the district. The survey data revealed that the BVIP classes
and workshops were the primary resource for most district teachers (94 out
of 142 respondents) who were learning how to use the Internet. In addition
to the formal training that they received, some participants requested
additional workshops in site-based small groups to meet the needs of
particular schools and programs. They also requested more emphasis on
authentic uses of the Internet as they can be applied to either an
existing or an enriched curriculum. Success facilitators included the
Trainer of Trainers approach, the training program's focus on classroom
teachers rather than a top-down approach, and the flexibility of the
training program in keeping up with the rapid changes in the technology.
Problems can happen in a lot of different places, so we are running
technical workshops for every school in the district. When they come to
the workshop, they get six hours of consulting time to help them figure
out what they are doing. We give them a troubleshooting list so they can
determine if the problem is in the building, the district, or the
Internet, so you can call the appropriate person. (A Trainer)
Barriers to success included insufficient staff development time, delays
between training and access at the school level, and cuts in district
funding in a variety of areas. These barriers were magnified when the
original, more progressive superintendent and school board were replaced
with new members who espoused a more conservative educational philosophy,
at about the same time as both the BVIP and Annenberg funding ended.
Since the training was aimed at typical classroom teachers rather than
administrators and policy makers, expertise was diffused horizontally at
the grassroots level, rather than percolating upwards toward all levels of
the educational system. The BVIP project director commented on the fact
that, in a site-based model, it is generally the administrators who are
A lot of principals don't use a lot of technology...so it is not really
clear to them how their kids are using it or what a difference it is
making. This has to do with the district having to take a stand on
believing that it is necessary for administrators to understand this and
to see what it can really do in the curriculum. If they are going to do
that, then everybody has to have easy access, and you have to train
people. (BVIP Project Director)
2. How did the project specifically affect the participants' use of the
The most significant influence on the participants' use of technology was
the increase in communication with colleagues, experts, and friends within
the building, within the district, or outside the district. Using e-mail
for communication and accessing the WWW for both teacher- and
student-initiated research became the most popular uses of the Internet.
Teachers and students became proficient in Internet use and accessed many
educational resources that supported their growth. One focus group
The main thing with the project is when it was first started, where the
technology was then, and where it is now. If I had to start getting back
with all those [individual tools], ftp, and so on...I mean, I love the
Web! That is what I live on. (A Teacher)
Barriers included an increased work load for some new trainees, lack of
observable benefits for the additional training involved, declining morale
because of district funding cuts, and the reassignment of in-building
technology resource personnel to the classroom.
Though the training program was successful in increasing the number of
Internet users in schools throughout the district, and though there were
examples of highly innovative student-generated projects, there was
evidence of a negative feedback loop that tended to slow the adoption
Bond money to purchase computers and connections was held up for over a
year due to a lawsuit. This led to a gap between training and curriculum
integration. The volume of content had increased dramatically, yet there
was still no common, district wide base of curriculum-related resources
for teachers to tap. To add to the turmoil, many of the in-building
technology specialists were reassigned to the classroom due to continued
funding cuts within the district and the phasing out of grant funding.
The kids come down and do a search on Netscape all the time. Teachers who
are trying to integrate technology need a good starting place. I'm
everybody's tech support. I teach classes for the kids, for the teachers,
do maintenance, and all the planning. (A Teacher)
Often, teachers questioned whether or not their training was really worth
all the effort, especially if they took on new duties and responsibilities
as a result of their newfound expertise. Though the core group of
teachers were given a choice of pay or credit for their participation in
the training program, that training was not integrated into building
professional development programs, nor was it seen as a must-have by
principals at the time. In other districts that are contemplating such a
training program but that may not have incentive programs in place,
expecting teachers to be trained on their own time means that only those
who are truly devoted and already have an interest will pursue training
3. What was the impact of the project on curriculum and instruction?
The impact of the project on teaching and learning was strongest when the
classrooms had direct access to the Internet and when the teachers'
instructional style matched their uses of the Internet. A "facilitator"
What works for kids is more important than somebody telling you what to
do. Why would I sit there and teach types of clouds when I know that, in
science, the process is sometimes more important than being able to show
them multiple choice. (A Teacher)
The project leader expanded on this issue:
One way in which the Internet enters the classroom scene at an opportune
time is that in many places the curriculu, especially science, is being
shittled down so that fewer topics are taught, but more in depth and more
for understanding. The Internet can really support this in-depth teaching
approach due to the vast amount of resources and options for collaborative
Teachers' need for control influenced the ways in which they used the
Internet for instructional purposes. Loss of control was due in part to
the open framework of the Internet, and in part to the rapid gain in
expertise by the students, which sometimes resulted in a novice/expert
role reversal between students and teachers.
Age had little effect on expertise, but gender affected both the amount
and type of use, with boys using the Internet more for exploring and
creating Web pages and girls using it more for communication. This is
consonant with Turkle and Papert's (1991) findings that girls tend to use
a negotiational approach with regard to computers, whereas boys prefer
activities that are characterized by abstract thinking and systematic
planning. Both boys and girls, however, used the WWW regularly for
research projects. Some teachers were concerned about what subject content
might be replaced by Internet-based activities.
The weather unit was a large expansion. It was so big that it replaced a
lot of things. It's OK at my school, but it meant skipping other topics.
If you are spending eight weeks doing weather, what are you not teaching?
Teachers who regularly used the Internet for class-related purposes had a
variety of strategies that they used to plan and prepare for their
classes, including having their students assist them in finding
curriculum-related resources on the WWW, participating in mailing lists
with colleagues and experts who shared their interests, and sharing
information and activities with other teachers and associates throughout
the district. As a result, the focus of the curriculum shifted from
developing technology skills to enriching instructional content. One focus
group participant commented,
There is so much stuff out there, just keeping up with it [presents a
problem]. I haven't done that much in the last couple of years, but I did
in the beginning. It was more personal to find [information on] the Web
out there, but now I turn it over to the kids, and the students have a
base that is available. (A Teacher)
Managing the volume of content and decreasing the complexity of searching
for relevant information could be made possible by developing local
resources. Teachers who participated in the summer work group designed a
classification scheme for organizing Internet resources and relating their
content to the curriculum. This classification could be used at the
district level to develop a foundation for curriculum applications. A
focus group participant discussed the issue of shared, locally created,
lesson plans and tests:
Regarding whether or not we talk about putting up lesson plans, I know
that the math group talked about whether we could try putting tests for a
particular course at one particular page on the BVSD Web site...different
schools would post tests that they use for a particular chapter so that
all the math teachers could see what other schools are doing and what
their expectations are for that particular course. That request came up in
the last curriculum council meeting. (A Teacher)
4. What was the impact on the school as a whole? the district as a whole?
Regarding the impact of the project on a whole school, the embedded case
study of a rural mountain elementary school revealed a wide range of
expertise among the staff. Some teachers had a high level of expertise,
whereas the support staff hardly used the network at all. Students and
teachers with home access used the Internet more at school than those who
lacked home access.
A sharp division existed between the use of the Internet as a special
class in a computer laboratory and as part of regular classroom
activities. Instructional use of the Internet was hampered because the
computer laboratory was often considered a place for students to gain
technology skills during teachers' planning periods.
There's a split in our school [between] people wanting it in the computer
lab and people wanting it in their room. I think that has been a divisive
focus because we have a group that really wants those kids to go to the
computer lab. Those teachers don't want anything to do with it. They just
want that instructor. But I see technology as more of a tool in education,
something that's woven throughout everything that we do. When it's
isolated in the computer lab, that's great for the kids, but the teachers
can't do anything with them, so I think there's a real breakdown here. (A
This issue was discussed in the local newspaper (Boulder Camera, May 19,
1996). Cuts in classroom budgets to pay for teachers' raises in 1996 and
1997 meant that fewer schools would be able to allocate funds to pay for
technology specialists. Some educators were concerned that they couldn't
continue making gains in expertise at the current rate, and that they
would not be able to deliver technology services without the specialists.
Others, however, indicated that the district's intent was to have
computers into the classroom anyhow. One school board member was
concerned that half the schools have technology integrated into the
classroom, while the other half subscribe to the "specialist" model.
I don't mean to penalize schools, but if some schools are doing it
differently than other schools, it would seem not a wonderful idea to
provide money to schools that have chosen something that is not in line
with the technology curriculum. (A School Board Member).
The district's technology director continued to work to help schools find
a short-term solution to the long term problem of getting all tachers
trained and comfortable using computers in the classroom, rather than
letting the equipment sit around unused. But such a short-term solution
is not found so easily. One of the peer trainers said:
You have to know the curriculum and the tie-ins with the curriculum. To
find that hook for the teacher has been different for every person.
Because of the efforts of the peer trainers, diffusion of
telecommunications throughout the district was successful at the
grassroots level, but over the course of the evaluation it was hampered by
limited connectivity. Two diffusion models were originally proposed but
were not implemented. The vertical model was not used because the middle
schools had Internet connections from the beginning of the project, but
the elementary and high schools did not get their Internet connections
until much later. All of the high schools were connected by January 1997.
The horizontal model was not pursued because, at the time, it was decided
that this innovation entailed changes in curriculum. With the
incorporation of standards and the ensuing curriculum revisions that were
taking place, pursuing these innovative approaches was simply not
5. What are the possible future uses of this model?
To answer this question we must focus on the actual successes of the BVIP.
1. It was a grassroots effort that successfully diffused horizontally
among those classroom teachers who participated in the training sessions.
2. The Trainer of Trainers model successfully capitalized on the empathy
between change agents and clients.
3. The project had initial buy-in by the superintendent and the school
board. The teachers, however, have felt less support from the new school
board, and perhaps even opposition to them being innovative in the
classroom, despite the board's continuance to pay the salaries of the
project leader and the webmaster.
4. The project worked successfully with other grants and sources of
support such as the Annenberg/CPB Math and Science Project, the district
bond funding, and the University of Colorado.
Future plans for the BVIP include access at all schools and greater
attention to curriculum content and instructional strategies. Other
districts that are considering replicating the BVIP efforts should
consider their own style of management and decision making processes when
designing approaches to the adoption and diffusion of technology.
Though the Rogers model may apply well to centralized organizations, it
fails to explain the intricacies of the learning and adoption process that
take place when an evolving technological innovation is introduced into a
district characterized by site-based management.
Conclusions and Recommendations for Further Research
The BVIP proved to be a dynamic, evolving program housed within a
decentralized educational organization. The reality of the BVIP was quite
different from Rogers' Diffusion of Innovations model. The Internet and
the user interface evolved dramatically over the five-year course of the
project, becoming far more user-friendly, far less complex, and far more
accepted by the general populace. Participatory design based on feedback
from typical users resulted in continuous improvement of the training
workshops. Thus, later adopters had a much easier learning process than
the innovators. As a result, it would be extremely difficult to
distinguish the impact of the BVIP per se from the impact of the Internet
upon the culture as a whole.
As students gained expertise, there were some reversals of novice/expert
roles between teachers and innovative students. Teachers who saw
themselves as facilitators capitalized on this process and utilized the
more knowledgeable students as a support network. Innovative teachers
continued to facilitate engaged learning among students, resulting in
exemplary student products and performances. In contrast, traditional
teachers perceived classroom use of the Internet as a threat that lessened
their control over the teaching and learning process.
During the first two years of the project, growth accelerated among the
initial core of 26 peer trainers and the more innovative teachers. Later
on, the lack of extrinsic incentives, district-wide cuts in funding and
planning time, and the increased work load for new trainees acted as a
reinforcing feedback process. Although there was initial buy-in by a
visionary school board and superintendent, that support faded as policy
makers were replaced by more conservative members. One teacher commented,
"The school board has to learn how important ongoing staff development is
in all areas of instruction and learning!"
Site-based management inhibited horizontal sharing of ideas and resources
among connected schools, even though the patterns of Internet use
increased among participating teachers. Decentralization also affected the
use of shared resources and centralized sources of technical support:
We have not used the district resource person as much as we could have,
and that's because we had a technology resource person right in the
building. We have a troubleshooter. He can fix machines, he can do
anything. I see the teachers pull back and not use the district resources
because we had people right there for us. (A Teacher)
Vertical communication was hampered because teachers did not migrate to
the Internet since they did not have access to the system that was used by
the administrators. Moreover, the middle schools were connected long
before the elementary or high schools. One teacher commented, "Students
come to my high school from middle schools with good connections and then
are disappointed when they get here and have poor access."
The Trainer of Trainers model was successful in increasing the expertise
of classroom teachers, but in its initial stages it ignored the rest of
the ecological system that included the school administrators, parents and
the community, the school board, and the new superintendent. Inservice
training for administrators and parent information sessions were only
offered when they began to be requested by schools and administrators - a
reactive, rather than a proactive, stance. Moreover, since training
occurred prior to full connectivity, a gap resulted between learning how
to use the system and effective classroom use.
This is a trend that has plagued other large-scale educational
telecommunications projects. It is especially evident when the major
policy makers and the teachers have different perceptions of
telecommunications and its appropriate uses in the classroom, when
equipment is purchased before a comprehensive, district-wide technology
plan is in place, when teacher-trainees are not given appropriate time and
incentives to pursue the training that is offered, when in-building
support begins to become scarce, and when the district's projected budget
and timeline run into snags several years into the implementation phase of
If there is a lesson to be learned from the BVIP, it is this - information
technology interventions cannot be separated from their ecological
contexts, nor from the educational activities that they enhance (Peled et
al., 1994). Interventions must aim not at individual participants, but at
the entire ecological environment - the classroom, the school, the
district, and the community in which the educational institutions are
housed. Since the use of the Internet becomes an integral part of program
improvement, it is important for those who control resources and carry out
district policies to be involved in both supporting and participating in
the training activities, so that they can create, sustain, and communicate
a clear administrative vision throughout the entire educational system. In
schools where principals and technology resource people provided vision
and support for technology, the use of technology flourished.
Advertising student successes via student-produced home pages and
high-quality student-generated products can be quite instrumental in
promoting the use of the Internet within those schools that are already
connected. It is also important to have policies in place that allow
students to use the Internet during their free periods, and before or
after school, especially if they do not have access at home.
Although teachers are aware of the many possibilities that the Internet
offers for enhancing teaching and learning in the classroom, many need
further guidance in strategies for integrating telecommunications into
their curriculum and instruction. Future research efforts at RMC
Research Corporation will focus on three important areas:
1. How can or will teachers use technology to attain their instructional
2. How can or will technology fit in with teachers' current teaching
3. What value can technology add to the teaching and learning process?
Boulder Valley Internet Project. (1993). An Overview of the Boulder Valley
Internet Project. (Available from Elizabeth Black, BVSD, Box 9011,
Boulder, CO 80301.)
Carlson, R.O. (1970). The adoption of educational innovations. In M.W.
Miles, & W.W. Charters, Jr. (Eds.), Learning in social settings. Boston,
MA: Allyn & Bacon.
Center for Applied Special Technology (CAST). (1996, November). The role
of online communications in schools: A national study. [On-line.]
Egan, G. (1985). Change agent skills in helping and human service
settings. Monterey, CA: Brooks/Cole Publishing Company.
Farquhar, J.D., & Surry, D.W. (1994). Adoption analysis: An additional
tool for instructional developers. Education and Training Technology
International, 31 (1), 19-25.
Gray, S. (1997). Training teachers, faculty members, and staff. In B. Khan
(Ed.), Web-Based Instruction, pp. 329-332. Englewood Cliffs, NJ:
Educational Technology Publications, Inc.
Gross, N., Giacquinta, J. B., & Bernstein, M. (1970). Failure to implement
a major organizational innovation. In M.W. Miles, & W.W. Charters, Jr.
(Eds.), Learning in social settings. Boston, MA: Allyn & Bacon.
Hall, G.E., & Hord, S.M. (1987). Change in Schools: Facilitating the
Process. Albany, NY: State University of New York.
Hoffman, R. (1996). School technology integration: An automated needs
assessment and planning tool. Technology and Teacher Education Annual,
1996 (Society for Information Technology and Teacher Education [SITE]).
Charlottesville, VA: Association for the Advancement of Computing in
Honey, M., & Henriquez, A. (1993). Telecommunications and K-12 educators:
Findings from a national survey. NY: Center for Technology in Education.
Jones, B.F., Valdez, G., Nowakowski, J., & Rasmussen, C. (1995, April).
Plugging in: Choosing and using educational technology. IL: North Central
Krueger, R.A. (1994). Focus Groups: A Practical guide for Applied Research
(Second Edition). Newbury Park, CA: Sage Publications.
Lawyer-Brook, D., & Sherry, L. (1996, March). Creating Connections: An
Internet training program for rural K-12 teachers. Jacksonville, FL: Paper
presented at the National Conference on College Teaching and Learning.
Levin, S. (1995). Teachers using technology: Barriers and breakthroughs.
International Journal of Educational Telecommunications 1 (1), 53-70.
Lewis, J.H, & Romiszowski, A. (1996, November). Networking and the
learning organization: Networking issues and scenarios for the 21st
Century. Journal of Instructional Science and Technology, 1 (4).
Main, T. (1996). Teaching teachers to use telecommunications: The British
Columbia experience. CSS Journal. [On-line.] Available:
Morison, E. (1984). Gunfire at sea: Conflict over a new technology. In R.
Westrum, & K. Samaha (Eds.), Complex organizations: Growth, struggle, and
change. Englewood Cliffs, NJ: Prentice-Hall.
Patton, M.Q. (1987). How to use qualitative methods in evaluation. Newbury
Park, CA: Sage Publications.
Peled, Z., Peled, E., & Alexander, G. (1994). An ecological approach for
information technology intervention, evaluation, and software adoption
policies. In E.L. Baker, & H.F. O Neil, Jr. (Eds.), Technology assessment
in education and training. Hillsdale, NJ: Erlbaum.
Peters, T. (1988). Thriving on Chaos. NY: Harper & Row.
Rogers, E.M. (1995). Diffusion of innovations (Fourth Edition). NY: The
Ruelle, D. (1991). Chance and Chaos. Princeton, NJ: Princeton University
Rossett, A. (1991). Needs assessment. In G.J. Anglin (Ed.),
Instructional Technology. Englewood CO: Libraries Unlimited.
Sherry, L. (1997, in press). An Integrated Technology Adoption and
Diffusion Model. To appear in International Journal of Educational
Spielberger, C.D, & Starr, L.M. (1994). Curiosity and exploratory
behavior. In H.F O Neil, Jr., & M. Drillings (Eds.), Motivation: Theory
and research. Hillsdale, NJ: Erlbaum.
Stockdill, S.H., & Morehouse, D.L. (1992). Critical factors in the
successful adoption of technology: A checklist based on TDC findings.
Educational Technology, 32 (1), 57-8.
Teasley, U.H. (1996, February). Factors affecting teachers decisions about
use and non-use of computers: Administrative actions. Indianapolis: Paper
presented at the AECT In-CITE 96 Conference.
Turkle, S., & Papert, S. (1991). Epistemological pluralism and the
revaluation of the concrete. In I. Harel, & S. Papert (Eds.),
Constructionism. Norwood, NJ: Ablex.
U.S. Department of Education. (1996). Advanced Telecommunications in U.S.
Public Elementary and Secondary Schools, 1995, FRSS 57, NCES 96-854,
Washington DC: U.S. Government Printing Office.
Wason, T.D. (1996). A Gutenberg Juncture: Changing the model of
instruction in a new technological environment. CSS Journal. [On-line.]
Wilson, B.G., Ryder, M., McCahan, J., & Sherry, L. (1996, February).
Cultural assimilation of the Internet. Indianapolis: Paper presented at
the AECT In-CITE 96 Conference.
Wolf, K., & Black, L. (1993). The Boulder Valley Internet Project first
annual progress report: 1992-1993. (Available from Elizabeth Black, BVSD,
Box 9011, Boulder, CO 80301.)
Yin, R.K. (1994). Case study research: Design and methods, Second edition.
Thousand Oaks, CA: Sage Publications.