Full citation:
Sherry, L., Lawyer-Brook, D., & Black, L. (1997). Evaluation of the Boulder Valley Internet Project: A Theory-Based Approach to Evaluation Design. Journal of Interactive Learning Research, 8 (2), 199-234.

Copyright © 1997 AACE All Rights Reserved

 
EVALUATION OF THE BOULDER VALLEY INTERNET PROJECT: 
A Theory-Based Approach to Evaluation Design

 
Abstract

This paper summarizes a two-year evaluation of the Boulder Valley Internet
Project (BVIP).  A wide variety of evalulation methods were used, most
notably the construction of a theoretical model called the "Integrated
Technology Adoption and Diffusion Model" That guided data collection,
analysis, interpretation, and reporting.  The results were interpreted in
relation to several theoretical models of the innovation adoption process
within complex educational environments.
_______________________________________________________________________


Overview

The Boulder Valley Internet Project (BVIP) was established to introduce
teachers in the Boulder Valley School District (BVSD), Colorado, to the
use of the Internet. The premise of this five-year, NSF-sponsored
initiative was that the Internet can provide a rich variety of tools and
resources that may be used to enhance instruction and communication by
students, teachers, and administrators. Five research questions were
aligned with the project's objectives. These focused on the effectiveness
of the training component; the change in the participants use of the
technology; the influence of the project on curriculum and instruction;
the project's impact on the school and district as a whole; and possible
uses of the model by other districts. Data addressing the research
questions were collected from surveys, focus groups, interviews, a work
group, an embedded case study, and an examination of BVIP system logs.
Though participants' use of telecommunications has increased due to a
successful peer training program and expanded district-wide connectivity,
integrating Internet-based activities into the classroom is encountering
barriers in terms of time lags and lack of classroom access. Moreover, the
technology and its associated tools changed significantly throughout the
duration of the project. 

The data collected from this case study yielded an enormous amount of
information and revealed the complexity of the issues. It became necessary
to create a model that could identify key explanatory factors and organize
them into a coherent framework. The data both supported and greatly
expanded previous models of technological barriers, individual user
perceptions, and technology adoption and diffusion that were identified in
the review of relevant literature. As a result, the evaluation not only
addressed the five research questions but also organized findings into a
comprehensive model. The Integrated Technology Adoption and Diffusion
Model has four strands with multiple sets of variables that impact the
effectiveness of the technology adoption/diffusion process: technological,
individual, organizational, and teaching and learning factors. 

Findings were interpreted using nationwide surveys of technology usage,
Rogers (1995) Diffusion of Innovations, and Farquhar and Surry's (1994)
Adoption Analysis Tool, and Jones et al.'s (1995) Engaged Learning model.
Generalizing these findings to other settings will depend on the
district's degree of site-based management, its level of support for
telecommunications, and the level of involvement of all parties comprising
the its ecological culture. 


History

The Boulder Valley Internet Project (BVIP) was a collaborative venture
between the University of Colorado at Boulder and the Boulder Valley
School District, primarily funded by the National Science Foundation. 
Activities and funding were also coordinated with the Annenberg/CPB Math
and Science Project. The BVIP was designed to deploy Internet connectivity
to the BVSD, enhance curricula, analyze the effect of the network
technology on the educational process, and integrate the activities into
the workings of the district and surrounding community. The project was
intended to pioneer a comprehensive approach to K-12 use of networking and
the Internet, where planning and implementation involve district, teacher,
university, community, and regional participation. The BVSD contributed
funds, staff, release time, and the testbed itself. The University
provided funding, facilities, staffing, assessment, network expertise, and
curricular guidance. In the initial phase, Colorado Supernet (the Internet
service provider for the State of Colorado) and Westnet (an NSF-funded
regional network that provides Internet connectivity to a six-state
region) furnished connectivity, network platforms, and operational
support. 

The project was housed in the district office. The project leader, 
Libby Black, was hired by the collaborative project team to lead the
project because of her background in K-12 education. After the second year
of the project, she was supported by an administrative assistant who
coordinated the Internet accounts throughout the district. In 1995, the
BVSD funded a half-time position for a webmaster to coordinate the BVSD
home page. At the close of the project in the fall of 1996, both the
project leader and the administrative assistant were hired by the BVSD as
full-time employees to sustain the project's momentum.

The project leader, with support by the University of Colorado, trained a
cohort of 26 participating teachers from the BVSD in the summer of 1992. 
These teachers, in turn, became instructors and resource personnel for
their fellow teachers. The objectives of the initial intensive training
for the core group of peer trainers, and for later generations of
participating teachers, were to become proficient in the use of e-mail, to
become comfortable investigating Internet resources, and to become
experienced at exploring ways to integrate the use of the Internet into
their curricula. During the training period, the peer trainers spent an
average of five to seven hours per week on the Internet, with most of that
time in electronic mail (e-mail). Both the teachers who were already
sophisticated users of technology and the self-described "non-techies"
reported that they learned a tremendous amount about electronic networks
and the Internet, and that they were very enthusiastic about the
Internet's potential for promoting teaching and learning. 

The overall project, from deployment to curriculum, was subject to ongoing
assessment, with active dissemination of results. At the beginning of the
project, Dr. Kenneth Klingenstein of the University of Colorado was the
principal investigator for the grant. A year later he was replaced by Dr.
Kenneth Wolf, also of the University of Colorado. Together with the
project leader, Dr. Wolf carried out a formative evaluation at the close
of the first year of the project in 1993. In 1994, RMC Research
Corporation, Denver Office, submitted a proposal to perform subsequent
formative and summative evaluations of the project as an external
evaluator. RMC was awarded the evaluation contract by the BVIP because of
its expertise with instructional uses of technology. The evaluation team
consisted of Dr. Dianna Lawyer-Brook and Lorraine Sherry, both of whom had
worked extensively with educational telecommunications.

Over the five year span of the BVIP, both the project and the evaluation
design evolved significantly as connectivity increased, Internet tools
became more user-friendly, teachers and students acquired Internet
knowledge and skills, and telecommunications began to garner popular
support. In 1991, however, when the project designers first created their
expansive vision for the BVIP, little was known about effective
integration of the Internet into a school district. Even potential
benefits and barriers were a matter of conjecture.
 
The project designers believed, from information available at that time,
that the Internet could be of benefit to the educational community. Thus,
they developed a plan that included the development and delivery of
comprehensive district-wide training over a three-year period beginning in
July 1992 for a critical mass of teachers who wished to use the Internet
for instructional purposes, and the establishment of an initial foundation
for curriculum-related resources and activities. The value that could be
added to the district's curriculum and instruction by the use of
Internet-based activities and resources, however, was an unknown quantity
at the time, and is just beginning to be explored at present.
 
The initial cadre of 26 teachers used a "trainer of trainers" model to
diffuse their expertise throughout the BVSD via formal classes, school
presentations, individual tutoring, troubleshooting, and technical
support. The project personnel also developed a set of ancillary materials
and resources, a series of structured classes, and a set of open,
unstructured laboratory workshops. They elicited feedback from new
trainees to continually update and redesign the training program. Later,
they created a home page for the BVSD to support the training program, to
render assistance to new users, and to disseminate information about the
project throughout the district.

Formal classes were offered for professional development credit or
sometimes for pay at the convenience of BVSD teachers, primarily during
the summertime. Structured classes included a summer Internet seminar
series, a technical support series, a Technology in Learning Institute, a
writing center support series, and a Learning Partners Program Workshop
series that extended into the fall/winter of 1996. The content spanned
introductory classes in Macintosh computers for absolute beginners,
maintenance and repair workshops, troubleshooting printers, ideas for
improving technical support in your school, using Eudora, using Usenet
News, using ftp, "Everything you always wanted to know about UNIX but were
afraid to ask", designing a curriculum project utilizing the Internet,
creating your own Web pages, using PPP with the BVSD modem pool, the
World Wide Web, and the like. 

Unstructured, open labs were offered approximately twice a month at four
different schools from 3:15 to 5 PM, during which time teachers could drop
in and practice their Internet skills, network with other BVSD teachers
using the Internet, and get help from a knowledgeable BVSD teacher.
Additionally, if a teacher wished someone to come to his/her home and help
install PPP and get it up and running, the project leader compiled a list
of students, staff, and community members who would be willing to provide
this service for a small, negotiable fee.

Since there were few books on the Internet in 1992, ancillary materials in
the form of handouts for the training sessions were all designed and
produced by the project leader, based on the best on-line documentation at
the time. They included information on e-mail and the various Internet
tools such as gopher, Archie, Veronica, ftp, telnet, and the like; a
sample Internet curriculum unit; a staff contract agreement and
application for a BVSD Internet account, together with an Acceptable Use
Policy; troubleshooting lists; links to exemplary Web sites, listservs,
and other on-line resources; and other instructional materials. As the
Internet evolved, these materials were continually updated, based on
feedback from participating teachers. Later, the project team designed a
notebook that organized all of these print resources and gave a copy to
every trainer and every school throughout the district.

Although the grant period has ended, the BVIP is continuing at a rapid
pace. All but three schools were connected by January 1997. The project is
not an isolated effort in communications, but rather, a system wide effort
that both impacts and is impacted by the numerous factors in the
educational system. Throughout the two years of the current case study,
the Internet itself evolved. So did the scope of the BVIP, and along with
it, the model that was used to gather and report data from the evaluation.
Teachers moved through various phases of growth, starting from exploration
and professional development, through communication and generation of
on-line learning communities, to eventual use of Internet activities in
the classroom.

The hopes of the project organizers for integration of
telecommunications-based activities into curriculum and instruction have
not been fully met due to the inhibiting characteristics of significant
educational reform itself, as well as the slow pace of connecting all 53
of the district's schools, paucity of incentives such as release time or
extra pay for teachers who have taken on new duties as a result of their
Internet expertise, and ongoing cutbacks in FTE and technical support at
the building level. Instead of simply being viewed as a teaching tool, the
use of the Internet becomes entwined within program improvement, a much
more complex undertaking. This study describes the complex system that has
evolved and is continuing to evolve. 


Purpose and Scope of Study

According to Patton (1987), evaluation research, particularly at the local
program level, has been largely non theoretical. In contrast, the RMC
Research Corporation evaluation team took a case study approach that was
inductive, pragmatic, and highly concrete. In addition to building a sound
theoretical base, the evaluators incorporated specific data that were
relevant to the project, focusing on issues that were helpful in
monitoring or fine-tuning program operations. The scope of the evaluation
began as a global inquiry using a survey and interviews; then shifted to a
detailed examination with a case study and focus groups. 

The role of the independent evaluation team was threefold. They provided: 

1. Formative evaluation: ongoing information to the district about what
was working, what was not working, and how the current system could be
improved; 

2. Summative evaluation: feedback via a comprehensive report at the end of
the project on the degree of success in meeting its objectives and its
impact on users; and 

3. An expanded theory base and model: data that were used to gradually
build a theoretical model throughout the two-year data collection phase,
grounded in observations of program activities, impacts, and the
relationship between treatments and outcomes. 

After the completion of the first year of the project, Wolf and Black
(1993) conducted a formative evaluation using a survey and interviews. The
data suggested that the initial cohort of 26 peer trainers were
successfully trained; they felt that they were more knowledgeable about
the use of Internet as a resource in teaching and learning; and they were
beginning to apply their skills and knowledge in their classrooms through
Internet-based activities and projects. Wolf and Black also identified
five barriers that directly impacted teachers' use of the Internet and
that have continued to affect the level of Internet usage throughout the
duration of the project: (a) access, (b) time, (c) training, (d)
resources, and (e) usability. 

In 1994, RMC Research Corporation was engaged to expand upon that
formative evaluation. The evaluation team examined the training component
to ascertain its value, both in terms of whether the training accomplished
the short-term goals of being clear, useful, and engaging, and to
determine whether the teachers were actually implementing in their
classrooms the information and skills they gained from the training. The
team also investigated the influence of the project on the development of
new curriculum and teaching strategies and the benefits of participation
for the entire school and district. Toward the end of 1996, they evaluated
the BVIP as a whole to determine whether it may be easily replicable or
transportable to other districts, and how it may be improved. The
Integrated Technology Adoption and Diffusion Model that emerged from the
BVIP evaluation is currently being restructured and will be used to both
design and evaluate technology implementation plans in other states. 


Review of Relevant Research and Development of the Model

In order to build a sound theoretical base for the case study, the
evaluation team reviewed current literature that dealt with barriers to
technological innovations and the factors that either enhanced or
inhibited the change process by the adopting educational organization.
Three approaches emerged. One looks at technology adoption from the
perspective of the technology, including access, cost, type and age of
available computers and hardware, the physical aspects of the school
network, reliability, and user interface or system ease of use. The second
considers the viewpoint of the user, encompassing both user
characteristics and users' perceptions of the technology. The third
focuses on the complex needs of the educational institution itself - a
school or district situated within an organizational environment, which,
in turn, is situated within the district and local culture as a whole. 

There was precious little information concerning how teachers were using
the Internet to assist them in their teaching, or whether they thought
that using the Internet in the classroom was a good idea or not. In fact,
most evaluations of the effectiveness of technology focused on the
technology itself; they did not examine the effectiveness of
telecommunications as a tool for learning. In evaluating the BVIP, the RMC
Research team utilized the Engaged Learning model of Jones, Valdez,
Nowakowski, and Rasmussen (1995). These authors stated that engaged
learners are responsible for their own learning, are energized by
learning, actively develop their own repertoire of thinking and learning
strategies, and develop new ideas and understanding through conversations
and collaborative work with others. 


Technological Factors

Based on Wolf and Black's (1993) initial findings, the evaluation team
began by examining recent studies of school districts that were building
telecommunications infrastructures to connect their schools. Honey and
Henriquez (1993), Heaviside and colleagues, (U.S. Department of Education,
1996), and Levin (1995) described a number of technological barriers to
successful adoption and diffusion of Internet-based classroom activities
including limited funding for advanced telecommunications hookups, lack of
equipment or poor equipment, and too few access points in the school
building. 

Results from a comprehensive national survey (U.S. Department of
Education, 1996) indicated that in about half the schools that were
currently connected to a wide area network, district and regional
administrators played a large role in developing the school's
telecommunications program, while only about a third reported that
teachers and other staff took the lead. Thus, the BVIP was in a rather
unique situation in that the staff development program was funded by the
National Science Foundation, whereas the technology was primarily
supported by the district and its collaborative partner, the University of
Colorado at Boulder. BVIP Project directors worked closely with district
teachers and technical support staff at a grassroots level to create and
disseminate its telecommunications program.

Despite the BVIP training program and technical support that was provided
through the grant, many teachers were not able to use the Internet to its
fullest potential because they could not gain regular and dependable
access to the information superhighway. Wolf and Black identified two
important technological barriers to Internet use: access and usability.
Data gathered from the current evaluation of the BVIP revealed that the
most important technological factors affecting Internet adoption were
access to the wide area network from home or school, access to the local
area network within the school, presence and capacities of servers,
modems, and computers, usability, network response time, and network
reliability. 

Though Internet drops to individual schools were gradually provided by
district bond funds, and though the BVIP provided funds for ongoing
network upgrades and technical support, full, district-wide connectivity
was not possible until January 1997. Moreover, because of the site- based
decision making process of the district, the local area network
configuration within each building was under the jurisdiction of each
individual school. 


Individual Factors

The second approach, namely, looking at the technology from the point of
view of the end user, is exemplified in Rogers' (1995) Diffusion of
Innovations model. In the case of the BVIP, the users include district
administrators, support staff, teachers, and students. Individual factors
include both the personal characteristics that these users bring to the
technology and their perceptions of the value that it adds to the
instructional process. 

Based on 30 years of research on the diffusion of innovations within
organizations, the Rogers model discusses the social process by which
people adopt a new idea or technology. An organization (such as a school
district) is envisioned as a social system in which people work to achieve
a common goal through established practices and processes. The novelty of
a technological innovation creates uneasiness and uncertainty among
members of the social system that can adversely affect the rate of
activities that take place as the new technology is adopted by the
organization. This uncertainty, in turn, creates a curiosity/anxiety
conflict in new users, that is described in detail by Spielberger and
Starr (1994). In the Rogers model, "Uncertainty implies a lack of
predictability, of structure, of information. In fact, information is a
means of reducing uncertainty." (Rogers, 1995, p. 6).

Some people are naturally cautious; others are more curious, more willing
to explore and innovate. Spielberger and Starr's (1994) model of epistemic
curiosity describes a dual process consisting of anxiety and curiosity.
Individuals possess normal, random variations in both of these
characteristics, not only as a personality trait, but also in relation to
their state of relaxation or agitation at any given time, which, in turn,
depends on the context and situation in which they find themselves. The
lower the comfort level of the new users of an innovation, the less
willing they are to experiment with it. 

Similarly, Hall and Hord (1987) describe the stages of concern that an
individual goes through when adopting a change or innovation: 

0. Awareness (little concern about or involvement with the innovation);
1. Informational (interest in learning more details about it);
2. Personal (concerns about its demands and their adequacy in meeting
them);
3. Management (processes and tasks of using the innovation);
4. Consequence (impact of the innovation on student outcomes);
5. Collaboration (coordination/cooperation with other users); and
6. Refocusing (altering or replacing the innovation).

Each of these stages is characterized by specific questions, concerns, or
anxieties about the innovation. In the initial, information-gathering
stage, a potential user has some general awareness of the innovation and
is interested in learning more about it. However, uncertainty and
self-concern enters the picture while the individual focuses on his/her
adequacy to meet the changing demands within the existing environment. At
the task management stage, the individual focuses attention on the
processes and tasks related to the use of the innovation. Later, this
focus shifts to reflections about how the innovation impacts students and
the community; its consequences for others in the environment; the
cooperative use of the innovation in coordination with other teachers and
students; and what benefits might arise from altering or replacing the
innovation. 

A critical element of the Rogers model is the concept of classifying
adopters and potential adopters on the basis of their innovativeness, in
other words, how soon they adopt an innovation. Potential adopters vary in
socioeconomic status, personality values, and communication behavior. The
five categories of adopters are (a) innovators, who are the first to
adopt; (b) early adopters, who are often the opinion leaders in the group;
(c) early majority and (d) late majority adopters, who form the bulk of
the adopter group; and (e) laggards, who are often the last to adopt.
According to Rogers, the "S-shaped adopter distributions closely approach
normality" (Rogers, 1995, p. 261.) 

Though the Integrated Technology Adoption and Diffusion Model drew heavily
from the Rogers model, the initial data that was collected revealed that
one cannot simply characterize early adopters as "techies" and late
adopters as "technophobic". Such simplistic labeling of the adopting
population cannot fully describe the complex relationship between the
technology and the human element. Identified user characteristics such as
motivation, need for control, and comfort level, coupled with expertise,
reasons for and patterns of use, gender, and special needs definitely
influenced both the content and the process of acquiring expertise in
telecommunications. Knowledge, comfort level, and usage levels were
closely related, as those with greater experience took on leadership
roles. 

In a recent study targeting users' perceptions, Wilson and colleagues
(Wilson, Ryder, McCahan, & Sherry, 1996) reported that some individuals
who resisted using a new schoolwide Internet system felt intimidated by
the experts within the school whom they labeled as "techno-gurus". They
felt that these communications experts were speaking a new, different,
technological language, filled with mystical expertise, jargon, and
commands that were simply not worth learning.

Rossett (1991) suggests that, when dealing with "new stuff", especially
new technologies that are being implemented, the feelings of the users
become the critical issue. In contrast to treating the situation as a
performance problem in which there is a gap between optimal and actual
performance, when potential users are adopting a new technology, there is
a gap between optimals and feelings. This is an important distinction.
Rogers himself states that "[p]ersonality variables associated with
innovativeness have not yet received full research attention, in part
because of difficulties in measuring personality dimensions in field
interviews" (Rogers, 1995, p. 272). 

Based in part on the Rogers model, Farquhar and Surry (1994) developed an
Adoption Analysis Tool, which takes both the characteristics and the
perceptions of the users into account, since their levels of concern and
uncertainty regarding the innovation are echoed in their perceptions of
the new technology. According to Farquhar and Surry, user characteristics
affecting adoption are (a) motivation, (b) anxiety, (c) knowledge base,
(d) prior experience, and (e) skill level. As mentioned above, these
characteristics, among others, were evident in the initial survey and
interview responses from the BVIP evaluation. 

The user perceptions enumerated by Farquhar and Surry are the same as
those of Rogers, namely, (a) relative advantage of the innovation over the
existing system; (b) observability of the innovation's consequences; (c)
compatibility with users' values and needs; (d) complexity vs. simplicity;
and (e) trialability - in other words, can the innovation be tried out on
a limited basis? Their Adoption Analysis model states that the more
positively new users perceive an innovation with regard to these five
characteristics, the greater the likelihood that the innovation will be
adopted. 

Wolf and Black's (1993) initial evaluation also described similar user
perceptions, as did the data collected from the present case study. Some
teachers reported that the Internet can be very difficult to navigate, and
that many of its avenues are open only to those who are quite
knowledgeable about the system. Hence, even if the network is fully
implemented and is functioning at full capacity, simply gaining access
will not solve all of the problems that teachers may encounter as they try
to find useful resources and integrate them into their classrooms. 


Organizational Factors

Besides individual factors, the organization as a whole must be
considered. In the third approach, organizational factors are concerned
with the complex needs of an institution. These deal with the overlap
between what the district brings to the diffusion process and the impact
of the innovation. In Farquhar and Surry's (1994) model, organizational
factors involve both the physical environment and the support environment
in which Internet-based classroom activities are to be used. An example of
the difference between an organizational factor concerning the physical
environment and a technological factor concerning availability of
technology is the use of a library modem. The modem may be working and in
good condition (a technological factor), but if it is constantly used by
the librarian and is not open for use by any of the teachers, that is an
organizational factor. 

The physical environment includes the availability of computers and
Internet connections in classrooms or other contexts in which learning
will take place, availability of other important resources, and existing
organizational arrangements such as scheduling practices. Key physical
environment factors identified in the BVIP evaluation included classroom
connectivity, network capacity, scheduling of laboratories and computers,
availability of equipment, and availability and proper use of supplies
such as diskettes, toner cartridges, and paper that were necessary for
storing and printing out student-generated products. 

The support environment, in Farquhar and Surry's Adoption Analysis model,
includes (a) production, (b) storage, (c) delivery, (d) dissemination, and
(e) support services. These factors were not entirely appropriate for the
BVIP, in which the organizational support environment includes district
policies such as planning time and grading policies, incentives for staff
development, administrative and technical support, problem-solving
mechanisms, training, maintenance, and information resources. 
Of the five key elements identified by Wolf and Black that affected BVSD's
use of the Internet, three were organizational factors - time, training,
and resources. Throughout the current evaluation, however, it became
evident that these three factors were multi-faceted, and that several
related issues were being confounded in an attempt to simplify and limit
the number of organizational factors that impacted the BVIP. 

Specific support environment factors identified in the BVIP case study
were administrative vision and support, district policies such as
site-based decision making, planning time, and grading policies,
acceptable use policies and handling of student accounts, availability of
communication channels, mechanisms for identifying and solving problems,
availability and timeliness of training workshops, availability of
resource and technical support personnel, availability of documentation,
troubleshooting lists, and job aids, availability of on-line support,
maintenance of the network and associated equipment, lack of release time
to pursue training, lack of incentives for teachers who take on new duties
as a result of their Internet expertise, lack of a shared knowledge base
of curriculum implementation resources, non-district and global support
via the Internet, and support from and cooperation with other grants. 
Beyond providing support through making the physical technology available,
a district as an organization needs to be sure that the new effort is
aligned with the district-wide vision and policies, as well as integrated
into the established communication system. As problems arise, the
administration needs to have mechanisms in place that can address new
challenges and improve upon the responses until they are seamless and
integrated into the whole. 

Hoffman (1996) identified eight success factors for teachers who are
trying to adopt technology in their classrooms and to build the support
they need for using technology: (a) administrative support; (b) staff
development and technical support; (c) availability of technology; (d) a
technology use plan; (e) a district computer or technology coordinator;
(f) facilities and maintenance personnel who are included in planning for
technology integration; (g) assessment and evaluation methods which
reflect new educational approaches; and (h) broad participation by
individual teachers, groups and committees of teachers, the school-site
computer coordinator, the principal and other administrators, and
district-level coordinators or administrators, superintendents, and school
boards. 

In contrast to the physical/support environment factors that were
identified by Farquhar and Surry, and Hoffman; Gross, Giacquinta, and
Bernstein (1970) focused on organizational factors from a management point
of view. Their factors included the following: (a) a clear vision of the
innovation provided by administrators to all teachers, stakeholders, and
users; (b) the staff's skills and capacity to implement the innovation;
(c) the availability of required tools and resources; (d) the
compatibility of the school's organizational arrangements (such as the
grading system and scheduling of classes) with the new innovation; (e)
taking account the difficulties to which teachers may be exposed as they
begin to implement the innovation; (f) providing mechanisms to identify
and cope with unanticipated problems that may emerge during the period of
implementation; and (g) the wholehearted support of the administration for
the innovation and for the teachers who are implementing it. 

Recent researchers have begun to integrate the technological, individual,
and organizational factors in a more systemic fashion. Stockdill and
Morehouse (1992) identified five critical factors that affect the
successful adoption of new technology: (a) educational need, (b) user
characteristics, (c) content characteristics, (d) technology
considerations, and (e) organizational capacity. Farquhar and Surry (1994) 
also built on Stockdill and Morehouse's model in developing their Adoption
Analysis tool. Successful implementation requires not only that adopters
buy in to the use and application of the innovation, but also that the
adopting organization provide a worthy environment in which to use the new
technology and all of the resources and services needed to install and
maintain it.

Studies of failed innovations such as those of Gross and his colleagues
(1970) and Teasley (1996) often provided insights into the organizational
factors that affected the diffusion process. These researchers reported
that, though the initial attitude of the school administrators and
teachers was positive and that they were willing to expend the time and
effort to implement the new technology, they ran into difficulties along
the way. Teachers began to express frustration with the project, and
administrators failed to give it their full support. Moreover, the
administrators had no mechanisms in place for dealing with unforeseen
problems that might arise during the implementation process.
Communications broke down and teachers continued to run into problems with
which they simply could not cope.

Like any other complex system, school districts depend on good internal
communications if they are to work effectively. A complex system, however,
often embodies information that is hard to get (Ruelle, 1991). In other
words, communication problems may arise between administrators and
teachers or between teachers themselves. Carlson (1970) suggested that
when the key change agents are dispersed (in his case, the district
superintendents), then those who have clear communication channels with
each other tend to work closely together. Those who do not have good
communication links with others tend to remain isolated from the change
process. 

Peled et al. (1994) and Egan (1985) viewed a school district not simply as
an educational organization, but as a cultural entity with a common
cultural blueprint that sets the pattern for the structures and processes
that occur within and across the systems (i.e., the classroom, the school,
the community, and the regional educational policy-making institutions).
Morison (1984) suggested that opposition to innovations is cultural in
nature and stems from the normal instinct to protect one's self and one's
way of life. People will withhold judgment or even express disbelief about
the dramatic claims of the new innovation. They will try to protect the
existing system with which they identify themselves as well as the
existing society of which they consider themselves a part. 

Lewis and Romiszowski (1996) studied learning organizations - educational
institutions which constantly update the skills of their members and in
the process, are themselves transformed. They used a five-element model
that is similar in structure to the BVIP technology adoption model
presented here, consisting of (a) technology-related, (b) organizational,
(c) personal, (d) pedagogic, and (e) cost and resource-related success
factors. They found that, from an organizational learning perspective,
there is a considerable organizational learning curve to be traversed
before organizations not only assimilate a new technology into their
culture, but also accommodate their culture to be able to profit from the
technology to the extent that is theoretically possible. They suggest that
familiarity with the new technology and the ability to apply it in one
context do not necessarily guarantee either the ability or the motivation
to use it successfully in novel contexts or to integrate it in a major way
into a new teaching program. When networking begins to be used seriously,
the nature of the teaching and learning environment is inevitably changed.
They recommend that further research efforts are needed to investigate the
optimum mix of networking and other learning activities to enhance the
learning environment for students and teachers, and to explore the
decision making factors that influence the level of support that is given
to new instructional innovations. The interaction of learning and adoption
that took place in the BVIP will be reported in a separate paper (Sherry,
1997).

Though a complex system tends to maintain a stable state, it may begin to
change itself, "provided it begins by defining for itself its grand
object, and see[ing] to it that [the] grand object is communicated to
every member of the group" (Morison, 1984, p. 142). Peters (1988)
remarked, 

What matters is that everyone who works for and with you observe you
embracing the topic with both arms and your calendar. What they need to
observe is your obvious, visible and dramatic, determination to batter
down all barriers to understanding and implementation. (p. 501)

In summary, the adoption and diffusion process can viewed in three ways:
as a social process (the Rogers model), as a cultural process (Peled and
Egan), and as a systems process (Ruelle and Morison). These overlap, since
systems are cultures, and cultures have social processes. Within the
system there are barriers to diffusion. Some of these are sociocultural,
such as the individual characteristics of the end users; others are
involved with maintaining the stability of the system as it presently
exists. Moreover, the system itself imposes constraints on the adoption
and diffusion process. To be successful, a new technological innovation
requires a worthy environment that can support it; resources to install,
implement, and maintain it; readily available, two-way communications
channels; and an overall administrative vision that is clearly
communicated to all members and key stakeholders. 


Teaching and Learning Issues

Teaching and learning issues center around the impact of technologies and
technology-enhanced educational programs on students engaged learning. As
Lewis and Romiszowski (1996) state, an educational system must be studied
as a learning organization in which all members are actively involved in
both planning and participating in learning programs adapted to the
specific requirements of the changing work or social environments in which
they find themselves. 

The use of new technologies, especially telecommunications, enables
teachers and students alike to focus on developing meta-skills that can be
transferred to real-world situations, rather than simply concentrating on
developing specific areas of content knowledge or subskills. In the Center
for Applied Technology's (1996) national study, a comparison of the work
of 500 students in fourth-grade and sixth-grade classes in seven urban
school districts showed that students with on-line access achieved
significantly higher scores on measurements of information management,
communication, and presentation of ideas. Student learning was measured by
evaluating students' final products based on nine learning measures in
which students were required to demonstrate their knowledge, apply their
skills, and illustrate their thinking processes. The authors state that
"using the Internet can help students become independent, critical
thinkers, able to find information, organize and evaluate it, and then
effectively express their new knowledge and ideas in compelling ways"
(p.1). 

This leads to a host of new instructional methods that are just beginning
to be tried out in the classroom. One such method is a variation of the
Socratic Method, described by Wason (1996). Instead of lecture-based
instruction followed by testing, students can learn by a system of
carefully developed questions that guide them through the learning
process, rather than using questions as vehicles to test the learning
product. This model of teaching combines an effective system of resource
access with content-relevant navigation and landmarks; sequences of
questions used as the underlying instructional tool that enable students
to discover knowledge relationships and acquire learning skills; and a
technology integrated model of teaching whereby teachers select questions
matched to the needs of the specific students, who then work
asynchronously, either alone or in self-selected groups. As a result, the
educational process becomes student centered and the learning process
takes precedent over the learning product. 

Such innovative models are fraught with difficulties, especially in a
district like the BVSD, where the current emphasis is now shifting toward
instruction based on model content standards and where the type of
assessment used by the district is for the most part still
norm-referenced. Though an instructional model similar to Wason's has been
used successfully in multi-age, multidisciplinary classrooms such as the
science class at Nederland Elementary School, it is not necessarily
applicable for typical classes that were observed in the BVIP case study. 

Initially, the Engaged Learning model of Jones and his colleagues (1995)
was incorporated into the Integrated Technology Adoption and Diffusion
Model. Engaged learning issues comprise the various learning styles and
roles of students in the classroom, authentic and relevant tasks,
multidisciplinary curriculum, interactive and generative activities, and a
learning context that emphasizes collaborative knowledge building. Jones
and his colleagues identified eight variables that are related to a set of
indicators of engaged learning: (a) the teacher's vision of learning; (b)
relevant, challenging tasks; (c) ongoing, authentic, performance-based
assessment; (d) a constructivist instructional model responsive to student
needs; (e) the concept of students as part of a learning community
incorporating multiple perspectives; (f) collaborative learning; (g) the
role of the teacher as a facilitator, guide, and
co/learner/co-investigator; and (h) the roles of students as cognitive
apprentices, peer mentors, and producers of products that are of real use
to themselves and others. 

In Boulder Valley, the teacher's vision of learning is closely related to
his/her role in the classroom and his/her perception of the relationship
of the classroom curriculum to state and district standards, whether the
existing curriculum is to be enriched, enhanced, or replaced, and the
precise role of Internet-based instructional activities in the classroom.
The teacher's role also involves using students as assistants,
collaboration in sharing ideas, resources and lessons with colleagues,
sifting through and organizing relevant classroom activities from the
massive volume of information available on the Internet, generating
meaningful lesson plans, ideas, activities, and resources for classroom
use, and tailoring existing Internet resources to meet the needs of the
classroom. 

In the Integrated Technology Adoption and Diffusion Model, teaching and
learning issues were divided into curriculum and engaged learning factors.
Curriculum factors comprised changes and enhancement in content, the
volume of content to be explored, planning and preparation, the
relationship of lessons and standards, the use and sharing of lesson
plans, and the evaluation and categorization of Internet activities. The
engaged learning factors were essentially those of Jones and his
colleagues, as mentioned above. 

Though the initial plan of the BVIP included both the development and
delivery of comprehensive district-wide Internet training for all
interested teachers and the establishment of a sharable resource of
curriculum-related resources and activities, only the former has been
fully implemented. The first steps toward establishing a resource access
with content relevant navigation and landmarks as described in the Wason
model were undertaken by the BVIP, in cooperation with the Annenberg/NSF
Creating Connections project, when they created the Science and Math
Initiatives (SAMI) database. They also assisted district teachers who
built several school home pages which were linked to student activities
and resources that could be shared by teachers. 

As part of the current evaluation, a work group of BVSD teachers also
created a classification scheme for Internet-related resources, titled
CLUE, which will be described later. This product will assist them in
forming the foundation for curriculum-related ideas, activities, lesson
plans, and resources that can be accessed via the Internet. 


The Integrated Technology Adoption and Diffusion Model

After compiling and reviewing the research on the four strands of
variables that impact the adoption and diffusion of a new educational
innovation by an organization, and after collecting the initial data via
surveys and interviews, RMC Research created a model that targeted the
most important variables and applied it to the BVIP. As the data
collection proceeded, and as the information that was gathered was coded
and sorted, the model was expanded with additional information. Table 1
presents a summary of all of the factors that influenced the adoption and
diffusion of the BVIP throughout the Boulder Valley School District. 

Study Methods and Data Sources

Setting

Because the purpose of the study was to build an expanded theory base and
an Integrated Technology Adoption and Diffusion Model as well as to
provide formative and summative evaluation to the funding agency, the
evaluation was conducted as a case study. According to Yin (1994), case
studies are generalizable to theoretical propositions, and not to
populations or universes. The applicability of the BVIP model to other
school districts will depend in large part upon the administrative vision
and support and the structure of the decision making process of each
individual district, whether site-based or district-wide. 

The District 

The Boulder Valley School District (BVSD) is a large, dispersed, partially
mountainous district comprising 53 elementary, middle, and high schools,
situated to the northwest of Denver. The City of Boulder is home to the
University of Colorado. 

An Individual School (an embedded case study)

Nederland Elementary is an isolated, rural K-5 school that is situated in
a small mountain town within the BVSD. It is a high-end user school that
made good use of the Internet to expand the learning environment beyond
the walls of the classrooms and beyond the small community of which it is
a part. For several years, it was also one of the few schools in the
district with schoolwide connectivity. 


Evaluation design

After carefully reviewing the relevant research literature and discussing
the project with the BVIP project director, five central research
questions were formulated that underlie the investigations of this case
study: 

1. How effective was the training component of the project? 

2. How did the project specifically affect the participants use of the
technology? 

3. What was the impact of the project on curriculum and instruction? 

4. What was the impact of the project on the schools as a whole? The
district as a whole? 

5. What are the possible future uses of this model? 

Because the BVSD was a complex system, multiple measures were used in
order to produce converging lines of inquiry. This approach involves both
qualitative and quantitative methods, as seen in Table 2. The five
research questions were examined and matched with the most appropriate
data collection instruments. The data collection instruments included (a)
an e-mail survey, (b) indepth interviews, (c) a curriculum focus group,
(d) an Internet activity classification work group, (e) an embedded case
study, and (f) documentation analysis.

Initial information on the effectiveness of the training component was
gathered from a district-wide e-mail survey of the 435 teachers who were
identified as regular Internet users via the system logs. The quantitative
data were supplemented and enriched by in-depth interviews with the BVIP
project leaders and eight teachers who were using the Internet in their
classrooms. More detailed data were obtained through the curriculum focus
group, the Internet activity classification work group, and the embedded
case study of Nederland Elementary School. The documentation analysis
involved examining the BVSD system logs, previous project documents and
evaluations, newspaper articles, white papers, and other artifacts. 

Methods of Data Collection and Analysis


E-mail Survey

A 42-item survey was e-mailed to the 435 district teachers who had e-mail
addresses.  A follow-up reminder was sent out one month later.  142
responses were received, a response rate of about 33%.  Responses to
forced-choice questions were reported as percentages or in chart form.
Responses to open-ended questions were reported verbatim and were
clustered into naturally emerging categories, which were then checked by a
second researcher to enhance reliability.


Indepth Interviews with Project Personnel and Teachers

Indepth interviews lasting approximately two hours were conducted with the
project leader, the project administrative assistant, and eight teachers
who were active users of technology and had used some Internet-based
activities in the classroom.  Tapes were transcribed, shown to
participants for their approval, and then coded. 


Focus Group

Ten educators from various schools throughout the district were convened
in a focus group that addressed strategies, benefits, problems, and
content of instruction that incorporates use of the Internet. All focus
group participants had worked with Internet-based curriculum development.


Internet Activity Classification Work Group

The work group took place after the focus group. It comprised a group of
teachers who intended to create an assessment product that could be used
for evaluating curriculum resources that utilized the Internet.  All work
group members were experienced users of the Internet in the classroom.
What eventually evolved from the group's efforts was a classification
system that would aid the district in organizing resources, lesson plans,
thematic information, and other helpful resources in a centralized
location. The resulting classification scheme, CLUE, is presented in Table
3.


Table 3
 
CLUE 
________________________________________________________________________
 
The BVIP Classification of Teaching and Learning Units (CLUE)
that use the Internet Effectively
 
Title: ______________________________________________
 
Content Area: (circle one)
Art Language Arts Math PE Science Social Studies 
Integrated (more than one area) Other: __________________
 
Specific Topic: _______________________________________
 
Grade Level: (circle one) K-3 4-5 6-8 9-12
 
Duration: (circle closest time frame) 
1 period more than 1 day weeks 1 or more semesters
 
Degree of Collaboration: (circle one)
within the classroom within the district outside the district
 
Technology Requirements: 
Software: ___________________________________________
Hardware: __________________________________________
Internet Connectivity: __________________________________
Other: ______________________________________________
 
Date of Entering on Clue: __/__/__
 
Abstract:________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
 
Teacher Comments: 
 
Date: Comments
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
_______________________________________________________________________
 

System Logs

BVIP system log information that covered the period from August 1995
through January 1996 was made available for all schools in the district.
The numbers of log-ins were tabulated for 1995 and 1996.


Embedded Case Study

As part of the BVIP evaluation, RMC Research carried out an embedded case
study at Nederland Elementary School (See Yin, 1994). This enabled the
evaluators to gain some insight into the school's organizational change
process, and to cross-check the self-reported data that were gathered in
the e-mail survey and the indepth interviews. The site was purposefully
selected because of several important factors, namely: 

1. The trust established between the evaluators and the school staff; 

2. The high participation of the school staff in the district's peer
training process; 

3. The high degree of connectivity in the school; and 

4. The high percentage of educators in the school who were regular
Internet users as demonstrated by the BVIP system logs. 

The Nederland Elementary School case study used three data collection
instruments: (a) classroom observations and focus groups with students
from a fifth grade class at nederland Elementary School and a multi-age
science class at Nederland Middle/high School who were currently using
Internet-based activities; (b) a brief survey of teachers,
paraprofessionals, school administrators and support staff; and (c) short,
focused interviews of selected teachers, the principal, and BVIP project
leaders. In the embedded case study, RMC Research concentrated on
organizational and individual factors that were not addressed by either
the e-mail survey or the indepth interviews. Attention was also devoted to
the degree of systemic change.


Findings

The Integrated Technology Adoption and Diffusion Model proved effective
for collecting, organizing, and reporting the findings from all phases of
the evaluation. The information that was gathered was then used to answer
the five research questions. 

1. How effective was the training component?

Overall, the BVIP was successful in meeting the goals of its training
component and raising the level of expertise in the use of the Internet
throughout the district. The survey data revealed that the BVIP classes
and workshops were the primary resource for most district teachers (94 out
of 142 respondents) who were learning how to use the Internet. In addition
to the formal training that they received, some participants requested
additional workshops in site-based small groups to meet the needs of
particular schools and programs. They also requested more emphasis on
authentic uses of the Internet as they can be applied to either an
existing or an enriched curriculum. Success facilitators included the
Trainer of Trainers approach, the training program's focus on classroom
teachers rather than a top-down approach, and the flexibility of the
training program in keeping up with the rapid changes in the technology. 

Problems can happen in a lot of different places, so we are running
technical workshops for every school in the district. When they come to
the workshop, they get six hours of consulting time to help them figure
out what they are doing. We give them a troubleshooting list so they can
determine if the problem is in the building, the district, or the
Internet, so you can call the appropriate person. (A Trainer)

Barriers to success included insufficient staff development time, delays
between training and access at the school level, and cuts in district
funding in a variety of areas. These barriers were magnified when the
original, more progressive superintendent and school board were replaced
with new members who espoused a more conservative educational philosophy,
at about the same time as both the BVIP and Annenberg funding ended.

Key finding

Since the training was aimed at typical classroom teachers rather than
administrators and policy makers, expertise was diffused horizontally at
the grassroots level, rather than percolating upwards toward all levels of
the educational system. The BVIP project director commented on the fact
that, in a site-based model, it is generally the administrators who are
trained last:

A lot of principals don't use a lot of technology...so it is not really
clear to them how their kids are using it or what a difference it is
making. This has to do with the district having to take a stand on
believing that it is necessary for administrators to understand this and
to see what it can really do in the curriculum. If they are going to do
that, then everybody has to have easy access, and you have to train
people. (BVIP Project Director) 


2. How did the project specifically affect the participants' use of the
technology?

The most significant influence on the participants' use of technology was
the increase in communication with colleagues, experts, and friends within
the building, within the district, or outside the district. Using e-mail
for communication and accessing the WWW for both teacher- and
student-initiated research became the most popular uses of the Internet.
Teachers and students became proficient in Internet use and accessed many
educational resources that supported their growth. One focus group
respondent commented,

The main thing with the project is when it was first started, where the
technology was then, and where it is now. If I had to start getting back
with all those [individual tools], ftp, and so on...I mean, I love the
Web! That is what I live on. (A Teacher)

Barriers included an increased work load for some new trainees, lack of
observable benefits for the additional training involved, declining morale
because of district funding cuts, and the reassignment of in-building
technology resource personnel to the classroom. 

Key finding

Though the training program was successful in increasing the number of
Internet users in schools throughout the district, and though there were
examples of highly innovative student-generated projects, there was
evidence of a negative feedback loop that tended to slow the adoption
process down. 

Bond money to purchase computers and connections was held up for over a
year due to a lawsuit.  This led to a gap between training and curriculum
integration. The volume of content had increased dramatically, yet there
was still no common, district wide base of curriculum-related resources
for teachers to tap. To add to the turmoil, many of the in-building
technology specialists were reassigned to the classroom due to continued
funding cuts within the district and the phasing out of grant funding. 

The kids come down and do a search on Netscape all the time. Teachers who
are trying to integrate technology need a good starting place. I'm
everybody's tech support. I teach classes for the kids, for the teachers,
do maintenance, and all the planning. (A Teacher)

Often, teachers questioned whether or not their training was really worth
all the effort, especially if they took on new duties and responsibilities
as a result of their newfound expertise.  Though the core group of
teachers were given a choice of pay or credit for their participation in
the training program, that training was not integrated into building
professional development programs, nor was it seen as a must-have by
principals at the time.  In other districts that are contemplating such a
training program but that may not have incentive programs in place,
expecting teachers to be trained on their own time means that only those
who are truly devoted and already have an interest will pursue training
(Gray, 1997). 


3. What was the impact of the project on curriculum and instruction?

The impact of the project on teaching and learning was strongest when the
classrooms had direct access to the Internet and when the teachers'
instructional style matched their uses of the Internet. A "facilitator"
teacher commented:

What works for kids is more important than somebody telling you what to
do. Why would I sit there and teach types of clouds when I know that, in
science, the process is sometimes more important than being able to show
them multiple choice. (A Teacher)

The project leader expanded on this issue:

One way in which the Internet enters the classroom scene at an opportune
time is that in many places the curriculu, especially science, is being
shittled down so that fewer topics are taught, but more in depth and more
for understanding.  The Internet can really support this in-depth teaching
approach due to the vast amount of resources and options for collaborative
work.

Key finding

Teachers' need for control influenced the ways in which they used the
Internet for instructional purposes. Loss of control was due in part to
the open framework of the Internet, and in part to the rapid gain in
expertise by the students, which sometimes resulted in a novice/expert
role reversal between students and teachers. 

Age had little effect on expertise, but gender affected both the amount
and type of use, with boys using the Internet more for exploring and
creating Web pages and girls using it more for communication. This is
consonant with Turkle and Papert's (1991) findings that girls tend to use
a negotiational approach with regard to computers, whereas boys prefer
activities that are characterized by abstract thinking and systematic
planning. Both boys and girls, however, used the WWW regularly for
research projects. Some teachers were concerned about what subject content
might be replaced by Internet-based activities.

The weather unit was a large expansion. It was so big that it replaced a
lot of things. It's OK at my school, but it meant skipping other topics.
If you are spending eight weeks doing weather, what are you not teaching?
(A Teacher)

Teachers who regularly used the Internet for class-related purposes had a
variety of strategies that they used to plan and prepare for their
classes, including having their students assist them in finding
curriculum-related resources on the WWW, participating in mailing lists
with colleagues and experts who shared their interests, and sharing
information and activities with other teachers and associates throughout
the district. As a result, the focus of the curriculum shifted from
developing technology skills to enriching instructional content. One focus
group participant commented,

There is so much stuff out there, just keeping up with it [presents a
problem]. I haven't done that much in the last couple of years, but I did
in the beginning. It was more personal to find [information on] the Web
out there, but now I turn it over to the kids, and the students have a
base that is available. (A Teacher)

Managing the volume of content and decreasing the complexity of searching
for relevant information could be made possible by developing local
resources. Teachers who participated in the summer work group designed a
classification scheme for organizing Internet resources and relating their
content to the curriculum. This classification could be used at the
district level to develop a foundation for curriculum applications. A
focus group participant discussed the issue of shared, locally created,
lesson plans and tests:

Regarding whether or not we talk about putting up lesson plans, I know
that the math group talked about whether we could try putting tests for a
particular course at one particular page on the BVSD Web site...different
schools would post tests that they use for a particular chapter so that
all the math teachers could see what other schools are doing and what
their expectations are for that particular course. That request came up in
the last curriculum council meeting. (A Teacher)


4. What was the impact on the school as a whole? the district as a whole?

Regarding the impact of the project on a whole school, the embedded case
study of a rural mountain elementary school revealed a wide range of
expertise among the staff. Some teachers had a high level of expertise,
whereas the support staff hardly used the network at all. Students and
teachers with home access used the Internet more at school than those who
lacked home access. 

Key finding

A sharp division existed between the use of the Internet as a special
class in a computer laboratory and as part of regular classroom
activities. Instructional use of the Internet was hampered because the
computer laboratory was often considered a place for students to gain
technology skills during teachers' planning periods. 

There's a split in our school [between] people wanting it in the computer
lab and people wanting it in their room. I think that has been a divisive
focus because we have a group that really wants those kids to go to the
computer lab. Those teachers don't want anything to do with it. They just
want that instructor. But I see technology as more of a tool in education,
something that's woven throughout everything that we do. When it's
isolated in the computer lab, that's great for the kids, but the teachers
can't do anything with them, so I think there's a real breakdown here. (A
Principal)

This issue was discussed in the local newspaper (Boulder Camera, May 19,
1996).  Cuts in classroom budgets to pay for teachers' raises in 1996 and
1997 meant that fewer schools would be able to allocate funds to pay for
technology specialists.  Some educators were concerned that they couldn't
continue making gains in expertise at the current rate, and that they
would not be able to deliver technology services without the specialists. 
Others, however, indicated that the district's intent was to have
computers into the classroom anyhow.  One school board member was
concerned that half the schools have technology integrated into the
classroom, while the other half subscribe to the "specialist" model.  

I don't mean to penalize schools, but if some schools are doing it
differently than other schools, it would seem not a wonderful idea to
provide money to schools that have chosen something that is not in line
with the technology curriculum. (A School Board Member). 

The district's technology director continued to work to help schools find
a short-term solution to the long term problem of getting all tachers
trained and comfortable using computers in the classroom, rather than
letting the equipment sit around unused.  But such a short-term solution
is not found so easily.  One of the peer trainers said:

You have to know the curriculum and the tie-ins with the curriculum. To
find that hook for the teacher has been different for every person.

Because of the efforts of the peer trainers, diffusion of
telecommunications throughout the district was successful at the
grassroots level, but over the course of the evaluation it was hampered by
limited connectivity. Two diffusion models were originally proposed but
were not implemented. The vertical model was not used because the middle
schools had Internet connections from the beginning of the project, but
the elementary and high schools did not get their Internet connections
until much later. All of the high schools were connected by January 1997.  
The horizontal model was not pursued because, at the time, it was decided
that this innovation entailed changes in curriculum. With the
incorporation of standards and the ensuing curriculum revisions that were
taking place, pursuing these innovative approaches was simply not
possible.


5. What are the possible future uses of this model?

To answer this question we must focus on the actual successes of the BVIP. 

1. It was a grassroots effort that successfully diffused horizontally
among those classroom teachers who participated in the training sessions. 

2. The Trainer of Trainers model successfully capitalized on the empathy
between change agents and clients. 

3. The project had initial buy-in by the superintendent and the school
board. The teachers, however, have felt less support from the new school
board, and perhaps even opposition to them being  innovative in the
classroom, despite the board's continuance to pay the salaries of the
project leader and the webmaster.

4. The project worked successfully with other grants and sources of
support such as the Annenberg/CPB Math and Science Project, the district
bond funding, and the University of Colorado.

Future plans for the BVIP include access at all schools and greater
attention to curriculum content and instructional strategies. Other
districts that are considering replicating the BVIP efforts should
consider their own style of management and decision making processes when
designing approaches to the adoption and diffusion of technology. 

Key finding

Though the Rogers model may apply well to centralized organizations, it
fails to explain the intricacies of the learning and adoption process that
take place when an evolving technological innovation is introduced into a
district characterized by site-based management.


Conclusions and Recommendations for Further Research

The BVIP proved to be a dynamic, evolving program housed within a
decentralized educational organization. The reality of the BVIP was quite
different from Rogers' Diffusion of Innovations model. The Internet and
the user interface evolved dramatically over the five-year course of the
project, becoming far more user-friendly, far less complex, and far more
accepted by the general populace. Participatory design based on feedback
from typical users resulted in continuous improvement of the training
workshops. Thus, later adopters had a much easier learning process than
the innovators. As a result, it would be extremely difficult to
distinguish the impact of the BVIP per se from the impact of the Internet
upon the culture as a whole.

As students gained expertise, there were some reversals of novice/expert
roles between teachers and innovative students. Teachers who saw
themselves as facilitators capitalized on this process and utilized the
more knowledgeable students as a support network. Innovative teachers
continued to facilitate engaged learning among students, resulting in
exemplary student products and performances. In contrast, traditional
teachers perceived classroom use of the Internet as a threat that lessened
their control over the teaching and learning process. 

During the first two years of the project, growth accelerated among the
initial core of 26 peer trainers and the more innovative teachers. Later
on, the lack of extrinsic incentives, district-wide cuts in funding and
planning time, and the increased work load for new trainees acted as a
reinforcing feedback process. Although there was initial buy-in by a
visionary school board and superintendent, that support faded as policy
makers were replaced by more conservative members. One teacher commented,
"The school board has to learn how important ongoing staff development is
in all areas of instruction and learning!"

Site-based management inhibited horizontal sharing of ideas and resources
among connected schools, even though the patterns of Internet use
increased among participating teachers. Decentralization also affected the
use of shared resources and centralized sources of technical support:

We have not used the district resource person as much as we could have,
and that's because we had a technology resource person right in the
building. We have a troubleshooter. He can fix machines, he can do
anything. I see the teachers pull back and not use the district resources
because we had people right there for us. (A Teacher)

Vertical communication was hampered because teachers did not migrate to
the Internet since they did not have access to the system that was used by
the administrators. Moreover, the middle schools were connected long
before the elementary or high schools. One teacher commented, "Students
come to my high school from middle schools with good connections and then
are disappointed when they get here and have poor access."

The Trainer of Trainers model was successful in increasing the expertise
of classroom teachers, but in its initial stages it ignored the rest of
the ecological system that included the school administrators, parents and
the community, the school board, and the new superintendent. Inservice
training for administrators and parent information sessions were only
offered when they began to be requested by schools and administrators - a
reactive, rather than a proactive, stance. Moreover, since training
occurred prior to full connectivity, a gap resulted between learning how
to use the system and effective classroom use.

This is a trend that has plagued other large-scale educational
telecommunications projects. It is especially evident when the major
policy makers and the teachers have different perceptions of
telecommunications and its appropriate uses in the classroom, when
equipment is purchased before a comprehensive, district-wide technology
plan is in place, when teacher-trainees are not given appropriate time and
incentives to pursue the training that is offered, when in-building
support begins to become scarce, and when the district's projected budget
and timeline run into snags several years into the implementation phase of
the project.

If there is a lesson to be learned from the BVIP, it is this - information
technology interventions cannot be separated from their ecological
contexts, nor from the educational activities that they enhance (Peled et
al., 1994). Interventions must aim not at individual participants, but at
the entire ecological environment - the classroom, the school, the
district, and the community in which the educational institutions are
housed. Since the use of the Internet becomes an integral part of program
improvement, it is important for those who control resources and carry out
district policies to be involved in both supporting and participating in
the training activities, so that they can create, sustain, and communicate
a clear administrative vision throughout the entire educational system. In
schools where principals and technology resource people provided vision
and support for technology, the use of technology flourished.

Advertising student successes via student-produced home pages and
high-quality student-generated products can be quite instrumental in
promoting the use of the Internet within those schools that are already
connected. It is also important to have policies in place that allow
students to use the Internet during their free periods, and before or
after school, especially if they do not have access at home.
 
Although teachers are aware of the many possibilities that the Internet
offers for enhancing teaching and learning in the classroom, many need
further guidance in strategies for integrating telecommunications into
their curriculum and instruction. Future research efforts at RMC
Research Corporation will focus on three important areas:

1. How can or will teachers use technology to attain their instructional
objectives?

2. How can or will technology fit in with teachers' current teaching
styles?

3. What value can technology add to the teaching and learning process?


References

Boulder Valley Internet Project. (1993). An Overview of the Boulder Valley
Internet Project. (Available from Elizabeth Black, BVSD, Box 9011,
Boulder, CO 80301.) 

Carlson, R.O. (1970). The adoption of educational innovations. In M.W.
Miles, & W.W. Charters, Jr. (Eds.), Learning in social settings. Boston,
MA: Allyn & Bacon. 

Center for Applied Special Technology (CAST). (1996, November). The role
of online communications in schools: A national study. [On-line.]
Available: http://www.cast.org/stsstudy.html 

Egan, G. (1985). Change agent skills in helping and human service
settings. Monterey, CA: Brooks/Cole Publishing Company. 

Farquhar, J.D., & Surry, D.W. (1994). Adoption analysis: An additional
tool for instructional developers. Education and Training Technology
International, 31 (1), 19-25. 

Gray, S. (1997). Training teachers, faculty members, and staff. In B. Khan
(Ed.), Web-Based Instruction, pp. 329-332. Englewood Cliffs, NJ:
Educational Technology Publications, Inc.

Gross, N., Giacquinta, J. B., & Bernstein, M. (1970). Failure to implement
a major organizational innovation. In M.W. Miles, & W.W. Charters, Jr.
(Eds.), Learning in social settings. Boston, MA: Allyn & Bacon. 

Hall, G.E., & Hord, S.M. (1987). Change in Schools: Facilitating the
Process. Albany, NY: State University of New York.

Hoffman, R. (1996). School technology integration: An automated needs
assessment and planning tool. Technology and Teacher Education Annual,
1996 (Society for Information Technology and Teacher Education [SITE]).
Charlottesville, VA: Association for the Advancement of Computing in
Education. 

Honey, M., & Henriquez, A. (1993). Telecommunications and K-12 educators:
Findings from a national survey. NY: Center for Technology in Education. 

Jones, B.F., Valdez, G., Nowakowski, J., & Rasmussen, C. (1995, April).
Plugging in: Choosing and using educational technology. IL: North Central
Educational Laboratory. 

Krueger, R.A. (1994). Focus Groups: A Practical guide for Applied Research
(Second Edition). Newbury Park, CA: Sage Publications. 

Lawyer-Brook, D., & Sherry, L. (1996, March). Creating Connections: An
Internet training program for rural K-12 teachers. Jacksonville, FL: Paper
presented at the National Conference on College Teaching and Learning. 

Levin, S. (1995). Teachers using technology: Barriers and breakthroughs.
International Journal of Educational Telecommunications 1 (1), 53-70. 

Lewis, J.H, & Romiszowski, A. (1996, November). Networking and the
learning organization: Networking issues and scenarios for the 21st
Century. Journal of Instructional Science and Technology, 1 (4).
[On-line]. Available:
http://www.usq.au/electpub/e-jist/vol1no4/abstrac4.htm#Abstractlewis 

Main, T. (1996). Teaching teachers to use telecommunications: The British
Columbia experience. CSS Journal. [On-line.] Available:
http://www.webcom.com/journal/t_main.html 

Morison, E. (1984). Gunfire at sea: Conflict over a new technology. In R.
Westrum, & K. Samaha (Eds.), Complex organizations: Growth, struggle, and
change. Englewood Cliffs, NJ: Prentice-Hall. 

Patton, M.Q. (1987). How to use qualitative methods in evaluation. Newbury
Park, CA: Sage Publications. 

Peled, Z., Peled, E., & Alexander, G. (1994). An ecological approach for
information technology intervention, evaluation, and software adoption
policies. In E.L. Baker, & H.F. O Neil, Jr. (Eds.), Technology assessment
in education and training. Hillsdale, NJ: Erlbaum. 

Peters, T. (1988). Thriving on Chaos. NY: Harper & Row. 

Rogers, E.M. (1995). Diffusion of innovations (Fourth Edition). NY: The
Free Press. 

Ruelle, D. (1991). Chance and Chaos. Princeton, NJ: Princeton University
Press. 

Rossett, A. (1991).  Needs assessment.  In G.J. Anglin (Ed.),
Instructional Technology.  Englewood CO: Libraries Unlimited.

Sherry, L. (1997, in press). An Integrated Technology Adoption and
Diffusion Model.  To appear in International Journal of Educational
Telecommunications.

Spielberger, C.D, & Starr, L.M. (1994). Curiosity and exploratory
behavior. In H.F O Neil, Jr., & M. Drillings (Eds.), Motivation: Theory
and research. Hillsdale, NJ: Erlbaum. 

Stockdill, S.H., & Morehouse, D.L. (1992). Critical factors in the
successful adoption of technology: A checklist based on TDC findings.
Educational Technology, 32 (1), 57-8. 

Teasley, U.H. (1996, February). Factors affecting teachers decisions about
use and non-use of computers: Administrative actions. Indianapolis: Paper
presented at the AECT In-CITE 96 Conference. 

Turkle, S., & Papert, S. (1991). Epistemological pluralism and the
revaluation of the concrete. In I. Harel, & S. Papert (Eds.),
Constructionism. Norwood, NJ: Ablex.

U.S. Department of Education. (1996). Advanced Telecommunications in U.S.
Public Elementary and Secondary Schools, 1995, FRSS 57, NCES 96-854,
Washington DC: U.S. Government Printing Office.

Wason, T.D. (1996). A Gutenberg Juncture: Changing the model of
instruction in a new technological environment. CSS Journal. [On-line.]
Available: http://www.webcom.com/journal/wason.html 

Wilson, B.G., Ryder, M., McCahan, J., & Sherry, L. (1996, February). 
Cultural assimilation of the Internet. Indianapolis: Paper presented at
the AECT In-CITE 96 Conference.

Wolf, K., & Black, L. (1993). The Boulder Valley Internet Project first
annual progress report: 1992-1993. (Available from Elizabeth Black, BVSD,
Box 9011, Boulder, CO 80301.) 

Yin, R.K. (1994). Case study research: Design and methods, Second edition.
Thousand Oaks, CA: Sage Publications.