Understanding the past – illuminating the future

Nick Rushby and Jan Seabrook

Nick Rushby and Jan Seabrook are the founders of Conation Technologies Limited, a strategic training consultancy that carries out research and development projects in technology based learning.  Both have worked in the industry for over 20 years and were closely involved in the developments that took place in the period 1980 to 2000.  Address for correspondence: Nick Rushby, Conation Technologies Limited, The Office Building, Gatwick Road, Crawley, West Sussex RH10 9RZ, UK. Tel: +44 (0)1293 459 807; email: nick.rushby@conation-technologies.co.uk

Those who ignore the lessons of history are doomed to perish by them (George Santayana)

Introduction

Many - perhaps the majority – of current learning technologists fail to take advantage of the lessons of earlier projects.   They do not link what went before with what is now.  This may be because of the excitement of working with leading edge technology so that there is no time to lose.  Perhaps it is because that they are unused to the effects of disruptive technologies.  It may also be because the research and development is driven, not by learning technologists but by information technologists who are unaware of what has gone before.

The evidence to support these assertions can be found by looking at the literature searches described in published papers which failed to find relevant earlier work and by talking to typical project staff at conferences.  Time and time again well informed researchers appear to be unaware of projects undertaken before 2000.  Indeed, in our own research for this project we were unable to find references to some projects from this era in our websearches – even though we knew details such as the project titles and the names of the key researchers.  An example illustrates this point.  Another paper in this special issue describes Project Author.  This was a significant initiative to train staff for industry in the design skills needed for what we now call e-learning. We were unable to find a single reference on the Web to this project.  If anyone reading this paper has more success then we would be delighted to be corrected! 

It is almost as if our field started in the late 1990’s and that nothing of importance happened before that time. Yet, the two decades from 1980-1999 encompass a great deal of UK research and development in the use of technology in education and training.  There were major government funded programmes into the use of microcomputers in primary, secondary and higher education, the use of expert systems, and technology based training.  Industry too, researched the effective use of newly developed technologies.

If we fail to learn from history of ICT in education and work-based learning we are condemned to repeat it.  We will continue to cycle round and round the innovation cycle, unaware of the lessons from which we could learn, making relatively little progress. The consequences of this failure are not only measured in poor progress but in the costs of projects which do not deliver their full potential. 

This work is based on the premise that we need to identify the relevant, key projects from, say 1980 to 1999, and to make their outcomes accessible to today’s practitioners.  And we need to do this soon, while we still have access to the key players from that period.  Some of them have already retired (some unfortunately have already died), library holdings are being reduced and reports discarded, and memories fade.  The costs of retrieving this information and making it accessible are probably less than the true costs of a single research project.  We only have to prevent one new project from failing in order to recover our investment.  The aim of the project was to enable The British Educational Communication and Technology Agency (Becta) and the UK ICT research community in general to build on what has gone before rather than to reinvent it, and thus achieve more from their research budget.  In addition to a report which would be available in print as well as on the web, the intention is that as much as possible of the information gathered during the project will be collated and made available on the Becta website.

Not all of the projects carried out between 1980 and 2000 are relevant and retrieving information on everything would be prohibitively expensive.  The search was therefore informed by the current agenda for ICT in learning – and the likely future agenda over the next five years.  Where we encountered work that was considered to be important but not immediately relevant, it was book-marked so that it can be investigated further if necessary at a later date.

Cycles of learning technology

If we carried out a quick poll on the state of ICT in education and training at the end of 2007 we would get a range of views.  Some would extol its success, but others would say that it has still failed to live up to its promise – although the breakthrough is probably near.  If we look back through recent history – say the last 40 years – there is a ring of familiarity in these views.  ICT in education and training seems to have been on the edge of a breakthrough, just reaching critical mass, starting to deliver real benefits, for all of that time.  In part it is because we keep raising our expectations as technology become more capable.  We are continually dissatisfied with what we have achieved and we strive to do better by pushing the technology further.  In part it is because we fail to learn from what has gone before.  The thing that is consistent is that history teaches us that we don’t learn from the past.

Those who have been working in the field for a long time recognise repetitive cycles of innovation.  As illustrated in Figure 1, the adoption of a typical innovation starts slowly, then picks up pace until the market is saturated at which point the curve levels out – then starts to decay as users find and change to other solutions.

Figure 1: A simplified innovation adoption curve

In the case of ICT in education and training this process has been relatively short. Each new advance in technology produces a surge of activity with research and development into the possibilities of using the technology within learning, and this gives rise to the ‘New Technology.’  The surge of activity attracts a group of enthusiasts who extol the features of the innovation and a new cadre of researchers eager to explore its potential.  Speed is important here, and in the rush, learning design at both the overall and detailed level tends to be poor, squeezed out because there is insufficient time to do it properly.  Consequently the new learning may look good superficially but misses its objectives, and consequently gets a poor image because it doesn’t deliver the promise.  The full benefit of the new technology is never achieved because the good examples are submerged under the morass of the mediocre.

But often before the technology matures, it is overtaken by a new innovation.  This attracts a new group of enthusiasts and researchers.  The earlier groups are associated with the older technology, and now there is something newer and better. Too often the researchers and their experience fade with the technology.  And so we go round the cycle again.

One consequence is that new advances in technology are often driven by a group of people who have not been closely associated with earlier technologies.  Seabrook and Grigg (2000) found that in the commercial sector, 27% of those involved in developing technology based training had less than two years experience in the field, and well over half had less than five years experience.  In one instance a large UK-based ‘Do it Yourself’ company recently launched e-learning to train its staff, but appeared unaware that in the late 1980s-early 1990s it was one of the national organisations leading innovative use of technology based training. Subjectively, it appears that a similar situation pertains in many educational organisations.

Corporate memory can be lost in a relatively short time, and with it the lessons learned in previous uses of learning technology.  The lack of organisational memory is compounded when a new use of technology is treated as a special project.  These project teams tend to be very interested in innovation and getting things up and running, but once the project is launched the team is dispersed and people return to other positions.  Important issues such as evaluation and maintenance are not given sufficient priority.

Figure 2: Successive cycles of adoption

In practice, this cycle is not as simple as it looks.  Some innovations reflect progressive change in the technology - an improvement on what already exists.  For example, in the 1980s to 1990s there were several progressive changes in the way that we could incorporate video in learning sequences, from videotape and cassette, through videodisc (both laser and capacitative) and CD-I and then to now-familiar CD.   Disruptive innovations come from the introduction of radically new technology, such as the introduction into learning in the 1950s of the computer itself, the advent of the Internet, mobile computing, and most recently Web 2.0 which is enabling a flood of constructivist learning in education.  Throughout this, the most astute learning technologists have realised that it is wasteful to cast earlier technologies to the dustbin of history if they are still fit for purpose and can be used effectively in a blend of learning.

A changing agenda

It is inevitable that any research agenda is a dynamic thing; as our understanding moves on so the agenda must develop.  In this case the project was looking at outcomes from the research agenda 7 to 27 years ago and seeking those with contemporary relevance.  This approach can be – and was – questioned.  One of the members of the Sage Group, David Hawkridge observed that as the technology moves on the context changes and so the reports of research findings are inherently ephemeral.

“It’s not that we’re re-inventing the wheel all the time. Rather we’re deploying new technology to address old educational problems. The Web and the Internet didn’t feature at all in the research of that period … of course. And Web 2.0 applications don’t feature in research done in 2005...

“The great questions remain largely unanswered despite the changing context of new technologies: How do we or the learners decide what to learn and why, and when, where and how to learn it? How and by whom can this learning be assessed? The case can perhaps be made for recent studies ignoring some of the lessons of earlier ones in respect of educational design and evaluation.

“Many of the recent papers … focus on applying new technologies to the task of improving learning. They report research projects that are, inevitably, limited in scope, in respect of either the content being learned, or the group of learners, or the technology itself.

“Another important but less obvious reason is that the ideology and paradigms change too. In 1980 social constructivism was just emerging as an ideology in education. The quasi-scientific research paradigm adopted during the days of behaviourism was losing ground fast, particularly in the UK. Attempts at creating evidence-based educational technology cannot use the same paradigm as evidence-based medicine (which has problems galore in using that experimental paradigm, anyway). Controlled trials of ‘technology pills’ to determine their value simply do not exist, for very good reasons. Findings, such as they were, from so-called experimental studies in the 1980s don’t count for much today because they were often based on a paradigm now rejected as unsound. Findings of no significant difference between experimental and control groups at best failed to reject the null hypothesis (that there was no significant difference) and at worst failed to control for several variables, such as the teacher, the content, the mode of assessment, and so on.” (Hawkridge, 2007 personal communication)

Despite these reservations that the context changes as technology moves on, there are some research findings that are applicable to the newer technologies and the current research agenda.  For example, the Apple Newton technology has long been superceded but the findings from the research carried out on workplace assessment with British Rail and Aer Lingus in in late 1990s (Johnson, Rushby and MacLean, 2000; Rushby and Fairbrother, 1995; Rushby, 1996) are still relevant to assessment today and contribute to the Balanced Scorecard (see 4.3 in table 1).

Project methodology

The work was divided into a number of inter-dependent workpackages as shown in figure 3.

Figure 3: The project methodology

A key component of the project was the group of experts known as the ‘Sage Group.’  As the name implies, they were invited to join the project team because of their knowledge and experience of learning technologies during the 1980s and 90s.  These were people whose judgement we respected and who were immensely helpful in helping us to identify the key projects and to locate documentary evidence.

There were initially two parallel lines of enquiry: to understand the current Becta research agenda so that we could focus on projects that could make a useful contribution, and to start work on identifying possible projects for inclusion in the case studies and report.

The Becta research agenda is driven by the balanced scorecard shown in Table 1.  This has evolved since the project was first devised and the most recent version from the delivery plan for Harnessing Technology (Becta, 2006) is shown here.

Table 1: Becta’s balanced scorecard

From these aspirations we produced an expanded table (Table 3) that set out the kinds of evidence that we believed would contribute to the agenda, and used those as criteria for selecting key projects.  The task of identifying possible projects for inclusion was addressed from three directions. These were a literature search, asking the Sage Group, and locating various archives of materials. 

The initial literature search

Firstly, we looked at the tables of contents for some of the key journals and conference proceedings during that period.  These included

a.       The British Journal of Educational Technology. BJET is available online from Blackwell Synergy at http://www.blackwell-synergy.com/ .  A user account is required to access the full text of articles, but the abstracts and references are available free of charge.  A recent project has scanned and digitised all of the earlier issues so that the entire run is now available.

b.      Programmed Learning and Educational Technology. PLET was published under its original title until 1988 (Volume 25) but then changed its name to Educational and Training Technology International (ETTI). It retained this title for six years from 1989 to 1994 (Volumes 26 to 31) and then changed its name again in 1995 to Innovations in Education and Training International.  In 2001 there was yet another name change to Innovations in Education and Teaching International. The contents pages for PLET are available online from Informaworld at: http://www.informaworld.com/smpp/title~content=t713685495

c.       Interactive Learning International.  This journal was published by Wiley between 1984 and 1992.  Although the journal published a significant number of important accounts of learning with advanced technologies in the UK and North America it has not yet been digitised.

d.      Journal of Computer Mediated Communication. JCMC is published by Indiana University and is available on the web at http://jcmc.indiana.edu/issues.html or from Blackwell Publishing at http://www.blackwell-synergy.com/loi/JCMC

e.       Journal of Computer Assisted Learning. Issues of JCAL published since 1997 (but not those published earlier) are available from Blackwell Synergy at http://www.blackwell-synergy.com/loi/jca

f.        Aspects of Educational Technology.  This series formed the proceedings of the annual conference of the Association for Programmes Learning and Educational Technology (APLET).

There were many other journals that we could have included: inevitably our search was limited by time and other resources and so we focused on those that reported work from the UK.  The list of tables of contents was helpful in identifying potential projects for more detailed study and as a quick reference to some of the relevant published literature.

It is clear from this list that the major publishers are making strong efforts to make the earlier versions of their journals more accessible through the web. The costs of scanning and digitisation are justified by the commercial benefits of offering a more comprehensive coverage.  However, the scanned text is not always searchable and the citations are not always linked into the web of knowledge that enables researchers to track lines of research across different authors. 

The Sage Group

Next we asked the members of the Sage Group for their input:

a.       What do you think are the most significant learning technology projects of that period – and why?

b.      How do the findings contribute to the current research agenda?

c.       Where were the finding published and do you know where we can get a copy?

The archives

The third line of enquiry was to raid the physical archives of some of those who had been working in the field at the time – including the authors’ own collections.  The quantity of off-prints and printed reports that we obtained was remarkable and it took several weeks to catalogue and make some order from the material. In particular this yielded project briefs and reports from the Training Enterprise and Education Directorate (TEED) based in Moorfoot, and from the EU-funded projects Delta (Developing European Learning through Technological Advance) and RACE (Research into Advanced Communications in Europe) We had originally hoped to scan in much of this material but the quantities were much greater than anticipated and we were only able to digitise a small – but hopefully relevant – fraction.

In particular we were given access to a structured database containing 50 reports of  projects undertaken for the Department for Education and Employment concerning Open and Flexible Learning and Technology Based Training (May 1999).  While this is not a comprehensive list (there were far more than 50 projects!), the list in Table 2 gives a flavour of the wide scope of this unit.

To assist in managing the wealth of information on the TEED projects, we developed a second hyperlinked database of project details and descriptions.  The intention is that this will be made available through the Becta website in due course.  It is clear from this database that a great deal of research into open and distance learning was carried out under the aegis of TEED and the experience gained was extensively documented in a series of reports.

Few of the publications from the Department for Education and Employment remain in libraries.  The library at Moorfoot (together with the extensive collection of demonstration material and equipment) was dismantled when the space was required for other purposes.  A selection of the reports were taken by a university library but this holding too has been diluted over time.

A similar fate befell the CEDAR Collection (Computers in Education as a Resource) at Imperial College. CEDAR was established in 1979 at the end of the National Development Programme in Computer Assisted Learning (Hooper, 1977) and operated an international information centre on computer assisted learning and training.  It built up an extensive collection of books and reports which were transferred to the College library which the project came to an end in 1984.  Inevitably, demands on space led to the collection being destroyed: only a few of the most important pieces were saved.

The National Interactive Video Centre was created by the National Council for Educational Technology and acted as a focus for developing and disseminating educational and training applications of video disc technology.  When the Centre closed in the 1990s, its collection was transferred to Essex where it formed the nucleus of the National Archive for Educational Computing (see http://www.naec.org.uk/).  Much of the archive was gathered in the years 1990 - 2006 by Prof Stephen Heppell and Richard Millwood.  It is now in storage looking for a permanent home.

In the United States the Education Resources Information Center (ERIC) at Syracuse has been more successful in surviving over the years.  With sponsorship from the U.S. Department of Education, Institute of Education Sciences (IES)  ERIC provides free access to more than 1.2 million bibliographic records of journal articles and other education-related materials and, if available, includes links to full text (see http://www.eric.ed.gov/ ).

 Findings

The amount of material discovered, or uncovered, or recovered, by this project has been vast.  Many projects were carried out with direct relevance to today’s technology and culture, provided time and effort is made to extract the information that can be used, and this information can be used to start a new project using today’s technology because people still ultimately learn in the same way.  The way humans have learned for thousands of years.  The mistake – if there is one – is to think that because the technology has changed then people are learning in a new way.

“Bernard of Chartres used to say that we are like dwarfs on the shoulders of giants, so that we can see more than they, and things at a greater distance, not by virtue of any sharpness of sight on our part, or any physical distinction, but because we are carried high and raised up by their giant size.”  John of Salisbury 1159, Metalogicon

By using relevant information from the projects from the 1980s and 1990s current research projects can start from an established point rather than repeating work already done and coming to similar, if not the same conclusion.  This project was conceived when we encountered two current projects had not found any reference to previous work where such a relevant topic had been explored and reported and we suspect that there are many others.  These relate to Workplace assessment in hostile environments (1993) and Learning credit cards (1990).  Both are found in the case studies in the Appendix to this report.

Several of the Sage Group remarked that during the 1980s in particular there was an explosion of creativity producing computer based learning making use of technology advances, many of which are difficult to imagine now – such as the change from two colour, to seven colour to 256 colours to 16 million colours; the move from character graphics to bitmap graphics; coupled with increasing resolution, and falling costs.  So in a period of about 6 years learning technology had moved from text and graphics constructed from various characters (remember ASCII characters?) to an ability to show moving video at a price that was affordable for many.  There were also lessons learned about the use of colour, graphics, pace, and screen layout during this time which assist the learning.  Those lessons still apply even though the technical capability has improved immensely.

What is interesting is that, where the learning design is good, people appear to have learned using CBT/CBL irrespective of the richness and technical quality of the presentation on screen.  It was during this time of great technological change the emphasis on learning design became less significant and its importance ‘lost out’ to technology.  Just as we have lost the research lessons from the 1980s and 90s, that generation forgot the lessons and techniques from the programmed learning experts. 

Constraints in the design of  learning in the early 1980s forced developers to be inventive with the options they had.  The plethora of visual and auditory ways of conveying a message which have become available since then appears to have taken precedence over the learning design underpinning the e-learning in some cases.  Some current corporate e-learning developers have re-discovered that simplicity in presentation of material conveys the learning just as well as e-learning rich in graphics, video and animations.

Since the 1980s, e-learning has been referred to as Martini learning – anytime, anyplace, anywhere and the emphasis is on the flexibility of the technology.  However another aspect of CBT important in the 1980s and 1990s was the idea of ‘One-to-One’ learning.  Courses were designed to flow as if you had a tutor/trainer with you, rather than an electronic box.  This meant answer analysis was far more complex and feedback more comprehensive than is often found now.  It was not unusual for a four option multiple choice question to have 10 or more feedback options, allowing learners to have more than one attempt at the question and never reading or hearing the same feedback.  This is now very rare .  By having complex answer analysis encourages the learners and simultaneously allows them to feel the e-learning is actually personalised.  Also more common in the 1980s and 1990s was a wider range of question types, adding variety and challenge to both the learners and programmers.  Today question types available tend to be restricted, unfortunately in some cases to just True/False and multiple choice, a restriction originally imposed by the web but not applicable now.  Many of our Sage group remarked on these changes.

Other projects achieved a great deal with the technology available at the time that has since been refined and evolved.   Networks were used, mostly conveying data rather than personal communications, but innovative use was made of these – for example the Scottish Police College with NCC developed very realistic simulation training for crowd control using a network and the lessons can be useful for those developing similar applications today (Eary, 1995).  The information gained from games and VR simulations does not negate other lessons from earlier less technologically advanced simulations.

Another change with the rapid evolution from CAL to e-learning has been the apparent urge to condense elearning to a ‘one size fits all’. Before any CBT/CAL solution was started the designers considered the learning situation they had to address.  Not all problems are solved by providing learning: some require other approaches.  Secondly developers matched the technology to the learning requirements, using the software and hardware available to address the situation with maximum benefit to the learners and their organisation.  Unfortunately this process now often appears to be reversed with the technology driving the learning.

Great attention was paid to soft factors, such as motivation of learners and how the e-learning is going to be used; those implementation issues which can make or break a project ranging from why anyone would want to do the learning to questions about whether they can they use the learning once they have done it, what will be the result (such as better grades, better pay, a qualification), and even how do they find out the learning exists and how to access it.  Current drives to keep costs down have restricted the chance for many of these types of questions to be addressed and this probably contributes to un-met expectations of various projects.

The question of why the promise shown in the 1980s of artificial intelligence and its associated technologies was never realised in education and training is discussed in detail by David Welham (2008).  He suggests one reason has been that the focus of technology based training research has moved rapidly towards exploring the capabilities of the internet.  “The advent of the internet and the ability to deliver training and learning to a wider populace has been accompanied by restrictions of bandwidth that have mitigated against the development of AI type programmes which by their nature are bandwidth intensive.”  If this is so, then it is an example of cyclic innovation where the potential benefits of an earlier technology are lost in the enthusiasm to embrace the new.

More encouraging is Welham’s suggestion “that some of the techniques that were originally considered to be AI are now themselves part of the learning mix. Natural language processing is now not considered to be particularly innovative and expert systems in various guises are embedded in many current e-learning offerings. Embedded and context sensitive help, so revolutionary in times gone by, are now a common requirement of learning programmes. .. AI, as it was then known, has crept up on the world of learning by stealth and is secretly alive and well but embedded in the mainstream.”

But perhaps the major reason for the relative failure of AI to gain a foothold in the training world is cost.  If adequate learning can be achieved by cheaper alternatives then there is no business justification, either in education or training, for relatively AI techniques.  “It is the use of AI, in its varied forms, in appropriate circumstances’ to ‘enhance and enrich’ training systems that seems to be the role that has emerged after all the years of research.”

Despite the current lack of visible impact it is possible that AI techniques will be incorporated in the new generation of web tools to make information dissemination and retrieval more effective.  “If this proves to be the case then AI will have had a truly major impact on learning for the future though not the one envisaged by the researchers of the 1980’s and 1990’s” (Welham, 2008).

So many of the projects within those two decades – as now – can be seen in retrospect as isolated, in that they did not have an explicit plan for the future.  One of the reasons for the impact of the National Development Programme in Computer Assisted Learning was that every project funded by the Programme was required to have a robust plan for assimilation and dissemination (Hooper, 1977).  The Programme referred to this as ‘getting into the brickwork of the institution.’  The consequence was that most of the projects continued well beyond the end of the external funding, and their findings were assimilated into the body of knowledge of computer assisted learning.

In the training arena, Esdale (2007) has identified organisational conservatism as a major reason for project results being lost - in this case through apathy.  He cites a project carried out by Wicat for on the job training and assessment for aircraft maintenance staff using personal digital assistants (PDAs).  “The area we were using it for, commercial aircraft maintenance, is an extremely conservative discipline and, frankly, the idea was ahead of its time - so it disappeared. Wicat was a very innovative company, sometimes too innovative for its own good!”  This experience parallels that of the use of PDAs for Assessment of Workplace Competence in Hostile Environments (see Johnston, Rushby and MacLean, 2000) and the case study in the appendix.

We found that the practitioners and researchers we talked to were divided as to the value of the research carried out between 1980 and 2000 – and in consequence were divided as to the value of this project.  Some were really not interested in the work carried out prior to 2000, could not see its relevance given that the technology has moved on and believe that the future is clearly Web 2.0.  Others, once they were introduced to some of the projects, agreed that there was valuable information there.  And some clearly saw the benefits and value to current practice.

The two issues here are awareness and availability.  Through this project and the special issue of The British Journal of Educational Technology we have been able to introduce some members of the ICT community to some of the lessons of the past and we have been able to make a small amount of the documentation more accessible by digitising it.

In the longer term, there needs to be a national archive which can accommodate the documents and other artefacts and which has security for the future.  That will require that it is supported with long term funding and by a library that is prepared to commit space into the future.  The past 25 years have seen the demise of the CEDAR collection at Imperial College and the fragmentation of the materials and documents from the Department of Education and Employment at Moorfoot.  The artefacts from the National Interactive Video Centre (NIVC) and the Microelectronic Programme (MEP) have a precarious existence in a storage unit in Essex (see http://www.naec.org.uk/), but no permanent home.  The Domesday material has only recently been rescued from obsolescent oblivion by the CAMiLEON project (Wheatley, 2004) and, unless it can find a home it may again be overtaken by technology to the point where it needs rescuing again.

Unless there is a commitment, perhaps by an appropriate agency or a charitable foundation, to establish a proper national archive, this material and the knowledge which is to be found within it will be lost forever.  That would be an expensive tragedy.

Identified materials

The projects, articles and other documents from 1980s and 1990s yielded far more than we anticipated.  As much as possible has been recorded in ways that more people can access.  The range of topics is comprehensive, from specific subjects or skills through technology applications to the learning achieved using technology and where it was helped or hindered.  An example of the range is shown in Table 3, from DfEE projects.

Table 2: Fifty of the many projects carried out by the Department for Education and Employment on Open and Flexible Learning and Technology Based Training

The short list

From this mass of material we then applied the criteria of the match to the research agenda and our views (together with those of the Sage Group members) to select the projects that we felt most relevant to those embarking on research projects today.  The short list is shown in Table 3 and descriptions of each are given in the Appendix.  Their links to the Becta Research agenda and the balanced scorecard are shown in Table 4.

Table 3: Short list of projects 

Table 4: Types of evidence required to support the Balanced Scorecard