Corporate Memory

Front matter

Corporate Memory

Records and Information Management in the Knowledge Age

2nd Edition

Preface to Second edition Introduction

Chapter I: Empowerment Information as a resource The demise of command structures Recycling Browsing and retrieving A new soul

Chapter 2: The memory problem The cost of losing corborate memory Causes of lost memory Premature discarding Not throwing things away in a timely manner Forgetting where it is Lack of documentation Loss of media The price of lost information

Chapter 3: The corporate memory and Records Management What should be part of the corporate memory?

Contents of corporate memory What should not be part of the corporate memory Purely personal information Copies Some copies of published materials

Chapter 4: The document as a verb The questioner Beyond storage and retrieval Assembling a document The disintegration of centralized computing Implications for the corporate memory viii Contents

Chapter 5: Valuing documents Confidentiality and privacy The Question of Worth Case files Subject files Retention schedules Rules as guidelines Rule of Worth I: Information used by important people is more likely to be important information Rule of Worth II: Information takes on value when it is used Rule of Worth Ill: Vital records are a part of the corporate memory Rule of Worth IV: Historical information is valuable Rule of Worth V: Information whose retention is required by law or regulation is valuable for the period specified in the law or regulation Applying the Rules of Worth

Chapter 6: The corporate memory manager The records manager Librarians Archivists An Archival Collection Preserving for Re-Use Records Managers and Archivists

Chapter 7: The technologies Requirements for the corporate memory system Browsing and searching Enabling technologies Relational and inverted index systems Thesauri and Communities Accessing multiple databases Desk-top scanning Standard generalized markup language Search tools Scripts The Internet Bringing the World Wide Web in-house Retrieving from the corporate memory Changing technologies

52 53

55 59 63 68 70 71 72

Contents ix

Chapter 8: Performance Centered Learning 89 Traditional training 89 Learning 90 Minimize training and support 9| The information specialist 92 Integrated learning 93 Tools 93

Chapter 9: Workflow 95 The revival of the registry system 95 Copying replaced registering 96 Management information systems 97 Integrating information 98 Business needs 98 SGML and workflow 99 Automating documents 100 Strategies for documentation 10]

Chapter 10: Improvements 105 A long-term goal 106 Improvement |: Taking inventory 107 Improvement 2: Duplicating copies or index cards 107 Improvement 3: Using shared servers 108 Improvement 4: Imaging 109 Improvement 5: PC-based file-finding software 110

Getting started bl

Chapter Il: Making it happen 113 Alternative Starting Point |: People 114 Alternative Starting Point II: Electronic data 116 Alternative Starting Point Ill: Electronic documents 116 Alternative Starting Point IV: What comes in the door 121 Differing scenarios 122 A Knowledge Application Service 123 Using the corporate memory 124

Appendix: ISO Standard 135

Glossary 137

Index 141

Preface to Second Edition

Much has happened in the decade since | first conceived this book on Corporate Memory, yet little has changed. Carlos Cuadra and Judy Wanger introduced me to the term "corporate memory," when | was working with them to develop one of the first automated records management software packages. Now corporate memory is a widely accepted and widely used term. It is now most often used in the discussion of "knowledge management" — a term that a decade ago did not exist. Ten years ago even the most optimistic of us underestimated the speed at which technology is changing the way we do our work.

A decade later | have returned, hopefully with a better understanding, to the problem of how an organization can capture, organize, retrieve and re-use the information that it creates. In the Knowledge Age that is developing all around us, the essential evidence! of an organization is a valuable resource that is lost or poorly utilized.

Since | wrote the first edition of this book, we now have new terms and words we can use to understand our situation. But the problem | identified in the first edition of this book — the loss of corporate memory — has not been solved. Indeed, a combination of factors, some technological and some social and political, have converged to make the identification, preservation and re-use of corporate memory even more necessary than before.

| have added discussions of three new topics — one on digital archives and one on Knowledge Applications Services and a discussion of ISO Standard 15489 which gives some order to the field of records management. These topics were not on the table when the first edition appeared. | have also changed the subtitle to reflect the fact that this discussion is within the field of records management and that we are, indeed, leaving the Information Age and entering into the Knowledge Age.

The chapter on digital archives is my attempt to link the discussions in the archival community with those in records management. They are clearly one community, even though there is still little practical collaboration. The

! "Essential evidence" is a term used by the United States National Archives to describe what a record is.

xii Preface to the Second Edition struggle between archivists and records managers has, unfortunately, been primarily waged between those who see valuable information from the viewpoint of history (the archivists) and those who focus on the practical, business uses of information (the records managers).

By talking about digital archives, | believe that the fundamental unity of these two disciplines and crafts can be demonstrated.

The last ten years have given me several opportunities to apply and amplify some of the concepts that | articulated in the first edition of this book. Much of what | have learned is summed up in the modifications | have made in the original text.

| have also published two additional books that deal with topics related to the concepts dealt with in Corporate Memory.

Document Management, which is a collaborative effort with my dear friend and colleague, Herb Schantz, who introduced me to the fascinating and challenging world of capturing data from thousands, hundreds of thousands and even millions of forms and other documents. We were able to take the notion of a document as a verb, which | articulated first in this book, and apply it to a very different technological environment. In the context of document management, a document is the answer to a question, not necessarily a physical object. The technologies and techniques used by document managers can be applied, when understood properly, to virtual digital documents.

My most recent book, Thinking for a Living. The Coming Age of Knowledge Work gave me a chance to return to my first discipline, philosophy, and think through the concepts of knowledge and work in light of the startling technological and social developments of the past half century.

| have, where it made sense, included in this edition references to my other books and changed some terminology. | was surprised, for example, that | included no specific discussion of metadata. My omission of the importance of networks and the Internet are more understandable, since they were just becoming part of our work life as | completed the first edition. Now we live in what some call a Network Centric Environment.

Despite the apparent speed of technological change, the basic argument that | put forward in the first edition is still valid — we need to identify and preserve information of value for reuse: corporate memory.

Introduction

Corporate Memory is information that a corporation (organization or business) creates which is of value for re-use. This book is written for the corporate memory manager of the future. Information managers in the knowledge age will come from different professions that are in the process of merging. | do not argue that such integration should take place: | just assume that it is. | try to point out practical ways in which the information of an organization can be managed in the electronic age.

| come to this activity as a professional records manager who, like most records managers, had a previous career. | began my professional life as a philosopher and moved on to the world of organizations and politics before coming to information management. My roots show in this discussion, which | hope will be a contribution to the development of a systematic theory for information management. My work with many organizations has convinced me that many, if not most, of the problems of organizations and businesses are information problems, and if we can find practical ways to identify and preserve the memory of the organizations in which we work, our lives will be not only more efficient but also fuller.

Two colleagues, Carlos A. Cuadra and Judith Wanger, encouraged me to develop a coherent theory for managing information in the electronic age. Carlos wrote a paper a few years ago entitled "The Corporate Memory and the Bottom Line', suggesting that managing wordprocessed documents would begin to bring electronic documents under control. When he asked me to look at a draft of that paper, | recognized that he

2 Introduction had pointed the way to solutions for many problems facing records management in the electronic age. Some of the material in this book is taken directly from Carlos' original paper. He has also used his extensive editorial skills to help make this work more intelligible.

At about that same time | began working with Judith Wanger, Vice- President of Cuadra Associates, to develop an application for the STAR software package. STAR is a text-based database application developed by Cuadra Associates that | had used in my work as a records manager for a federal agency. When we began to develop the application, | tried to elaborate on the work of records management and identify how to automate its essential activities. Six months later Judith was able to show me an application that managed records from cradle to grave, without regard to the media in which they are stored.

| was attracted to records management in 1987 as an emerging profession of information management that focused on throwing things away in a timely and orderly manner. Establishing disposition schedules — the heart of traditional records management — was the primary tool used to bring records under control. In about 1992, a shift in the attitudes of the profession became apparent. One reason for this was the arrival of image technology as a usable and cost-effective tool for records management. The other was the development of wordprocessing in a networked environment.

With the arrival of imaging, wordprocessing and networking technologies, most records were no longer kept on paper. Rather, they were created and stored in electronic form. Thus a convergence of imaging, wordprocessing and networking enables offices to move records from place to place electronically without copying them on paper. Paper copies became copies of records, not the records themselves.

Many other people have helped me with this project. Most of all, | want to thank the many students who stimulated me to develop a theory for records and information management that will meet the needs of the next century. Leonard Mignerey, whom | worked with when he was MIS Director at Catholic University, introduced me to the world of management information systems and showed me how the world of data and the world of records are colliding. Most of the ideas in the chapter on the ‘new document’ came out of discussions with him and other colleagues in an informal seminar in 1995.

Barry Wheeler, my former colleague at Catholic University, who is now working to create the digital library at the Library of Congress. During the time | was writing the first edition, he was the engineer next door who patiently explained how to use the new tools that constantly arrive.

Introduction 3

At the time | was writing the first edition, he insisted that | move to Windows even though | had just barely begun to manage the strange world of DOS. Since then he has worked with me through many lunches where we mused over new technologies and how to make them really useful. Barry also showed me the difference between training and learning, and convinced me that in order for technology to be used anywhere, we needed to develop new ways to bring it into our lives. Noel Dickover later showed me how Performance Centered Learning can be done.

Sheryl K. Rosenthal worked with me to develop my first records management database at the Comptroller of the Currency in 1988. When | worked as a consultant to Cuadra Associates, she invited me to several demonstrations of STAR software to potential clients who helped me think through the requirements of automated records management and archival applications. As a teacher of online searching, she helped me understand the power of this technology and, when | wrote this book, as Manager of Research Systems at the Washington Post, she helped me understand the application of searching technologies in an informationintensive environment. In her current position at the US News and World Report she continues to make practical use of technologies described here to make information accessible to those who need it.

Herb Schantz introduced me to the notions of workflow and showed its integral relationship with imaging technology and its relationship to document management. Dr. Letty Schantz did a careful editing job of the first edition that was particularly helpful. Herb and | went on to collaborate on a book on Document Management and our professional lives have been intertwined ever since.

In addition, a large number of students, friends and colleagues made comments on various drafts of the book. These include (but are not limited to) Clay Cochrane, Fred Jordan, Deanna Marcum, Bob Nawrocki, Fred Stielow and Betsy Woods (who also assisted with the indexing). Finally, | must thank my partner of twenty years, Lawrence Tan, for his patience, understanding and support. As a businessman and accountant, he tried to make this philosopher as practical as he could make him.

For the second edition, in addition to those mentioned above, several of whom took the time to read the revised copy, | am gratefully to Deb Marshall, Marilyn Barth, Dean Harris, Rebecca Weiner, Evie Lotze, Bill Larsen, John Latham and Susan Brown for their assistance.

Kenneth Megill Knowledge Application Services Washington, D.C.

Chapter One

The information revolution (i.e. making it possible for new things to happen), at its best is about empowerment. Like all empowering forces, information is a tool that can be used for many purposes. It can enslave people. If information is controlled and shaped, people can be made to act in what seem to be strange ways: they buy new and useless products; they vote against what appears to be their best interests; they believe what appears to be false. Yet information is also an empowering force: it enables people to know and understand the situation they are in. We are awash with information and are faced with a situation in which there is too much to know and so little understanding.

This book is about managing information in the knowledge age. The technology that enables the knowledge age is the computer, a tool that is based on a simple choice: is something on or off? Is it yes or no? The genius of the computer is that it enables us to compute rapidly. It takes away much of what we used to do — counting and calculating. It gives us the possibility of understanding because it takes away many of the tasks that we once thought were so important and makes them routine. It makes it possible for work to focus on making knowledge — justified true belief — the answer to the question, "Is it a good idea to . . .?"

As the technology develops, the possibility for liberation and empowerment is built. The change in the way in which information is produced, used, and managed is profoundly changing the nature of the organizations in which we work.

Information as a resource

Information has always been important: those who knew what was going on had power over those who did not. The chancellor of the royal court was often the person who was able to read, and functioned as the keeper of the Seal. This person controlled information by deciding what would become an official document and what would not. Information was power and the ability to know was power.

Now, however, we think of information as not just an important tool, but as a resource to be managed ? In the electronic age information is an asset that can be managed like other assets. Money, people, land and natural resources have long been assets. If you had money, you had power. If you had people you could dominate the land. If you had territory and lived in a land of milk and honey you had power. Your assets enabled you to have power, and power was the ability to use and manage assets. In the Information Age, information was an asset and therefore power. It empowered corporations with it to do things that corporations without it could not do.

Information is now generally recognized as an asset. Like other resources, information can be created, stored, kept and used. It can be sold and traded.

And it can be used and reused.

Using information is a little different from using other resources. When land is used for one purpose, it cannot normally be used for another. When personnel resources are being used to doing one thing, then they cannot usually do something else. When money is taken from your bank account to pay a bill, there is less in the account. Most resources are diminished when they are used.

Information, however, is different. Shared information is not lost. In fact, information often loses value if it is not used — by the owner/creator and by others. When information is shared and put into context, it often gains value for the creator as well as for the person with whom it is shared.

Information is not only not a depletable resource, it is one that grows and thrives with use.

? See Megill, Kenneth A, et.al. Making the Information Revolution. A Handbook for Federal Information Resource Managers. Washington: Association for Information and Image Management |996.

The demise of command structures

Corporations are, by and large, established in order to manage scarce resources. They operate from the top down in a structure in which one person commands and allocates resources. Control of the budget, the ultimate resource of the corporation, means power.

The strain on the modern corporation created by the impact of information technology is just beginning. Like any technology, information technology holds within it the possibility for enslavement and control as well as liberation and empowerment. Under a social order, characterized by hierarchical and command-driven organizations, information is used as a means of control.

Information cannot be managed like money or other resources since information grows and prospers in an environment in which it is shared? used and reused, the better an organization is able to share its information, the more valuable that information becomes. The more information can be reused, the greater its value becomes, for information gains value in context with other information.

The memory of an organization is one of its greatest assets and one that the traditional hierarchical organization is badly equipped to manage. In order to utilize information to the fullest, it needs to be shared and dispersed, not hoarded and doled out. Like other resources, information needs to be organized so that it can be retrieved and used. Unlike other resources, it is not diminished with use.

In order to manage information resources effectively, a new kind of corporation is developing. The traditional organizational structure based on command from the top and obedience at the bottom is transforming into one that rewards and encourages ingenuity and openness. The ability

"Collaboration is a creative process that emerges out of a common desire to achieve a goal. Collaboration takes place when we all play the same game, with the same goals and a common understanding of what the game is about."

Sharing is used in the sense of collaboration, which is a two-way street. Sharing is not synonymous with a one-way giving away of information.

to analyze becomes more important than the ability to follow directions. Information can be effectively reused if people are empowered. They need to have access to information, which must be organized so that it is available and can be retrieved quickly and easily. Not only must workers be empowered to get at specific pieces of information, they need to be able to browse through the corporate memory in order to understand a subject.

Recycling

Recycling is a part of everyday life. We know that if we are to survive as a planet and as a society, then we need to manage our environment with care. We need to replenish our resources, not simply deplete them. Information is one of our most valuable resources, yet it is generally created over and over again, at great cost to those who make up the ranks of information workers. Just as the rise of mass production made it possible to produce farm products and industrial goods more efficiently, so the invention and mass application of computing power makes it possible for the management of information to be more effective and efficient.

The management of corporate memory is a task of recycling information. It requires the identification of information that is created for one purpose but may also be used for another. It requires seeing information as not just a product of a particular department or business unit of an organization, but as a product that belongs to the entire organization. Instead of being the property of the person or department that created it, information belongs to the corporation — the organization as a whole — and needs to be managed as one of the organization's valuable resources.

Browsing and retrieving

Too often information is viewed as simply an item that can be picked up and put into another place. The true value of information, however, is often found in the context in which it is created and used. Taking it out of that context can sometimes destroy its value, whereas putting it in another context can often enhance its value. Often the most valuable information is found in unexpected places. A properly managed corporate memory system enables us to discover the unexpected.

The information worker needs the power and the ability to browse through the corporate memory in order to see what information might be valuable in unexpected ways. Browsing can only be done by someone who is familiar with a subject and is seeking to expand his or her understanding.

The classic act of browsing is standing in front of a shelf full of books and seeing what else there is about the subject that might be interesting. Such browsing is possible because library books are arranged together with others on the same topic so that users can not only find the specific book that is required, but also expand their understanding of the topic being investigated. That browsing pales when we have corporate memory systems with retrieval systems that deliver us the unexpected.

A new soul

The first stage of the information revolution, the increasing of production and the flattening of organizations, is well under way. The groundwork for the next stage is developing. This is the stage of liberation and empowerment. The soul of the new corporation will emerge when the information worker can combine knowledge of the business with the ability to make decisions. The mediating power of middle management will no longer be needed in the new corporation. The function of these managers to receive, process and pass on information, will become increasingly irrelevant as information technology develops. The ability to make decisions will belong to those meeting customer needs, and the manager will become a liberator and coordinator.

The Knowledge Age Work in the Age of Knowledge is . . . largely the production of knowledge. As we enter this Knowledge Age, our work takes on very different characteristics than the work we do in an industrial environment. In this new age, in order for work to be done most efficiently, we will not be paid for our work time, but for what we know and what we get done.

Chapter Two

An organization can no longer rely on traditional ways of keeping its corporate memory. In the past, the memory of an organization was kept in paper documents and in the collective memory of its employees, but these ways are now obsolete. As a result of the information revolution and corporate downsizing the ways of doing business are changing, and today the memory of an organization is increasingly housed in electronic form.

Corporate memory consists of the active and historical information in an organization that is worth sharing, managing and preserving for later reuse. The concept of corporate memory is not limited to commercial organizations: we use the term ‘corporate’ to mean ‘total’ or ‘organizational’.

Businesses, government agencies, educational institutions, nonprofit research institutes and other types of organizations have essential information that should be safeguarded, able to be retrieved on demand and reused. Such organizations may prefer to think in terms of an institutional memory, but the concept is the same.

Corporate and organizational culture is changing in fundamental ways to alter the relationship between staff members and the corporation. Employees seldom spend their entire career with a single organization: They move from place to place. Most corporations no longer prize long service. Organizations undergo fundamental transformations that make employees superfluous and, as a result, the carriers of corporate memory are eliminated.

I2. The memory problem

The corporate memory of a given institution may be embedded in millions of documents, all of which might become endangered by loss or mismanagement. Over the years several attempts have been made to define the quantity of information produced and the effects of increasing storage capacities on managing the memory of an organization.

The proliferation of documents and copies of documents in an organization is a major cause of the loss of corporate memory. An equally important cause is a significant change in corporate management. Instead of one or two changes of leadership in 10 years we are seeing three or four. Instead of one or two crises in the organization, such as a major lawsuit, we are seeing many more. As a result, those who carry the corporate memory of the organization disappear as the tenure of staff and officials shortens.

Today, the availability of shared file servers encourages users to maintain multiple copies or near-copies on their personal computers, in various shared spaces and as attachments to e-mails. The servers become the electronic equivalent of the communal refrigerator, complete with unidentifiable fossils lurking in the corners. Storing a potentially damaging item of information after its useful or legally mandated life may cost only a few pennies or dollars per month, but if the item is subpoenaed and used against the organization, such needless retention can be costly.

The cost of keeping too much corporate memory

In the paper era, when the organization faced a crisis (such as a lawsuit, for example) it often relied on records that were kept in the basement. Retrieval of information stored in the basement depended on the knowledge of staff that had been with the organization for a long time. When the leadership of the organization changed, records managers were often called in to sort out the paper records. Keeping inactive records in the basement in this way worked fairly well because they were on paper and, if found, could be still be read. Also, since records from the same office or related offices were normally stored together, it was often possible to get the information needed even if a particular document was not available or had been destroyed. Mildew or water might have damaged some of the records, but the information needed might be found in others stored nearby.

Now, however, with most information stored in various digital forms (in addition to paper in many cases) the task of piecing together evidence of what happened after the fact is compounded. The volatility and complexity of corporate memory repositories demands a way to manage corporate memory in a digital environment.

Box 2.1 Endangered memory It is difficult to quantify the loss of memory. The first edition of this book, written a decade ago, contains the following attempts to quantify memory. The trends identified here have, no doubt, continued: Eighty percent of corporate electronic information is in documents, as opposed to structured database records.

Estimates of the number of documents produced each year in the United States alone include:

* 100 billion documents with an average of 19 copies of each business document.?

* 320 billion documents per year in the United States alone.’

* 92 billion documents per year.*

The capacities of the typical PC network will grow from 7.2 Gb in 1993 to 4| Gb by 1997. When you consider that the average 5 Gb of network data contain 500,000 files, how will anyone be able to manage the number of files on these networks as they double each year?

Network data storage requirements are increasing at an estimated 60—10096 per year. Given that trend, some analysts predict that the average local area network will require more than 50 GB of storage within the next couple of years.

"The average user has enough trouble creating directories or structuring files into folders, much less remembering later where he or she has put files. The inadequacies of contemporary file systems have never been more apparent, nor the need for powerful document management tools greater."

From a report by Peripheral Strategies, cited in Aardvark (July 1994).

? From a study by the Gartner Group, cited in Modern Office Technology (April 1992).

3 A 1991 report citing Fujitsu data cited in Computer Reseller News (May 20, 1991).