Issues And Solutions

There are a multitude of issues associated with recordkeeping in the current climate. A number of the major issues are highlighted in this Wiki to deliver a glimpse of the problematic nature of compliant recordkeeping. It is in no way an exhaustive list and no doubt further issues will become evident with the passing of time.

Advancing Technology

Rapid advances in technology result in recordkeeping systems enjoying a relatively short life cycle, usually between 5-10 years (Brogan, 2010). Signs that a system may need replacing are;

  • maintenance requirements and/or operational costs are steadily increasing,
  • users report unsatisfactory data retrieval and users request additional features not supported by the current system.

The System Development Life Cycle (SDLC) describes the process of planning, analysis, design, implementation and support of the new system until it is superseded once again. See the National Archives of Australia's Records Management Manual).

Implementing a new system can be very costly and time consuming for organisations. There are numerous considerations that need to be factored into the design. For example, a new system would need to be scalable, i.e. it would need the ability to be scaled up (if the business needs increased) or down(in the case of economic decline), it would need to be interoperable so that any data can be read and understood by any new systems implemented in the future.

Due to new and emerging products, companies now have the option to outsource the project and purchase a product built and sometimes maintained by an external company or they may even ‘rent’ software from an external company, called Software as a Service (SaaS) or Cloud Computing.

“Software-as-a-service (SaaS), and a growing array of Cloud Computing solutions, is one of the fastest growing segments of the information technology (IT) industry because it provides a more cost-effective alternative for enterprises to achieve their business objectives than traditional packaged applications. SaaS solutions are delivered via a network, most often the Web. They are priced on a subscription service basis, often based on the number of users or seats.”

What business trends are fuelling the growth of SaaS?

A combination of macro-trends are driving companies of all sizes to consider SaaS alternatives to traditional, on-premise software applications to better achieve their corporate objectives. These market trends include,


• Changing competitive forces
• Changing workplace requirements
• Changing economic and ecological conditions” (THINKstrategies, Inc. - 2005-2009)

What is the most common reason for a company to choose to develop its own information system?

The main reason a company may develop its own system is Total Cost of Ownership (TCO). In considering whether to outsource or build in-house a company should consider both direct and indirect costs. The initial costs of the new system (direct cost), may seem viable but after considering indirect costs such as ongoing support, delay in response to technical issues , productivity losses when the system is encountering problems and is offline, it may actually be more expensive than designing in-house, (Shelly & Rosenblatt, 2010). The organisation may feel it will have the competitive edge by designing in-house by having greater control of the potentially costly variables by utilising existing IT staff to provide prompt, comprehensive support and thereby minimising downtime losses.

What reasons are there for a company to choose the in-house approach?

1. They may have no choice as there may not be a product available to meet their specific business needs. A large number of outsourced products are horizontal, in that they are generic products able to accommodate a wide variety of applications. However, if a company has specific requirements that cannot be met by any commercially available product then the company has no choice but to build the system in-house. Although there are vertical products, designed for a specific business requirement, the company may not feel this is a cost effective choice (see TCO above). A vertical system designed to meet specific business requirements may also lack the ongoing support necessary for seamless implementation and ongoing maintenance.

2. System incompatibility. If it is critical that the new system interface with the old system, and there is not a commercial product available which can do this, then the company has no choice but to build in-house.


In today's 'Information Society', computer breaches, identity theft, hackers and viruses are common place and attacks are becoming more sophisticated and damaging. In the real world absolute security is an unrealistic goal (Shelly & Rosenblatt, 2010) Below is a list of security threats with realistic examples of each.


There are numerous reasons for a system to encounter an attack. Shelly & Rosenblatt illustrate the different types of security attacks and the level of skill set required for each attack.

So what are the main types of Security risks associated with recordkeeping systems?

1. Physical Security
2. Network Security
3. Application Security
4. Procedural Security
5. File Security
6. User Security

What threats and solutions are associated with these security risks?


Keystroke logger - A malicious program that records keystrokes. Often, a keylogger also keeps track of visited web sites, and time-stamps when you visit, as well as what keys you type. More advanced keyloggers can limit their recordings to certain prompts / fields (like the user-name and password boxes) and for only 'secure' web sites (like you bank or EBay etc.) Since the keylogger records information as you type it, as opposed to intercepting it when you submit information on a web page, the information is unencrypted even from secure web sites. Keyloggers are one of the most dangerous forms of malware (due to identity theft), and therefore it is recommended that if you find one, immediately unplug your ethernet / modem cable, back up your important files, and reformat, (University at Albany, 2010)

Security hole – a weak area in a software system by which an attacker can gain access to an operating system and install malicious software.


Malware - Malicious software (mal-ware) is a form of computer program designed with malicious intent. This intent may be to cause annoying pop-up ads with the hope you click on one and generate revenue, or forms of spyware and viruses that can be used to steal your identity or track your activities, (University at Albany, 2010)

Denial of service (DOS) - An attack on a TCP/IP-based network that is designed to bring the network and/or access to a particular TCP/IP host/server to its knees by flooding it with useless traffic, (Gregg & Kim, 2005).

Privilege escalation attack – an unauthorized attempt to raise user permission levels on a compromised user account, commonly due to poor habits and practices of an authorized user.

Intellectual property — Theft of customer data, customer databases, application data, application databases, information, and data assets. Intellectual property may have an intrinsic value to an organization depending on what the intellectual property is and whether the organization generates revenue from it.

Social engineering and Pretexting – this is the method by which an attacker will contact a member (or several members) of an organization and attempt to acquire information that will enable access to the operating system, such a generation of a new password. “Pretexting is the practice of getting your personal information under false pretenses. Pretexters sell your information to people who may use it to get credit in your name, steal your assets, or to investigate or sue you. Pretexting is against the law, (Federal Trade Commission, 2006).

Dumpster Diving – attackers raid trash to acquire sensitive information and passwords.


Physical Security – restricting physical access to data.

  • Operations center security – restricted access to physical location of servers
  • Biometric scanning systems –e.g. retina and fingerprint scans

Servers and desktop computers

  • Tamper-evident cases
  • BIOS-level password
  • Boot-level password
  • Power-on password
  • Uninterruptible power supply (UPS)

Notebook computers

  • Select an operating system that allows secure logons and BIOS-level passwords
  • Mark or engrave the computer’s case
  • Consider notebook models that have a built-in fingerprint reader
  • Back up all vital data
  • Use tracking software
  • Establish stringent password protection policies

Network Security

Ports and services

  • Port scans


  • Firewalls can be configured to detect and respond to denial-of-service attacks, port scans, and other suspicious activity
  • Network intrusion detection – network intrusion detection system (NIDS)

Application Security

  • Hardening – reducing unnecessary areas of an operating system thereby reducing the options for attackers to gain access to the system
  • Application permissions
  • Administrator – enhanced permissions
  • User rights - apply permissions to control access to programs
  • Input validation- to protect the integrity of the data by designing the interface so that only certain terms can be inputted, thereby reducing the risks of input errors.

Patches and updates

  • Patch - A small piece of software or code snippet that the vendor or developer of the software typically releases as software updates, software maintenance, and known software vulnerabilities or weaknesses, (Gregg & Kim, 2005)
  • Third-party software, usually licensed with automatic update service

File Security

  • Permissions - apply permissions to control access to and manipulation of data
  • Define User Groups
  • Digital Signatures – ensure a document is authentic.

User Security

  • Identity management
  • Password protection
  • User resistance and incorrect use resulting in corruption of data, combated by training and raising awareness together with adequate support, especially when implementing new technology.

Procedural Security or Operational security

  • Ensuring that adequate policies and procedures are in place to maintain integrity of data and best practices are adopted by employees. This is usually the realm of Management whose responsibility it is to foster cultural fit throughout it’s employees.
  • Paper shredders combat dumpster diving attacks

Storage and Retention

As information is: inconsumable, untransferable, indivisible and accumulative, (Combes, 2009), it stands to reason that recordkeeping systems will be faced with the dilemma of an ever increasing quantity of data for a finite area of storage. Advances in technology have resulted in the amount of data stored on a given amount of silicon chip, increasing every 18-24 months (see Moore’s Law). If the objective of recordkeeping is to ensure that all evidence is captured to ensure accountability, then collecting ALL transactions and communications should be practiced….yes?

Consider then, should we keep ALL that information? How are we going to FIND the information when it is needed? What is going to happen to the system when it is FULL?

Obviously recordkeeping systems cannot capture and retain every piece of information created and transmitted within an organisation. So what is the answer?

What should we keep?

Organisational policy will state WHAT should be kept. The Public Records Office of Victoria recommends;

“As a public sector employee……ask yourself

Is it work-related?

– did I make, send or receive it as part of my work?
– did I use it to make a decision or perform an action?
– do I need to act on it?

If the answer to any of the above is yes,
make a record!” (Public Record Office Victoria, n.d.)

Even applying this methodology, there will still be a huge quantity of data collected over the years. Is there then, a time at which we can dispose of records and if so how do we do this? The short answer is, yes!

There are legal requirements for the retention of documents, (State Records Act).

For specific retention times of documents please visit the State Records Office Retention Guidelines:

Financial and Accounting Records
Administrative Records
Human Resources and Management Records

There should be a disposal policy for each organisation which clearly states how long each records should be kept and, if disposal is permitted, how to dispose of the record to satisfy legal requirements. Many documents however, must be permanently available for scrutiny. This poses serious complications for recordkeeping systems in terms of storage capacity. The answer to this dilemma is Archival Databases.

What is Database Archiving?

Database Archiving is the process of removing selected data records from operational databases that are not expected to be referenced again and storing them in an archive data store where they can be retrieved if needed. The whole purpose of archiving is to maintain the data in case it is required for some purpose, while releasing valuable storage space on the operation database.
Database Archiving is part of a larger topic, namely Data Archiving. Data exists in many formats and for many purposes, and only a small percentage of it is actually in a database. Physical documents, electronic documents, computer files and data sets, e-mail, and multimedia files are all examples of data that may reasonably need to be archived at some point to fulfill regulatory, legal, and business requirements.

But each type of data requires different archival processing requirements due to its form and nature. What works to archive e-mail is not sufficient for archiving database data, and so on. In other words, each type of data may need to command its own technology.

So the archival of data from a database (Electronic Document Records Management System (EDRMS)) requires knowledge of, and operation in conjunction with, the mechanisms and interfaces of the EDRMS in order to retain the context of the data and thereby the integrity, authenticity and credibility (Mullins, 2006).

Archival databases may be created and maintained in-house or outsourced to an external organisation(see SaaS).

Freedom of Information, Privacy Act and Redactibility

The Freedom of Information Act 1982 provides for public access to documents kept by Australian Government departments which are less than 30 years old. The Archives Act regulates access to records older than 30 years.

In their annual reports, departments must state the types of information and records they keep. When a member of the public requests information, the department must identify and preserve all relevant sources, including records, until a final decision on the request is made. The Freedom of Information Act also sets out how the department may correct, annotate or update records if a member of the public shows that the information in them is false.

The Privacy Act 1988 aims to prevent the misuse of personal information. It specifies that agencies can use information about individuals only for lawful purposes that relate directly to the role of the agency. Agencies must ensure that the information they keep is secure, accurate, relevant, complete, and not misleading. The Act also gives people a right to see records about themselves.

Records over 30 years old are exempt from the Privacy Act. Access to these records is controlled through the Archives Act.

(National Archives of Australia, 2010).

So here is the problem. How does a EDRMS ensure that it complies with both of the above acts, as they appear to contradict each other. If free access is given to information contained in the EDRMS then the Privacy Act will be breached. The answer to this problem is to ensure that any EDRMS has, in it's core design, redactability.


This is the ability of a system to 'mask' information of a personal or sensitive manner or that which could pose a threat to national security. This is a vital requirement for contemporary Records Management Systems. Permission levels built into the system design would ensure that information is protected from unauthorised access.

Finding it!

So we have now discussed issues relating to changing technology, security, storage and censorship, but how do we locate a single piece of information amongst the huge volume of documents contained in modern EDRMS?

Metadata and Data Dictionaries

Metadata refers to labelling, cataloguing and descriptive information structured in such a way to allow a records management system to be properly searched and processed. Recordkeeping metadata within an EDRMS allows easy retrieval of documents together with allowing context to be added to data and thus maintains the integrity of the information by restricting entry terms. A Data Dictionary is a repository of metadata contained within a system and maps the relationship between data elements, fields, tables and other entities that comprise the data model underlying a database. The State Records of South Australia state that Recordkeeping metadata is required for the following reasons:

•Unique identification of records
•Authentication of records
•Persistence of records content, structure and context
•Accountability, management of records over time in an electronic environment
•Administration of terms and conditions of access and disposal
•Tracking and documenting the use and history of records, including specific recordkeeping and archival processes
•Enabling discovery, retrieval and delivery of records to authorised users, and
•Restricting unauthorised use.

Recordkeeping metadata includes:

• Registration and classification metadata, which gives a record its unique identity in the system and classifies records through a classification scheme
• Content, structure and context metadata, which describes record content such as a title, its structure, who created it, where, when, its relationship with other records and why (i.e. what legislation or regulations required the record to be created)
• Recordkeeping process metadata provides information or evidence about processes a record may have undergone such as viewing, transmitting, transferring custody, accessing, reviewing, sentencing, etc.

(State Records of South Australia)

Electronic Document Discovery and Litigation

Efficient location of records within a system is necessary to satisfy the needs of the Corporation. Metadata and Data dictionaries aid the organisation of data and ensure discoverability. However, a major issue currently facing records and information professionals is the possibility of undertaking Electronic Data Discovery (EDD) due to litigation proceedings. EDD is the “identification, gathering, processing, organisation and production of appropriate electronic information as part of the legal discovery process during litigation”, (Dearstyne, 2006).

A rapid shift towards electronic communication has taken place over the past two decades and has resulted in a variety of new forms of evidence submitted to court. There are obvious issues associated with electronic evidence, such as authenticity and the possibility of spoliation (intentional destruction of a document or an alteration of it that destroys its value as evidence, ( Princeton University, 2010).

Historically the production of evidence to support the defendant would be of a tangible nature, easily retrieved from filing systems without exposing the entire record keeping system and practices to Prosecution Lawyers. In addition, paper resources are static and give a clear representation of the transaction or communication at a set time. They are easily destroyed but not easily altered. Evidence of this nature is permissible in court provided there was no evidence to suggest it had been altered. Currently there is no consistency within the courtroom with regards to permissible electronic evidence. This is because electronic evidence is transient, it is subject to change, as it exists within a system that is also transient. Electronic data is also vulnerable to spoliation and is therefore questionable in evidential value. It is also difficult to destroy completely and is usually recoverable even after deletion from a system. The EDD process is highly technical in nature as it is necessary to discover what evidence existed and in what format, at a given time. This may necessitate the retrieval of deleted records and thereby exposing any attempts to destroy or mask evidence pertinent to prosecution.

The EDD process can be costly and has the potential to disclose Corporate sensitive information and/or Trade Secrets. It is for this reason that numerous organisations decide to settle ‘out of court’ rather than undertake the EDD process.
Numerous software companies have realised the potential of this area of expertise and are offering solutions to the EDD problem. However, as legislation in this area remains vague with regards to recordkeeping standards and guidelines, mainly as a result of rapid technological changes, there still exists the real possibility of incorrect recordkeeping practices resulting in litigation. It is therefore vital for Records and Information Professionals to factor the possibility of EDD into the initial design of their EDRMS to ensure that should the Organisation be faced with Prosecution, they do not need to fear the EDD process.



Email archiving is fast becoming a serious problem within the corporate sector. A large portion of work based communication takes place either via or email or other forms of electronic devices.

It is imperative that organisations maintain accurate, reliable and compliant records of all business transactions. This is not only for their own business needs but also to ensure they are able to provide viable evidence should the organisation be the subject of litigation.

How then does an organisation ensure they capture all relevant emails and still maintain efficient discovery. If 100% capture of emails is employed there will eventually be too much data to be stored on working servers and locating a single piece of data may be time consuming.

If organisations rely on employees to record their emails to the recordkeeping system, they run the risk of potentially losing important evidence of communication through non compliance of recordkeeping practices by its employees. This is a serious breach of Corporate recordkeeping responsibilities and could be costly should the organisation find itself in dispute or the subject of an audit.

So what is the answer?

Contemporary solutions to this problem lie in Email Archiving. This allows 100% capture of any email passing through the corporate server but automatically archives emails after a predetermined period of time thus freeing up valuable space on the server. The working server retains a ‘stub’ of the original records thereby allowing the record to be located in the archive easily and quickly should it be needed by the organisation.

One such solution is offered by CommVault.

CommVault® Solution

Data Archiving: The Business Challenge

The amount of data being stored, administered and protected by organizations is exploding. – and so are costs to manage it. Valuable resources are being diverted to control and administer storage growth. Power and cooling demands are draining budgets. eDiscovery timeframes and compliance SLAs are regularly being overshot. One of the primary reasons for these issues is the continued proliferation of unmanaged, stale or duplicate data.
Fortunately there is a solution. Simpana® software allows you to retain data for longer periods, in a space efficient way, while simplifying the ability to search and retrieve it.

The Simpana® Data Archiving Solution

Simpana® Archive enables enterprise data archiving solutions, storage management and eDiscovery capabilities with solutions for email, collaboration, file system and Network Attached Storage (NAS) data. With built-in tiered storage and multi-platform support including Microsoft Exchange servers, IBM Lotus systems, and Microsoft SharePoint data, comprehensive archive management is simplified. Active migration of network attached storage (NAS), e-mail and file system data reduces primary storage use, extending the life of storage systems and reducing the time required for data protection. CommVault active enterprise data archiving solutions preserve e-mail for compliance and eDiscovery purposes.

Microsoft SharePoint services support end-user productivity with shared files and information. Keeping this information protected and available is a critical requirement for keeping your business productive. Preventing SharePoint databases from growing out of control is a critical requirement for keeping your IT systems efficient and affordable.

CommVault, 2010

News Articles

Recently an interesting situation has been highlighted in the media about legal deposit of culturally important records and archives in the UK.

It appears that the current legislation in the UK, dating from 2003 has not been effective in capturing and preserving a representative number of websites for the future. Websites are gaining increasing recognition as being important and valuable snapshots of our cultural and social history. Just as the National Library does in Australia, the British Library has been archiving websites for a number of years. Recently, it unveiled the culmination of this work — an archive of 6,000 websites it deems culturally significant — called the UK Web Archive. These include the websites of high street shops that fell prey to the recession; and the website for Antony Gormley's Fourth Plinth art installation in Trafalgar Square.
However the team from The British Library currently has to contact the copyright holder of every website it wants to archive and this process has just a 24 percent response rate. Consequently the majority of websites created are lost to future historians.

A proposal that could give select institutions the power to take snapshots of websites without their owners' permission is being considered by the British Government. Civil servants at the Department of Culture, Media and Sport are now processing opinions on whether websites should be archived for future generations without the copyright owner’s permission.

This situation highlights the importance of electronic archiving and the fragility of electronic records such as websites. It also inevitably begs the question : “how are we doing in this area in Australia?”

Images: The Most Evil People in History Forum Trolls Health Records I have a Dream


Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License