The Extent of our Care: Archives, Memory and Information Rights

Gigacycle > Information & Guidance  > The Extent of our Care: Archives, Memory and Information Rights

The Extent of our Care: Archives, Memory and Information Rights


Thank you Julie, and many thanks for inviting me to speak today. It is a great pleasure and a privilege to address you this Wednesday morning.

What an honour for my first time in Newcastle to be to convene with my fellow information managers and archivists.

I pay tribute to the British Records Association for supporting this conference.

And what a pivotal time it is for us. In an age where fake news threatens to erode democracy, the reliability of professional record-keepers can counter the expanding glut of misinformation, sensationalism and false fears.

Add to that the Government putting information management back in the spotlight last week by publishing its response to Sir Alex Allan’s review– and you can see just how relevant, how essential, the contribution you in this room make to society is.

We need archivists and information managers to advocate and promote the importance of good, accurate, accessible information management systems. And yet, most people remain flummoxed as to what exactly constitutes information management, records systems, or data protection and information law.

I’m going to talk to you today about my journey to the UK, where I serve as Information Commissioner, as well as my views on information management and the duty for public authorities to create records.

I cannot fix the year in which emerged my passionate belief that proper information management is an essential element in our democratic society – both in the immediate term and over time. But it must have been before I entered academic study because I remember being obsessed with the “ah ha” moments of discoveries by investigative journalists, digging through government records to uncover the “truth” behind a wall of secrecy.

And I have since become a firm believer that we must hold accountable public officials, both elected and appointed, – through the immediate process of FOI and through historical study by scholars.

This means well organised, funded and complete public records in The National Archives and every other institution that oversees the keep-sakes of our collective journey.

Background in archiving

Here’s my story. It begins when I graduated from the first class in the master of archival and information studies at the University of British Columbia.

I won’t tell you precisely when that was – but we did have a mandatory course in “machine readable records” which involved main frame computers and Hollerith punch cards.

This program launched me into a fascinating career. For the next dozen years I worked in local government archives, including a position as City Archivist in Calgary Alberta, first incorporating the records of the 1988 Winter Olympics. With future projects, I loved chronicling the life of that city, and constructing a tangible memory that people could access.

I made the move to Freedom of Information and Privacy rights when FOI laws were first introduced in the Canadian provinces in the 1990s.

For some of my colleagues, my move from archival and cultural work to FOI represented some kind of conversion to a legislative dark side. But it felt natural for me.

Many of the ethical and policy issues archivists face are very much at play in access and privacy work. Professional and considered management of information underpins both professions. And the archivist, like an information commissioner, arbitrates access and interests between the depositor of the records and the researcher or requester.

I then worked as a regulator in the privacy and access area for more than twelve years, across two provincial and one federal jurisdiction.

Aside from how the wording of the laws vary, according to place and time, all Commissioners in democracies implement and defend the same access principles.

As most of you know, my office regulates both the Data Protection Act and Freedom of Information Act. Our job is to regulate both the right to know, and the right of privacy. This dual mandate helps us adjudicate and balance between public interest scrutiny of government with protection of personal privacy.

I strongly believe in access to information by citizens – the right to personal privacy balanced with the fundamental right to know.

And for the public to have access to information, the information needs to be there for them to access.

Records matter and so do the keepers of records – archives preserve and remember the multitudinous paths that brought to us this fraught, confounding, and yes, even joyful contemporary moment. You say archival nerd? I say “archival joy”.

Information Management

The essential value of maintaining records in an organised national archive has long been acknowledged, as Arthur Doughty, Dominion Archivist of Canada, explained in 1924 when he said:

“Of all national assets, archives are the most precious, they are the gifts of one generation to another, and the extent of our care of them marks the extent of our civilisation…

“As a rule the papers of a given generation are seldom required after their reception and primary use; but when all personal touch with that period has ceased, then these records assume a startling importance, for they replace hands that have vanished and lips that are sealed.”

And, whilst the mandate for records and archiving has been around for eons, information management in the digital age presented entirely fresh problems.

Tomorrow’s technology today and the daily data tsunami that follows have made the discipline of information management more challenging than ever.

The explosion of information and communication technologies has increased the availability, mobility and value of data.

As data and information become more mobile, storage needs expand in kind, and its flows blur borders between organisations, government agencies and nations.

As just one recent example of information advocacy done well, The Records Review by Sir Alex Allan issued in August 2014 offers a hopeful blueprint for the future.

His views on the TNA’s programme of Information Management Assessments accord with mine. While IMAs are currently voluntary, they ought to become mandatory in my view, and as soon as possible. I commend and support the TNA in pursuing their work with Information Management Assessments.

While Sir Alex acknowledges that his mandate only asked him to make a brief assessment of the preparations for review and release of digital records under the Public Records Act, I found his observations on digital records very telling and affirming of my experience as a regulator.

Two of many such observations ring true with my years of practice in the field.

Firstly: “The experience of various inquiries which have relied on information since the widespread introduction of digital records has revealed deficiencies in record keeping and in organisation of records”.

I particularly saw this in Canada with provincial government litigation against tobacco companies to recover health care costs.

Secondly, he said: “There is also confusion over what departments should keep and who should be responsible for decisions. Many departments have placed the onus on individuals to transfer records (including both emails and documents) into department filing systems, sometimes with the warning that emails in inboxes over six months old would automatically be deleted.”

I have seen this confusion in British Columbia in many “adequate search” investigations under FOI. Which is the “office of record”?

The Cabinet Office published its response to this review by Sir Alex last week. In it, they acknowledged that “government information management has yet to meet the challenge of the information age”.

Their response identifies the need for a change in the culture of information management. We need to persuade civil servants of the corporate value of their information, by enabling them to engage with the value of information themselves.

The government plans to consider issues raised by the review including a consideration of existing legislation and encouragement for more structured evaluations of information management than the ones that currently exist.

I welcome this response and I look forward to discussing how the ICO can support the work of the Cabinet Office and TNA in pushing for an evolution in information management culture.

The Duty to Create Documents

I’d be pleased to see a review of existing legislation in this area.

The duty to create records in appropriate circumstances, often called the “duty to document” has been on information managers’ and regulators’ minds for a decade or more.

I am talking about a positive duty in law to create records of significant decisions, actions and events. That means records explaining and providing context to why a specific course of action was taken. 

Minutes of important meetings, decisions, that led to policy change and new initiatives.

The Duty to Document, d2d, is fundamental to the process of information management and eventual archival preservation.

Proper documentation, retention and accessibility also nurture reputation and trust.

I can think of few better examples of damage to reputation than an investigation I headed during my time in British Columbia. It involved records which were potentially responsive to an FOI request relating to the subject of murdered and missing women on Highway 16. This road is known as the “Highway of Tears” because of the number of young aboriginal women who disappeared in that area over a 30 year period. Many were later found to have been murdered.

This 2015 investigation led to a resignation, prosecution, and weeks and weeks of negative media coverage about the records management practices of the provincial government.

My office examined government FOI responses and allegations of records destruction and published a report entitled: Access Denied: Records Retention and Disposal practices of the Government of British Columbia.

It dealt with two government departments and the Office of the Premier.

In the case of one Minister’s Office, a former employee provided a whistle blower account of his supervisor’s alleged destruction of emails.

Our investigation used forensics to confirm that an employee had deleted potentially responsive emails to the FOI request about the Highway of Tears – and lied to the Commissioner under oath – an offence he was prosecuted for under the FOI Act. The work also revealed a deeply concerning practice of “triple deletion of emails” among some ministerial staff. 

Triple deleting means first moving an email to the computer system’s “deleted” folder, expunging the email from the folder itself, and then manually overriding a backup that allows the system to recover deleted items for up to 14 days.

We also uncovered troubling records practices in the Office of the Premier – including failure to produce records for important decisions and activities. The Deputy Chief of Staff in the premier’s office had no email records, despite working for the Premier for two years.

Our investigation revealed that many employees assumed emails and other electronic records were impermanent and transitory; and therefore of little value. As we all know – many electronic records are transitory, but some should be retained. I made 11 recommendations in that report and I mention three particularly relevant ones here:

  • Mandatory training in records management across government, including training on what constitutes a transitory record;
  • best practice guidelines to ensure that employees follow correct processes when responding to FOI requests and meet their duty to assist; and
  • changes to information management law including a legislative duty to document key decisions of government agencies, oversight of records management and new sanctions for wilful destruction of records.

The British Columbia government accepted all the recommendations in my report and committed to pass duty to document legislation.

As a result of my experience, I, and a number of fellow regulators from other democratic jurisdictions, advocated for a legislated duty to document. While the exact form of this duty needs to be discussed and debated in each jurisdiction, I think the principle is shiningly clear.

The Future of Information Management in the UK

I closely follow what’s happening in other jurisdictions and have discussed these emergent policy and regulatory challenges with international colleagues. I look forward to speaking about these challenges in more detail at the International Commissioners Conference we will host later this year.

Given that all of us now work digitally, it’s easy to fail to identify and organise evidence of our decisions. Smart phones, instant messaging and social media challenge our ability to ensure records and decisions are documented and preserved.

When FOIA was enacted, technology had already surpassed the law.

What concerns me is that in a society where most of us walk around with our digital workplace in our purses and pockets, we all become our own records managers.

If important texts, emails, instant messaging and tweets aren’t retained, then there’s a greater risk of reliance on oral government, or what the Butler report after the Iraq war called ‘sofa government’.

In a democracy, how does the public hold government accountable and enter into an informed debate if the record is incomplete and unreliable?

Accurate records are necessary to protect citizens’ rights and inform historic study.

The ICO has done some excellent work in this arena.

In 2012 the office ruled that emails sent by then Education Secretary Michael Gove from a private email account were subject to FOI.

And last year the issue hit the headlines when The Sun ran a front page story claiming David Cameron’s aides had been using WhatsApp to discuss the EU Remain campaign outside of official government channels.

Our view is that information held in WhatsApp accounts, is subject to the FOIA if that information relates to the official business of the public body. That includes messages by government officials, advisors, or ministers. With apologies to Marshall McLuhan, in the context of information rights, it’s the message, not the medium that matters.

The ICO takes up this dictum in guidance that we’ve issued to help public bodies deal with FOI requests and private email accounts. Important records captured and communicated by private accounts are easily forgotten or deleted.

Use of private email accounts and instant messaging to conduct government business can frustrate good governance and undermine the public’s right to know.

I see the job of the ICO not only to administer access to information legislation, but to help make sure proper records exist in the first place.

So I will be studying the evidence, as I did throughout my time in Canada, to become fully informed of the scale of challenge in the UK. Staff in my office will conduct policy work in the coming year to help us obtain a deeper understanding in this area.

What exists at the moment is a patchwork of guidance consisting of section 46 of FOIA, the Public Records Act and the Civil Service Code.

The ICO will continue the positive relationship we have with The National Archives and build on the good work they do with information management assessments.

I had the pleasure of meeting with Jeff James and his senior staff at the end of November when we talked about the important issues around identifying and preserving digital information assets and the danger of oral government. We hope to have the opportunity to work with The National Archives in new ways in the future.

Evidence bubbles up sporadically that suggests a potential problem with record keeping by public authorities, but the exact scope of the problem remains unclear. If we, as a community, can develop a comprehensive understanding of the problem, from that vantage we may evaluate where we stand and where we need to be.

We already have the policy but I’m talking about whether or not we need a legislated d2d, with oversight and consequences if people fail to document key decisions.

Organisations should know what they’ve got and where it is, what they’ve done and why. Staff must have time, resources and training for creating and filing records.

Those in positions of leadership must also encourage staff to consider whether some discussions are appropriate for instant messaging or Twitter. But also to foster practices in which, if a decision is made on instant messenger, say, one would take a subsequent step and make a record of the decision elsewhere – cognisant of the fact that the original messages might not provide an adequate corporate record.

It all comes back to the fundamental point of making sure that staff are aware of the need to record their key decisions in an accessible way – providing enough context and detail for that record to be meaningful in the future.

Organisations also need to make sure they have the appropriate technologies going forward to ensure that digital information is properly managed in the future.

That means technologies that can help to organise and search existing digitally stored data, as well as helping with disposition. Skills in digital management of records must be stepped up to meet these needs.

Locating records also becomes more important as greater outsourcing of public services takes place and the audit trail of key decisions can become more fragmented. On this wider point I will be submitting a report to Parliament later this year on why FOIA needs to be extended to cover private organisations delivering public services on behalf of public bodies.

Data Protection

Lastly, we must recognise another key driver for records management – data protection – my other core responsibility. I could easily fill another lecture about this topic. The arrival of the EU General Data Protection Regulation (GDPR) in May 2018 will highlight the importance of effective records management. I talk a lot about the importance of accountability programmes delivering compliance with the GDPR and records management programmes are an important component. The GDPR should help drive the business case for investing in these programmes and new information management technologies.

The GDPR makes it mandatory for Data Protection Officers to be appointed by organisations who are involved in ‘risky’ processing of personal data, I see records management as an important skillset for DPOs.

I also recognise that the GDPR has raised a number of uncertainties for archives about their legal basis for processing records containing personal data. Despite the derogation for archiving purposes, there is more work for the government to do in implementing this in national law. We’ll be providing advice to government about a number of these derogations to ensure they provide protections for individuals.

My office is also very aware of the important role that archives play and how the GDPR can be applied in a way that respects both.


So, even though I ostensibly left my archival work behind me, good information management is still the pillar for my work in both data protection and freedom of information. 

I cannot stress how fundamental it is to citizens’ information rights that we have a working system of creating records and providing access to them.

The responsible management of these records ensures the maintenance of institutional memory, that appropriate information is available to decision-makers, that evidence of a public body’s activities is retained, and that legal requirements are met. Information management is also necessary to meet the goals and requirements of the Freedom of Information Act. 

Access to authentic government records underpins democracy by holding powers to account.

I am committed to do everything I can to improve the state of information management in the UK. Those of you here today also have an important role to play – together we can build a better system.

Thank you again for inviting me to this conference. If we have time for questions I am happy to take them.

No Comments

Sorry, the comment form is closed at this time.