ID 3.2.1 Threats to data integrity from use of large-scale data management environments

Contents:

  • Scope
  • Executive summary
  • 1. Assessing the risks to data integrity when preserving and accessing AV files
  • 2. Example Risk Analysis focussing on data integrity
  • 3. Failure modes and data corruption from mass storage technology
  • 4. Errors for video, audio and images and their detection
  • 5. File Quality-control in current use
  • Annex 1: OCTAVE Allegro Worksheets
  • Annex 2: OCTAVE Allegro Risk Analysis Example
  • Annex 3: Quality Control case studies.

This report investigates threats to audiovisual files from mass storage technologies, reviews file corruption and presents a ‘risk of loss’ model for assessing the threats. The first section presents an overview of digital preservation and security industry risk analysis tools, concluding with a model combining DRAMBORA and OCTAVE Allegro elements. Section two illustrates the model applied in an av context. Section three concerns file corruption and mass storage: where and what type of errors occur, current storage trends and the risks with HDD and data tape in particular. The use of compression on av files is detailed as well. The fourth section presents some real case data integrity and error detection examples experienced by the ORF archive (Oesterreichischer Randfunk). The report concludes with a brief look at four broadcast archive (INA, ORF, BBC and B&G) approaches to quality control, concluding that no norm exists among them.

This report’s conclusions are clearly stated and can be acted on. The risk analysis template is especially helpful for digital content and ICT managers. It makes a good case about the need for av archives to share real case data concerning errors and failure rates in order to fully enable address this problem.