A mathematical framework for modeling and analyzing migration time

ABSTRACT

File format obsolescence has so far been considered the major risk in long-term storage of digital objects. There are, however, growing indications that file transfer may be a real threat as the migration time, i.e., the time required to migrate Petabytes of data, may easily spend years. However, hardware support is usually limited to 3-4 years and a situation can emerge when a new migration has to be started although the previous one is still not finished yet. This paper chooses a process modeling approach to obtain estimates of upper and lower bounds for the required migration time. The advantage is that information about potential bottlenecks can be acquired. Our theoretical considerations are validated by migration tests at the National Library of Norway (NB) as well as at our department.

References

1. J. F. Gantz, et al., "The Diverse and Exploding Digital Universe: An updated forecast of worldwide information growth through 2011", 2008.
2. The Digital Preservation TestBed, "Migration: Context and Current Status", 2001.
3. K.-H. Lee, et al., "The State of the Art and Practice in Digital Preservation", Journal of Research of the National Institute of Standards and Technology, vol. 107, pp. 93--106, 2002.
4. K. Thibodeau, "Overview of Technological Approaches to Digital Preservation and Challenges in Coming Years", CLIR Reports, Conference Proceedings of The State of Digital Preservation: An International Perspective, 2002.
5. S. Granger, "Emulation as a Digital Preservation Strategy", D-Lib Magazine, vol. 6, 2000.
6. R. A. Lorie, "A methodology and system for preserving digital data", in Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, Portland, Oregon, USA, 2002, pp. 312--319.
7. P. Mellor, et al., "Migration on Request, a Practical Technique for Preservation", in Proceedings of the 6th European Conference on Research and Advanced Technology for Digital Libraries, 2002, pp. 516--526.
8. N. Lormant, et al., "How to evaluate the ability of a file format to ensure long-term preservation for digital information?", in Ensuring Long-term Preservation and Adding Value to Scientific and Technical data, Edinburgh, 2005.
9. S. Strodl, et al., "How to choose a digital preservation strategy: evaluating a preservation planning procedure", in Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries, Vancouver, BC, Canada, 2007, pp. 29--38.
10. J. Hunter and S. Choudhury, "PANIC: an integrated approach to the preservation of composite digital objects using Semantic Web services", International Journal on Digital Libraries, vol. 6, pp. 174--183, 2006.
11. M. Ferreira, et al., "A foundation for automatic digital preservation", Ariadne, vol. 48, 2006.
12. D. Waters and J. Garrett, "Preserving Digital Information. Report of the Task Force on Archiving of Digital Information", 1996.
13. P. Wheatley, "Migration - a CAMiLEON discussion paper", Ariadne, vol. 29, 2001.
14. Consultative committee for Space Data Systems, "Reference Model for an Open Archival Information System (OAIS) ", 2002.
15. A. Crespo and H. Garcia-Molina, "Cost-driven design for archival repositories", in Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, Roanoke, Virginia, United States, 2001, pp. 363--372.
16. T. Mest, K. Nymoen, J. I. Bøyum, L. Gaustad "Survivability of Digital Records", DNV Technical Record 2007-1623.
17. E. Grochowski and R. F. Hoyt, "Future trends in hard disk drives", IEEE Transactions on Magnetics, vol. 32, pp. 1850--1854, 1996.
18. E. Grochowski and R. D. Halem, "Technological impact of magnetic hard disk drives on storage systems", IBM Syst. J., vol. 42, pp. 338--346, 2003.
19. S. Gurumurthi, "Should disks be speed demons or brainiacs?", SIGOPS Oper. Syst. Rev., vol. 41, pp. 33--36, 2007.
20. S. Gurumurthi, et al., "Disk drive roadmap from the thermal perspective: a case for dynamic thermal management", in Computer Architecture, 2005. ISCA '05. Proceedings. 32nd International Symposium on, 2005, pp. 38--49.
21. G. E. Moore, "Progress in digital integrated electronics", in 1975 International Electron Devices Meeting, 1975, pp. 11--13.
22. Web-Feet Research, Inc., "Solid State Drives (SSD) Markets and Applications 2006-2011", 2006, pp. 68--70.
23. M. Ferreira, et al., "An intelligent decision support system for digital preservation", International journal on digital libraries, vol. 6, pp. 295--304, 2007.

Index Terms

• Applied computing

  • Computers in other domains

    • Digital libraries and archives

• Computer systems organization

  • Dependable and fault-tolerant systems and networks

• General and reference

  • Cross-computing tools and techniques

    • Performance

• Information systems

  • Information storage systems

    • Record storage systems

  • Information systems applications

    • Digital libraries and archives

• Networks

  • Network performance evaluation