MapReduce and parallel DBMSs: friends or foes?

Abstract

MapReduce complements DBMSs since databases are not designed for extract-transform-load tasks, a MapReduce specialty.

References

1. Abadi, D.J., Madden, S.R., and Hachem, N. Column-stores vs. row-stores: How different are they really? In Proceedings of the SIGMOD Conference on Management of Data. ACM Press, New York, 2008
2. Abadi, D.J., Marcus, A., Madden, S.R., and Hollenbach, K. Scalable semantic Web data management using vertical partitioning. In Proceedings of the 33rd International Conference on Very Large Databases, 2007
3. Abadi, D.J. Column-stores for wide and sparse data. In Proceedings of the Conference on Innovative Data Systems Research, 2007.
4. Abouzeid, A., Bajda-Pawlikowski, K., Abadi, D.J. Silberschatz, A., and Rasin, A. HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads. In Proceedings of the Conference on Very Large Databases, 2009
5. Boral, H. et al. Prototyping Bubba, a highly parallel database system. IEEE Transactions on Knowledge and Data Engineering 2, 1 (Mar. 1990), 4--24.
6. Chaiken, R., Jenkins, B., Larson, P., Ramsey, B., Shakib, D., Weaver, S., and Zhou, J. SCOPE: Easy and efficient parallel processing of massive data sets. In Proceedings of the Conference on Very Large Databases, 2008.
7. Dean, J. and Ghemawat, S. MapReduce: Simplified data processing on large clusters. In Proceedings of the Sixth Conference on Operating System Design and Implementation (Berkeley, CA, 2004).
8. DeWitt, D.J. and Gray, J. Parallel database systems: The future of high-performance database systems. Commun. ACM 35, 6 (June 1992), 85--98.
9. DeWitt, D.J., Gerber, R.H., Graefe, G., Heytens, M.L., Kumar, K.B., and Muralikrishna, M. GAMMA: A high-performance dataflow database machine. In Proceedings of the 12th International Conference on Very Large Databases. Morgan Kaufmann Publishers, Inc., 1986, 228--237.
10. Englert, S., Gray, J., Kocher, T., and shah, P. A benchmark of NonStop SQL Release 2 demonstrating near-linear speedup and scaleup on large databases. Sigmetrics Performance Evaluation Review 18, 1 (1990), 1990, 245--246.
11. Fushimi, S., Kitsuregawa, M., and Tanaka, H. An overview of the system software of a parallel relational database machine. In Proceedings of the 12th International Conference on Very Large Databases, Morgan Kaufmann Publishers, Inc., 1986, 209--219.
12. Isard, M., Budiu, M., Yu, Y., Birrell, A., and Fetterly, D. Dryad: Distributed data-parallel programs from sequential building blocks. SIGOPS Operating System Review 41, 3 (2007), 59--72.
13. Monash, C. Some very, very, very large data warehouses. In NetworkWorld.com community blog, May 12, 2009; http://www.networkworld.com/community/node/41777.
14. Monash, C. Cloudera presents the MapReduce bull case. In DBMS2.com blog, Apr. 15, 2009; http://www.dbms2.com/2009/04/15/cloudera-presents-the-mapreduce-bull-case/.
15. Olston, C., Reed, B., Srivastava, U., Kumar, R., and Tomkins, A. Pig Latin: A not-so-foreign language for data processing. In Proceedings of the SIGMOD Conference. ACM Press, new York, 2008, 1099--1110.
16. Patterson, D.A. Technical perspective: The data center is the computer. Commun. ACM 51, 1 (Jan. 2008), 105.
17. Pavlo, A., Paulson, E., Rasin, A., Abadi, D.J., DeWitt, D.J., Madden, S.R., and Stonebraker, M. A comparison of approaches to large-scale data analysis. In Proceedings of the 35th SIGMOD International Conference on Management of Data. ACM Press, new York, 2009, 165--178.
18. Stonebraker, M. and Rowe, L. The design of Postgres. In Proceedings of the SIGMOD Conference, 1986, 340--355.
19. Stonebraker, M. The case for shared nothing. Data Engineering 9 (Mar. 1986), 4--9.
20. Teradata Corp. Database Computer System Manual, Release 1.3. Los Angeles, CA, Feb. 1985.
21. Thusoo, A. et al. Hive: A warehousing solution over a Map-Reduce framework. In Proceedings of the Conference on Very Large Databases, 2009, 1626--1629.

Index Terms

• Hardware

  • Communication hardware, interfaces and storage

• Information systems

  • Data management systems

    • Data structures

  • Information storage systems

    • Record storage systems

    • Storage management

• Networks

• Software and its engineering

  • Software notations and tools

    • General programming languages

      • Language features

        • Data types and structures

  • Software organization and properties

    • Contextual software domains

      • Operating systems

        • File systems management

        • Memory management

    • Extra-functional properties

      • Interoperability

    • Software system structures

      • Software architectures

• Theory of computation

  • Design and analysis of algorithms

    • Data structures design and analysis

  • Theory and algorithms for application domains

    • Database theory

      • Data structures and algorithms for data management