Java support for data-intensive systems: experiences building the telegraph dataflow system

Abstract

Database system designers have traditionally had trouble with the default services and interfaces provided by operating systems. In recent years, developers and enthusiasts have increasingly promoted Java as a serious platform for building data-intensive servers. Java provides a number of very helpful language features, as well as a full run-time environment reminiscent of a traditional operating system. This combination of features and community support raises the question of whether Java is better or worse at supporting data-intensive server software than a traditional operating system coupled with a weakly-typed language such as C or C++.In this paper, we summarize and discuss our experience building the Telegraph dataflow system in Java. We highlight some of the pleasures of coding with Java, and some of the pains of coding around Java in order to obtain good performance in a data-intensive server. For those issues that were painful, we present concrete suggestions for evolving Java's interfaces to better suit serious software systems development. We believe these experiences can provide insight for other designers to avoid pitfalls we encountered and to decide if Java is a suitable platform for their system.

References

1. IBM alphaWorks Home Page. http://www.alphaworks.ibm.com/, 2001.
2. Java 2 Platform API Specification, JDK 1.4. http://java.sun.com/j2se/1.4/docs, 2001.
3. OptimizeIt Home Page. http://www.optimizeit.com/, 2001.
4. PostgreSQL. http://www.postgresql.com/, 2001.
5. VTune Tool Home Page. http://developer.intel.com/vtune/analyzer/, 2001.
6. R. Avnur and J. M. Hellerstein. Eddies: Continuously Adaptive Query Processing. SIGMOD, 2000.
7. M. K. Bergman. The Deep Web: Surfacing Hidden Value, White Paper. http://www.brightplanet.com/deepcontent/index.asp/, September 2001.
8. B. Bershad, S. Savage, P. Pardyak, E. G. Sirer, D. Becker, M. Fiuczynski, C. Chambers, and S. Eggers. Extensibility, Safety and Performance in the SPIN Operating System. SOSP, 1995.
9. G. Czajkowski, T. Mayr, P. Seshadri, and T. von Eicken. Resource Control for Java Database Extensions. 5th USENIX Conference on Object-Oriented Technologies and Systems, May 1999.
10. O. Doederlein. The Java Performance Report - Part III. http://www.javalobby.org/fr/-html/frm/javalobby/features/-jpr/part3.html, September 2000.
11. M. Godfrey, T. Mayr, P. Seshadri, and T. von Eicken. Secure and Portable Database Extensibility. SIGMOD 1998, pages 390-401, 1998.
12. J. Gray and A. Reuter. Transaction Processing - Concepts and Techniques. Kaufmann, 1993.
13. Hamilton, James. Personal Communication. Feb 2001.
14. A. Heydon and M. Najork. Performance Limitations of the Java Core Libraries. ACM Java Grande, June 1999.
15. Hong, Wei. Personal Communication. Feb 2001.
16. IBM DB2. IBM DB2 Reference Manual. Version 6.1.
17. M. F. Kaashoek, D. R. Engler, G. R. Ganger, H. M. Briceno, R. Hunt, D. Mazieres, T. Pinckney, R. Grimm, J. Jannotti, and K. Mackenzie. Application Performance and Flexibility on Exokernel Systems. SOSP, 1997.
18. D. Kegel. The C10K Problem. http://www.kegel.com/c10k.html, 2001.
19. B. Lampson and D. Redell. Experiences with Processes and Monitors in Mesa. CACM, 1980.
20. C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, and P. Schwarz. ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging. TODS, pages 94-162, 1992.
21. C. Mohan and F. Levine. ARIES/IM: An Efficient and High Concurrency Index Management Method Using Write-Ahead Logging. SIGMOD, pages 371-380, 1992.
22. D. C. Schmidt. Reactor: An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events. Pattern Languages of Programs Conference, August 1994.
23. M. Stonebraker. Operating System Support for Database Management. CACM, 24(7):412-418, 1981.
24. M. Welsh. Non-blocking I/O for Java. http://www.cs.berkeley.edu/ mdw/proj/java-nbio, September 2001.
25. M. Welsh and D. Culler. Jaguar: Enabling Efficient Communication and I/O in Java. Concurrency Practice and Experience, Dec 1999.
26. M. Welsh, D. Culler, and E. Brewer. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. SOSP-18, October 2001.
27. N. Wyatt and A. Carlson. Cloudscape 3.6, A Technical Overview. A Cloudscape White paper, March 2001. see http://www.cloudscape.com/products/whitepapers.jsp.

Index Terms

• Software and its engineering

  • Software notations and tools

    • General programming languages

      • Language types