Amazon Aurora: On Avoiding Distributed Consensus for I/Os, Commits, and Membership Changes

ABSTRACT

Amazon Aurora is a high-throughput cloud-native relational database offered as part of Amazon Web Services (AWS). One of the more novel differences between Aurora and other relational databases is how it pushes redo processing to a multi-tenant scale-out storage service, purpose-built for Aurora. Doing so reduces networking traffic, avoids checkpoints and crash recovery, enables failovers to replicas without loss of data, and enables fault-tolerant storage that heals without database involvement. Traditional implementations that leverage distributed storage would use distributed consensus algorithms for commits, reads, replication, and membership changes and amplify cost of underlying storage. In this paper, we describe how Aurora avoids distributed consensus under most circumstances by establishing invariants and leveraging local transient state. Doing so improves performance, reduces variability, and lowers costs.

References

1. David F. Bacon, Nathan Bales, Nico Bruno, Brian F. Cooper, Adam Dickinson, Andrew Fikes, Campbell Fraser, Andrey Gubarev, Milind Joshi, Eugene Kogan, Alexander Lloyd, Sergey Melnik, Rajesh Rao, David Shue, Christopher Taylor, Marcel van der Holst, and Dale Woodford. 2017. Spanner: Becoming a SQL System. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD '17). ACM, New York, NY, USA, 331--343.
2. Larry Carpenter, Joseph Meeks, Charles Kim, Bill Burke, Sonya Carothers, Joydip Kundu, Michael Smith, and Nitin Vengurlekar. 2009. Oracle Data Guard 11G Handbook (bibinfoedition1 ed.). McGraw-Hill, Inc., New York, NY, USA.
3. David K. Gifford. 1979. Weighted Voting for Replicated Data. In Proceedings of the Seventh ACM Symposium on Operating Systems Principles (SOSP '79). ACM, New York, NY, USA, 150--162.
4. Leslie Lamport. 1998. The Part-time Parliament. ACM Trans. Comput. Syst. Vol. 16, 2 (May . 1998), 133--169.

Index Terms

• Computer systems organization

  • Dependable and fault-tolerant systems and networks

    • Availability

• Information systems

  • Data management systems

    • Database management system engines

      • Database transaction processing

        • Database recovery

        • Transaction logging

      • Distributed database transactions

      • Parallel and distributed DBMSs

        • Relational parallel and distributed DBMSs

  • Information storage systems

    • Storage architectures

      • Cloud based storage

      • Distributed storage