Readings in Databases (转）
Readings in Databases
A list of papers essential to understanding databases and building new data systems.
Basics
The Five-Minute Rule Ten Years Later, and Other Computer Storage Rules of Thumb: This paper (and the original one proposed 10 years earlier) illustrates a quantitative formula to calculate whether a data page should be cached in memory or not. It is a delight to read Jim Gray approach to an array of related problems, e.g. how big should a page size be.
Paxos Made Simple: Paxos is a fault-tolerant distributed consensus protocol. It forms the basis of a wide variety of distributed systems. The idea is simple, but notoriously difficult to understand (perhaps due to the way the original Paxos paper was written).
AlphaSort: A Cache-Sensitive Parallel External Sort
Essentials of Relational Databases
Anatomy of a Database System: Joe Hellerstein's great overview of relational database systems. This essay walks readers through all components essential to relational database systems.
A Relational Model of Data for Large Shared Data Banks: Codd's argument for data independence (from 1970), a fundamental concept in relational databases. Despite the current NoSQL trend, I believe ideas from this paper are becoming increasingly important in massively parallel data systems.
ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging: The first algorithm that actually works: it supports concurrent execution of transactions without losing data even in the presence of failures. This paper is very hard to read because it mixes a lot of low level details in the explanation of the high level algorithm. Perhaps try understand ARIES (log recovery) by reading a database textbook before attempting to read this paper.
Efficient Locking for Concurrent Operations on B-Trees and The R*-tree: An Efficient and Robust Access Method for Points and Rectangles: B-Tree is a core data structure in databases (not just relational). It is optimized and has a low read amplification factor for random lookups of on-disk data. R-tree is an extension of B-tree to support lookups of multi-dimensional data, e.g. geodata.
On Optimistic Methods for Concurrency Control: There are two ways to support concurrency. The first is the pessimistic way, i.e. to lock shared data preemptively. This paper explains an alternatively to locking called Optimistic Concurrency Control. Optimistic approaches assume conflicts are rare and executes transactions without acquiring locks. Before committing the transactions, the database system checks for conflicts and aborts/restarts transactions if conflicts arise.
Access Path Selection in a Relational Database Management System: The basics of query optimization. SQL is declarative, i.e. you specify using a query language what data you want, not how you want it. There are usually multiple ways (query plans) of executing a query. The database system examines multiple plans and decides on an optimal one (best-effort). This process is called query optimization. The traditional way of doing query optimization is to have a cost-model for different access methods and query plans. This paper explains the cost-model and a dynamic programming algorithm to pick the best plan.
Eddies: Continuously Adaptive Query Processing: Traditional query optimization (and the cost model used) is static. There are two problems with the traditional model. First, it is hard to build the cost model absent of data statistics. Second, query execution environment might change in long running queries and a static approach cannot capture the change. Analogous to fluid dynamics, this paper proposes a set of techniques that optimize query execution dynamically. I don't think ideas in Eddies have made their way into commercial systems yet, but the paper is very refreshing to read and might become more important now.
Systems Design
A History and Evaluation of System R: There were System R from IBM and Ingres from Berkeley, two systems that showed relational database was feasible. This paper describes System R. It is impressive and scary to note that the internals of relational database systems in 2012 look a lot like System R in 1981.
C-Store: A Column-oriented DBMS: Describes the columnar database system optimized for analytical workloads. C-Store consists of two components: a read-optimized store and a (in-memory) writeable store optimized for updates.
The Google File System and Bigtable: A Distributed Storage System for Structured Data: Two core components of Google's data infrastructure. GFS is an append-only distributed file system for large sequential reads (data-intensive applications). BigTable is high-performance distributed data store that builds on GFS. One way to think about it is that GFS is optimized for high throughput, and BigTable explains how to build a low-latency data store on top of GFS. Some of these might have been replaced by newer proprietary technologies internal to Google, but the ideas stand.
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications and Dynamo: Amazon’s Highly Available Key-value Store: Chord was born in the days when distributed hash tables was a hot research. It does one thing, and does it really well: how to look up the location of a key in a completely distributed setting (peer-to-peer) using consistent hashing. The Dynamo paper explains how to build a distributed key-value store using Chord. Note some design decisions change from Chord to Dynamo, e.g. finger table O(logN) vs O(N), because in Dynamo's case, Amazon has more control over nodes in a data center, while Chord assumes peer-to-peer nodes in wide area networks.
The New Paradigm
MapReduce: Simplified Data Processing on Large Clusters: MapReduce is both a programming model (borrowed from an old concept in functional programming) and a system at Google for distributed data-intensive computation. The programming model is so simple yet expressive enough to capture a wide range of programming needs. The system, coupled with the model, is fault-tolerant and scalable. It is probably fair to say that half of the academia are now working on problems heavily influenced by MapReduce.
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing: This is the research paper behind the Spark cluster computing project at Berkeley. Spark exposes a distributed memory abstraction called RDD, which is an immutable collection of records distributed across a cluster's memory. RDDs can be transformed using MapReduce style computations. The RDD abstraction can be orders of magnitude more efficient for workloads that exhibit strong temporal locality, e.g. query processing and iterative machine learning. Spark is an example of why it is important to separate the MapReduce programming model from its execution engine.
A View of Cloud Computing: This paper discusses the economics and obstacles of cloud computing (referring to the elasticity of resources, not the consumer-facing "cloud") from a technical perspective. The obstacles presented in this paper will impact design decisions for systems running in the cloud.
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines: Google's Luiz André Barroso and Urs Hölzle explains the basics of data center hardware and software for warehouse-scale computing. There is an accompanying video. The video talks about the importance of cutting long-tail latency in massively parallel systems. The other key idea is the disaggregation of resources. Technologies such as GFS/HDFS already disaggregate disks because of high network bandwidth, but yet to see the same trend applying to DRAMs because that'd require low-latency networking.
CAP Twelve Years Later: How the "Rules" Have Changed: The CAP theorem, proposed by Eric Brewer, asserts that any networked shared-data system can have only two of three desirable properties: Consistency, Availability, and Partition-Tolerance. A number of NoSQL stores reference CAP to justify their decision to sacrifice consistency. This is Eric Brewer's writeup on CAP in retrospective, explaining "'2 of 3' formulation was always misleading because it tended to oversimplify the tensions among properties."
New Hardware
Performance Modeling and Analysis of Flash-based Storage Devices
Miscellaneous
Reflections on Trusting Trust: Ken Thompson's Turing Award acceptance speech in 1984, describing black box backdoor issues and pointing out trust is not absolute.
Database Reading Lists
A number of schools have their own reading lists for graduate students in databases.
- Berkeley: http://www.eecs.berkeley.edu/GradAffairs/CS/Prelims/db.html
- Brown: http://www.cs.brown.edu/courses/cs227/papers.html
- Stanford: http://infolab.stanford.edu/db_pages/infoqual.html
- Wisconsin: http://www.cs.wisc.edu/sites/default/files/db.reading.pdf
This list is maintained by rxin@cs.berkeley.edu (@rxin)
Theme modified from orderedlist
