如何实现线程安全的LRU缓存
如何实现线程安全的LRU缓存？
1 不考虑缓存大小
方法1：使用ConcurrentHashMap即可。并发度高。【推荐】
class LRU<K, V> { // 正确：之所以不直接使用ConcurrentHashMap，是因为要减少暴漏的接口
private Map<K, V> cache = new ConcurrentHashMap<>();
public V get(K key) {
return cache.get(key);
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
return cache.put(key, val);
}
}
方法2：使用读写锁+普通的Map，get操作是并发的，put操作是串行化的。并发性能没有方法1好。
class LRU<K, V> {
private Map<K, V> cache = new HashMap<>();
private ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
private Lock readLock = readWriteLock.readLock();
private Lock writeLock = readWriteLock.writeLock();
public V get(K key) {
readLock.lock();
try {
return cache.get(key);
} finally {
readLock.unlock();
}
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
writeLock.lock();
try {
return cache.put(key, val);
} finally {
writeLock.unlock();
}
}
}
方法3：加锁，或使用同步类Hashtable, 或者Collections.synchronizedMap这样性能更差，get和put操作都是串行化了。
// 加锁：
class LRU<K, V> {
private Map<K, V> cache = new HashMap<>();
public synchronized V get(K key) {
return cache.get(key);
}
// return null if key not exists, otherwise return old value
public synchronized V put(K key, V val) {
return cache.put(key, val);
}
}
// 使用同步包装类：
class LRU<K, V> {
private Map<K, V> cache = Collections.synchronizedMap(new HashMap<>());
public V get(K key) {
return cache.get(key);
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
return cache.put(key, val);
}
}
// 使用同步类 Hashtable
class LRU<K, V> {
private Map<K, V> cache = new Hashtable<>();
public V get(K key) {
return cache.get(key);
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
return cache.put(key, val);
}
}
2 考虑缓存大小
Lab 1 直接使用并发容器，没有加锁【错误，非线程安全】
lab1: 直接使用ConcurrentHashMap，没有加锁正确同步，导致出问题。本质是 put操作里面，size()+remove() 不是原子操作，有可能导致多个线程进入到这个if里面，同时都去remove了，导致size小于capacity - 1。多线程put也会导致出问题，size超过了capacity，这是无法接受的。【错误，非线程安全】
public class Exp3WithConcurrent {
static class LRU<K, V> {
private final int capacity;
private Map<K, V> cache = new ConcurrentHashMap<>();
public LRU(int capacity) {
this.capacity = capacity;
}
public V get(K key) {
return cache.get(key);
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
if (cache.containsKey(key)) {
return cache.put(key, val);
}
if (cache.size() >= capacity) {
// random select one
ArrayList<K> keys = new ArrayList<>(cache.keySet());
int i = new Random().nextInt(keys.size());
cache.remove(keys.get(i));
if (cache.size() < capacity - 1) {
System.out.println("concurrent remove cause issue!!");
}
}
V ans = cache.put(key, val);
if (cache.size() > capacity) {
System.out.println("concurrent put cause issue!!");
}
return ans;
}
}
public static void main(String[] args) {
LRU<Integer, String> cache = new LRU<>(3);
Runnable task = () -> {
for (int i = 0; i < 100; i++) {
randomSleep();
String name = Thread.currentThread().getName();
int id = new Random().nextInt(10000);
String ans = cache.put(id, name);
// System.out.println(Thread.currentThread().getName() + " put: " + ans);
}
};
for (int i = 0; i < 1000; i++) {
new Thread(task, "T" + i).start();
}
}
static void randomSleep() {
int time = new Random().nextInt(1000);
try {
System.out.println(Thread.currentThread().getName() + " will sleep " + time + " ms");
TimeUnit.MILLISECONDS.sleep(time);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
运行结果：
线程数量100，然后单个线程put 100次。
T17 will sleep 776 ms
T65 will sleep 737 ms
T77 will sleep 944 ms
concurrent remove cause issue!!
T4 will sleep 942 ms
T56 will sleep 781 ms
T18 will sleep 933 ms
T82 will sleep 17 ms
T30 will sleep 616 ms
T101 will sleep 80 ms
concurrent put cause issue!!
T677 will sleep 605 ms
concurrent put cause issue!!
线程数量1000，然后单个线程put 100次。
T524 will sleep 333 ms
T480 will sleep 294 ms
concurrent remove cause issue!!
T666 will sleep 173 ms
T444 will sleep 963 ms
concurrent put cause issue!!
T889 will sleep 54 ms
concurrent put cause issue!!
T191 will sleep 716 ms
concurrent put cause issue!!
T606 will sleep 134 ms
concurrent put cause issue!!
T417 will sleep 50 ms
Lab 2 对put加锁 【正确，线程安全】
lab2: 对put加锁。能保证正确性，但是扩展的性能很差，put操作变成串行化了。【正确，线程安全】
修改put方法即可：public synchronized V put(K key, V val) {... } 或者使用ReentrantLock。
Lab 3 使用读写锁且考虑访问顺序。写操作互斥 【错误，非线程安全】
lab3: 使用读写锁且考虑访问顺序。写操作都是互斥的了，看起来没有问题。但是LHM实现的LRU，每次get方法会调整内部的链表结果，多线程情况下不是线程安全的。也就是get方法可以多线程执行，然后里面的get操作因为要维护access链表会导致并发问题。【错误，非线程安全】
static class LRU<K, V> {
private final int capacity;
private Map<K, V> cache;
private ReadWriteLock lock;
public LRU(int capacity) {
this.capacity = capacity;
cache = new LinkedHashMap<K, V>(capacity, 0.75f, true) {
protected boolean removeEldestEntry(Map.Entry<K, V> eldestEntry) {
return size() > capacity;
}
};
lock = new ReentrantReadWriteLock();
}
public V get(K key) {
lock.readLock().lock();
try {
return cache.get(key);
} finally {
lock.readLock().unlock();
}
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
lock.writeLock().lock();
try {
V ans = cache.put(key, val);
if (cache.size() > capacity) {
System.out.println("concurrent put cause issue!!");
}
return ans;
} finally {
lock.writeLock().unlock();
}
}
public V remove(K key) {
lock.writeLock().lock();
try {
return cache.remove(key);
} finally {
lock.writeLock().unlock();
}
}
}
观察到很神奇的现象，在get方法里，如果在加读锁之前，判断size() 是否超过 capacity，那么结果是可能会超过的。
public V get(K key) {
if (cache.size() > capacity) {
System.out.println("concurrent put cause over-size issue!!");
}
lock.readLock().lock();
try {
return cache.get(key);
} finally {
lock.readLock().unlock();
}
}
// 下面是添加了并发读的测试代码
public static void main(String[] args) {
LRU<Integer, String> cache = new LRU<>(3);
Runnable task = () -> {
for (int i = 0; i < 100; i++) {
randomSleep();
String name = Thread.currentThread().getName();
int id = new Random().nextInt(10000);
String ans = cache.put(id, name);
// System.out.println(Thread.currentThread().getName() + " put: " + ans);
}
};
for (int i = 0; i < 1000; i++) {
new Thread(task, "T" + i).start();
}
for (int i = 0; i < 10; i++) { // test concurrent read
new Thread(() -> {
for (int j = 0; j < 1000; j++) {
randomSleep();
cache.get(1);
}
}, "R" + i).start();
}
}
static void randomSleep() {
int time = new Random().nextInt(100);
try {
TimeUnit.MILLISECONDS.sleep(time);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
// got some output:
concurrent put cause over-size issue!!
concurrent put cause over-size issue!!
concurrent put cause over-size issue!!
然而如果把检测size()是否超过capacity的代码，放到读锁加锁之后，那么不会出现over-size的问题。这说明什么问题？对于get()操作的线程，只有在获取到read lock之后，意味着这时没有并发的写线程操作map，这样读是正常的。读到的数据就是一致的，size()就不会超过capacity。在get方法执行后，没有获取到lock之前，这时如果去读size()可能会得到一个不一致的结果，因为写线程可能正在做put操作。
public V get(K key) {
lock.readLock().lock();
try {
if (cache.size() > capacity) {
System.out.println("concurrent put cause over-size issue!!");
}
return cache.get(key);
} finally {
lock.readLock().unlock();
}
}
Lab 4 Lock + 并发安全容器类 【正确，线程安全，推荐】【更新：get并发可能还有点问题】
lab4: 采用Lock+并发安全的容器类实现。【正确，线程安全，推荐！】
static class LRU<K, V> {
private final int capacity;
private final Map<K, V> internalCache; // 注意命名
private final Queue<K> trackingQueue; // 按功能命名
private final ReadWriteLock lock;
public LRU(int capacity) {
this.capacity = capacity;
this.internalCache = new ConcurrentHashMap<>(capacity);
this.trackingQueue = new ConcurrentLinkedDeque<>();
this.lock = new ReentrantReadWriteLock();
}
public V get(K key) {
lock.readLock().lock();
try {
V val = internalCache.get(key);
if (val != null) {
// 更新思考：如果两个get(k1), get(k2)同时执行了这个判断，进入if里。那么他们进入Queue的顺序是不确定的。是否有问题？
// 如果严格定义get的顺序是按照从internalCache中取元素的顺序，那么下面这个入队列就有问题了
// KEY POINT: avoid adding twice in multiple thread env for get same key
if (trackingQueue.remove(key)) {
trackingQueue.offer(key);
}
}
return val;
} finally {
lock.readLock().unlock();
}
}
// return null if key not exists, otherwise return old value
public V put(K key, V val) {
lock.writeLock().lock();
try {
V oldVal = internalCache.get(key);
if (oldVal != null) {
internalCache.put(key, val); // update cache entry
trackingQueue.remove(key);
trackingQueue.offer(key);
return oldVal;
} else {
if (internalCache.size() == capacity) {
K deletedKey = trackingQueue.poll();
internalCache.remove(deletedKey);
}
internalCache.put(key, val);
trackingQueue.offer(key);
return null;
}
} finally {
lock.writeLock().unlock();
}
}
public V remove(K key) {
lock.writeLock().lock();
try {
V val = internalCache.remove(key);
trackingQueue.remove(key);
return val;
} finally {
lock.writeLock().unlock();
}
}
}
这个实现参考以下两个视频：
- Code Review: Thread-safe LRU Cache Implementation in Java - YouTube
- Code Review: java Thread-safe LRUCache implementation (3 Solutions!!) - YouTube
- 视频总结：
- rename concurrentHashMap to internalCache 按照容器的用途命名，rename ConcurrentLinkedQueue to trackingQueue.
- 并发容器，可以通过函数的返回值，判断有没有成功操作 (remove/add), 从而避免多线程get(key) 同一个key时，导致多次添加到trackingQueue。
进一步思考, ConcurrentLinkedHashMap?
问题：还有更好的方式？或者并发更好的？锁粒度更细。
参考源码：ConcurrentLinkedHashMap （https://github.com/ben-manes/concurrentlinkedhashmap）
这里提到，LinkedHashMap实现的LRU，底层链表和Map，分开考虑，Map需要同步，每次读总能读到最新的结果。access 链表并不会对外暴露，因此可以考虑缓存。直到下次write，或者缓存超过一个阈值。好处是能避免get操作，获取list的写锁，每次都要修改list。
An alternative approach is to realize that the data structure can be split into two parts: a synchronous view and an asynchronous view. The hash-table must be synchronous from the perspective of the caller so that a read after a write returns the expected value. The list, however, does not have any visible external properties.
This observation allows the operations on the list to be applied lazily by buffering them. This allows threads to avoid needing to acquire the lock and the buffer can be drained in a non-blocking fashion. Instead of incurring lock contention, the penalty of draining the buffer is amortized across threads. This drain must be performed when either the buffer exceeds a threshold size or a write is performed.
网上看到其他人的版本
版本1：网上看的，有问题，并非线程安全，虽然使用了并发容器和锁。
class LRUCache {
class Node {
int key;
int value;
Node next;
Node prev;
public Node(int key, int value) {
this.key = key;
this.value = value;
next = null;
prev = null;
}
}
private ConcurrentHashMap<Integer, Node> map;
private AtomicInteger size;
private int capacity;
private Node head, tail;
private ReentrantLock lock;
public LRUCache(int capacity) {
map = new ConcurrentHashMap<>();
size = new AtomicInteger();
this.capacity = capacity;
head = new Node(0, 0);
tail = new Node(0, 0);
head.prev = tail;
tail.next = head;
lock = new ReentrantLock();
}
public int get(int key) {
if (map.containsKey(key)) {
Node node = map.get(key);
removeNode(key);
addNode(key, node.value);
return node.value;
} else {
return -1;
}
}
public void put(int key, int value) {
if (map.containsKey(key)) {
removeNode(key);
addNode(key, value);
} else {
addNode(key, value);
}
}
// Remove the Node from DLL
private void removeNode(int key) {
lock.lock();
try {
Node node = map.get(key);
node.prev.next = node.next;
node.next.prev = node.prev;
size.decrementAndGet();
map.remove(node.key);
} finally {
lock.unlock();
}
}
// Add the Node at the head of DLL
private void addNode(int key, int value) {
lock.lock();
try {
Node node = new Node(key, value);
node.prev = head.prev;
head.prev = node;
node.prev.next = node;
node.next = head;
map.put(key, node);
size.incrementAndGet();
if (size.get() > capacity) {
removeNode(tail.next.key);
}
} finally {
lock.unlock();
}
}
}
/**
* Your LRUCache object will be instantiated and called as such:
* LRUCache obj = new LRUCache(capacity);
* int param_1 = obj.get(key);
* obj.put(key,value);
*/
From: Thread safe Java Implementation with ConcurrentHashMap and ReentrantLock - LRU Cache - LeetCode
分析：这个版本有问题，get方法不是线程安全的，因为map.containsKey的判断和map.get(key)，以及addNode()并不具有原子性，可能被中断。会导致的情况是，两个线程同时进入if语句，同时执行了removeNode（当然后面执行的可以修改代码，让它比较key再删除，如果已经删了那么就不会再删除），但仍然会导致addNode(key, node.value) 添加两次。同一个key被添加两次，在access list中存在两个相同的key的节点，这应该就是不一致了。
我修改之后的版本：所有方法基本都加锁了，能保证线程安全，但是并发度很低，一次只能允许一个线程操作。这个其实加synchronized关键字也可以，它相比ReentrantLock的优点在于，它有优化（偏向锁、轻量级锁、重量级锁）。java的ConcurrentHashMap的锁，1.7用的是ReentrantLock，1.8就用的是synchronized了。
TODO: 看下还有没有其他的方法提高并发度。
public class LRUCache {
private final Map<Integer, Node> map;
private final int capacity;
private final Node head, tail;
private final ReentrantLock lock;
public LRUCache(int capacity) {
map = new HashMap<>();
this.capacity = capacity;
head = new Node(0, 0);
tail = new Node(0, 0);
head.prev = tail;
tail.next = head;
lock = new ReentrantLock();
}
public int get(int key) {
lock.lock();
try {
if (map.containsKey(key)) {
Node node = map.get(key);
removeNode(key);
addNode(key, node.value);
return node.value;
} else {
return -1;
}
} finally {
lock.unlock();
}
}
public void put(int key, int value) {
lock.lock();
try {
if (map.containsKey(key)) {
removeNode(key);
}
addNode(key, value);
} finally {
lock.unlock();
}
}
// Remove the Node from DLL
private void removeNode(int key) {
Node node = map.get(key);
node.prev.next = node.next;
node.next.prev = node.prev;
node.prev = node.next = null;
map.remove(node.key);
}
// Add the Node at the head of DLL
private void addNode(int key, int value) {
Node node = new Node(key, value);
node.prev = head.prev;
head.prev = node;
node.prev.next = node;
node.next = head;
map.put(key, node);
if (map.size() > capacity) {
removeNode(tail.next.key);
}
}
private static class Node {
final int key;
int value;
Node next;
Node prev;
public Node(int key, int value) {
this.key = key;
this.value = value;
}
}
}
浙公网安备 33010602011771号