EpiChord: Parallelizing the Chord Lookup Algorithm with Reactive Routing State Management

Abstract

EpiChord is a DHT lookup algorithm that demonstrates that we canremove the O(log n)-state-per-node restriction on existing DHTtopologies to achieve significantly better lookup performance andresilience using a novel reactive routing state maintenance strategythat amortizes network maintenance costs into existing lookups and byissuing parallel queries. Our technique allows us to design a newclass of unlimited-state-per-node DHTs that is able to adapt naturallyto a wide range of lookup workloads. EpiChord is able to achieveO(1)-hop lookup performance under lookup-intensive workloads, and atleast O(log n)-hop lookup performance under churn-intensiveworkloads even in the worst case (though it is expected to performbetter on average).Our reactive routing state maintenance strategy allows us to maintainlarge amounts of routing state with only a modest amount of bandwidth,while parallel queries serve to reduce lookup latency and allow us toavoid costly lookup timeouts. In general, EpiChord exploits theinformation gleaned from observing lookup traffic to improve lookupperformance, and only sends network probes when necessary. Nodespopulate their caches mainly from observing network traffic, andcache entries are flushed from the cache after a fixed lifetime.Our simulations show that with our approach can reduce both lookuplatencies and pathlengths by a factor of 3 by issuing only 3 queriesasynchronously in parallel per lookup. Furthermore, we show that weare able to achieve this result with minimal additional communicationoverhead and the number of messages generated per lookup is no morethan that for the corresponding sequential Chord lookup algorithm overa range of lookup workloads. We also present a novel token-passingstabilization scheme that automatically detects and repairs globalrouting inconsistencies.