Because the leases were sorted inside UnsafeLeases() the lessor mutex
ended up being locked while the whole map was sorted. This pulls the
soring outside of the lock, per feedback on
https://github.com/coreos/etcd/pull/9699
Since heap is already sorted, we can just check first element
to see if anything is expiry, rather than popping and pushing
it back. If nothing is expiry, pop operation is unnecessary.
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
This adds a heap acting as a priority queue to keep track of lease
exiprations. Previously the whole lease map had to be iterated through
each time.
The queue allows us to check only those leases which might be expired.
When the expiration changes, we add an additional entry. If we check an
entry that isn't expired, it means that the lease got extended.
If we find a entry in the heap that doesn't have a corresponding entry in
the map, we know that the lease has already been expired or revoked.
The golang/time package tracks monotonic time in each time.Time
returned by time.Now function for Go 1.9.
Use time.Time to measure whether a lease is expired and remove
previous pkg/monotime. Use zero time.Time to mean forever. No
expiration when expiry.IsZero() is true.
Instead of unconditionally randomizing, extend leases on promotion
if too many leases expire within the same time span. If the server
has few leases or spread out expires, there will be no extension.
Randomize the very first expiry on lease recovery
to prevent recovered leases from expiring all at
the same time.
Address https://github.com/coreos/etcd/issues/8096.
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Revoke expects the BatchTx lock to be held when holding the TxnDeleter
because it updates the lease bucket. The tests don't hold the lock so
it may race with the backend commit loop.
Fixes#7662
Demote was racing on expiry when LeaseTimeToLive called Remaining. Replace
with intrinsics since the ordering isn't important, but torn writes are
bad.
suppose a lease granting request from a follower goes through and followed by a lease look up or renewal, the leader might not apply the lease grant request locally. So the leader might not find the lease from the lease look up or renewal request which will result lease not found error. To fix this issue, we force the leader to apply its pending commited index before looking up lease.
FIX#6978
Would retry a few times before returning a not primary error that
the client should never see. Instead, use proper timeouts and
then return a request timeout error on failure.
Fixes#6922
Micah Gates