Computing the acl involving large groups is costly. So we had better keep them in the acl cache. Some of the reason are:
creating acl cache for large group is costly a group is evicted out of the acl if its entry pointer changed. (i.e that the group was removed from the entry cache then reloaded)
If values are not already sorted, the time spent to sort the membership values is significant.
A way to improvie thing is to try to keep the large group entries in the entry cache as long as possible.
New parameters in the backend config entry
| Parameter | Type | Default value | Description | Comments |
|---|---|---|---|---|
| nsslapd-cache-preserved-entries | Integer >= 0 | 10 | Number of entries that are preserved from eviction | Should stay reasonably small because pinned entries consume memory that cannot be used for something else |
| nsslapd-cache-debug-pattern | String | NULL | Debug feature allowing to log INFO message when an entry whose dn matches the value is added/removed from the entry cache | Intended for the CI tests and not really usefull for the users |
| Struct | field name | Description |
|---|---|---|
| backentry | ep_weight | Entry weight (load time in microseconds) |
| cache | c_stats(*) | cache_stats statistics |
| cache | c_lrus[2] | LRU queues anchor |
| cache | c_inst | ldbm_instance struct (to access the config parameters) |
| ldbm_instance | cache_preserved_entries | number of entries to preserve |
| ldbm_instance | cache_debug_pattern | nsslapd-cache-debug-pattern value (may be NULL): Regular expression used to log INFO messages in error log when adding/removing entries in cache (if their dn matches) |
| ldbm_instance | cache_debug_re | Compiled version of cache_debug_pattern |
(*) All statistics fields of struct cache are moved in new cache_stats struct
(**) Using the generic word weight instead of loadingtime because its meaning may change
struct cache_stats
| Field | Description |
|---|---|
| uint64_t hits | for analysis of hits/misses |
| uint64_t tries | for analysis of hits/misses |
| uint64_t nentries | current # entries in cache |
| int64_t maxentries | max entries allowed (-1: no limit) |
| uint64_t size | current size in bytes |
| uint64_t maxsize | max size in bytes |
| uint64_t weight | Total weight of all entries |
| uint64_t nehw | current # entries having weight in cache |
dirsrvtests/tests/suites/features/entrycache_eviction_test.py verifying pinned-entry caching and eviction thresholds.
These alternatives have been implemented and rejected because of the tests results:
Entry weight is the elapsed time (in microseconds) while loading the entry in the cache. Have a configurable factor While evicting the entries from the cache keep all entry whose loading time is higher than the configurable factor multiplied by the average loading time. (The idea behind that is that if the entry is long to load we want to try to keep it) We can store (in the backentry) the time needed to compute an entry when reading it if not in the cache (and probably also when adding it in the cache too) (Simply by getting current time when getting the original backentry then just before replacing it.) and compute statistics like the average time (by keeping the number of entry (which is already done) and the total time of all the entries that are in the cache) and determine a threshold (tunable ?) about when we keep an entry in the cache IMHO a 100 or 1000 factor will do . Comparing average time to entry time has an advantage: big group entries get naturally evicted if too many of them are in the cache (because in such case, the average building time will then increase until some of the protected entry are no more protected from eviction)
But after the first tests I discovered that it is pretty hard to determine the right value for the threshold to get reliable results. The value is between 1.2 and 1.5
While evicting the entries:
build an array of preserved entries.
while walking the entry to evict (stating from the queue tail):
insert the costly entry in the array if neeeded. At the end put back the remainding entries in the head of LRU (which tend to sort the LRU queue by putting the hight weight around the head (and avoid trying to evict them)
Behavior is more predictable than first version but moving the entries has some drawback:
some entries that should be eveicted are no more evicted (i.e in practice more than n entries get preserved
My first trials were done using the elapsed time while loading the entry in micro seconds as weight
Noise caused by the cpu context switch makes too many regular entry having higher load time than the large groups
Some other alternatives has been rejected without being impmlemented
Having a flag for large groups in the backentry and simply skip them.
Cannot prioritize which group will be preserved (between two large groups) and could get in trouble if there are two many large groups. (no more spaces for regular entries)
Using getrusage(RUSAGE_THREAD, \&usage) instead of clock_gettime
Although it provides more accurate data and limit the cpu context switch noise, getrusage is 10 times more costly than clock_gettime.
checking if w > a+y*v would allow to preserve x% of the entries
it seems an overhead especially since we need to compute a square root to do that. maybe checking w2 > a2+y*v2 will do something but I am not convinced
Use the number of members as weight instead of the loading time.
We may perhaps also preserve some other complex entries
Have a parameter holding a list of DN and never put the entries having these dn in the LRU (or skip these DN while evicting entries)
Embedding the handling of the pinned entries within the LRU code as originally intented
Would make more difficult the replacement of LRU by something more efficient like ARC