Allow compression of interior quicklist nodes

Let user set how many nodes to *not* compress.

We can specify a compression "depth" of how many nodes
to leave uncompressed on each end of the quicklist.

Depth 0 = disable compression.
Depth 1 = only leave head/tail uncompressed.
  - (read as: "skip 1 node on each end of the list before compressing")
Depth 2 = leave head, head->next, tail->prev, tail uncompressed.
  - ("skip 2 nodes on each end of the list before compressing")
Depth 3 = Depth 2 + head->next->next + tail->prev->prev
  - ("skip 3 nodes...")
etc.

This also:
  - updates RDB storage to use native quicklist compression (if node is
    already compressed) instead of uncompressing, generating the RDB string,
    then re-compressing the quicklist node.
  - internalizes the "fill" parameter for the quicklist so we don't
    need to pass it to _every_ function.  Now it's just a property of
    the list.
  - allows a runtime-configurable compression option, so we can
    expose a compresion parameter in the configuration file if people
    want to trade slight request-per-second performance for up to 90%+
    memory savings in some situations.
  - updates the quicklist tests to do multiple passes: 200k+ tests now.
This commit is contained in:
Matt Stancliff
2014-12-10 21:26:31 -05:00
parent 5127e39980
commit abdd1414a8
5 changed files with 1451 additions and 962 deletions

View File

@ -307,7 +307,7 @@ void debugCommand(redisClient *c) {
int remaining = sizeof(extra);
quicklist *ql = val->ptr;
double avg = (double)ql->count/ql->len;
int used = snprintf(nextra, remaining, " ql_nodes:%lu", ql->len);
int used = snprintf(nextra, remaining, " ql_nodes:%u", ql->len);
nextra += used;
remaining -= used;
snprintf(nextra, remaining, " ql_avg_node:%.2f", avg);