Commit Graph

8233 Commits

Author SHA1 Message Date
32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
a67d0411e7 Remove experimental warning from diskless replication. 2019-07-24 11:33:53 +02:00
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
47ce1cebeb Merge pull request #6267 from soloestoy/opt-claim-trackingtable
Client side caching: do not reclaim tracking table if it's empty
2019-07-23 09:43:41 +02:00
9012e587ca Client side caching: do not reclaim tracking table if it's empty 2019-07-23 15:25:00 +08:00
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
09c06698e9 Client side caching: redis-cli ability to enable tracking.
This is extremely useful in order to simulate an high load of requests
about different keys, and force Redis to track a lot of informations
about several clients, to simulate real world workloads.
2019-07-22 18:45:47 +02:00
436d02dd7b Move signalFlushedDb() into a better place.
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
2019-07-22 12:37:47 +02:00
7d3992a80a Client side caching: reclaim the tracking table on FLUSHALL. 2019-07-22 12:33:49 +02:00
842b44dc46 Client side caching: call the invalidation functions always.
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
2019-07-22 12:29:54 +02:00
f850201c64 Client side caching: don't hash the key if not needed. 2019-07-22 12:10:51 +02:00
e00442eb13 Clinet side caching: take count of used caching slots. 2019-07-22 11:47:44 +02:00
bc5cb168f5 RDB: fix MODULE_AUX loading by continuing to next opcode.
Thanks to @JohnSully for noticing this problem.
2019-07-19 11:12:46 +02:00
5f450e4928 RDB: make sure to abort on LZF encoding error. 2019-07-18 18:59:38 +02:00
bd0f06c18c RDB: handle encoding errors with rdbExitReportCorruptRDB().
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.
2019-07-18 18:51:45 +02:00
41ed85bd26 Merge pull request #6252 from soloestoy/tracking-flushdb
Tracking flushdb
2019-07-18 16:41:37 +02:00
947319cadd RDB: update rdbLoadRio comment about EOF condition. 2019-07-18 12:37:55 +02:00
f59119aad6 Client side caching: filter clients untracking 2019-07-18 16:46:19 +08:00
42b6305964 RDB: try to make error handling code more readable. 2019-07-17 17:30:02 +02:00
6b72b04a37 Rio: when in error condition avoid doing the operation. 2019-07-17 16:46:22 +02:00
5189db3d81 RDB: modify rdbReportError() var name for clarity. 2019-07-17 16:46:22 +02:00
48d91cf4cc Rio: remember read/write error conditions. 2019-07-17 16:46:22 +02:00
c56b4ddc6f prevent diskless replica from terminating on short read
now that replica can read rdb directly from the socket, it should avoid exiting
on short read and instead try to re-sync.

this commit tries to have minimal effects on non-diskless rdb reading.
and includes a test that tries to trigger this scenario on various read cases.
2019-07-17 16:46:22 +02:00
6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
241d18d954 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-17 12:26:38 +02:00
5827b69861 Merge pull request #6185 from AlexRenCN/unstable
fix readme.md,Redis data types should add `t_stream.c`.
2019-07-16 18:33:55 +02:00
4b2579a064 Merge pull request #5779 from madolson/dev-unstable-geo-bugfix
Fixed a rounding bug in geo.tcl
2019-07-15 23:41:57 +02:00
3f1c84751a Merge pull request #6196 from AngusP/resp3
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
2019-07-12 13:08:46 +02:00
6100a82a3a Merge pull request #5738 from bmerry/fix-swapdb-int-overflow
Make dbSwapDatabases take args as long
2019-07-12 12:36:33 +02:00
54c4e7f86c Vertically compact code in aofWrite(). 2019-07-12 12:18:33 +02:00
2a99f199bc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-10 19:48:44 +02:00
dad0e916ff Client side caching: filter clients no longer there. 2019-07-10 19:46:46 +02:00
7567323071 Merge pull request #6198 from oranagra/skiplist-mem-usage
missing per-skiplist overheads in MEMORY USAGE
2019-07-10 19:43:24 +02:00
c7aaf8db4d Client side caching: implement CLIENT GETREDIR.
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
2019-07-10 18:17:10 +02:00
923e4fb312 Client side caching: add tracking clients in INFO. 2019-07-10 18:08:31 +02:00
7c9f6ebc8f Diskless replica: fix disklessLoadRestoreBackups() bug. 2019-07-10 12:36:26 +02:00
3bbb9a1413 Diskless replica: refactoring of DBs backups. 2019-07-10 11:42:26 +02:00
3300e98ff0 Diskless replica: fix mispelled var name. 2019-07-10 09:34:21 +02:00
dfcbeaf115 Diskless replica: a few aesthetic changes to rio.c 2019-07-08 18:39:59 +02:00
81b18fa3a0 Diskless replica: a few aesthetic changes to replication.c. 2019-07-08 18:32:47 +02:00
d984732b35 Merge pull request #6193 from oranagra/diskless_slave_refresh3
diskless replication on slave side (don't store rdb to file), plus some other related fixes
2019-07-08 18:10:16 +02:00
2de544cfcc diskless replication on slave side (don't store rdb to file), plus some other related fixes
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
2019-07-08 15:37:48 +03:00
722446510f Merge pull request #6116 from AngusP/scan-types
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
2019-07-08 12:53:34 +02:00
6eb52e200c Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
4c4b7023dd Merge pull request #6210 from gkorland/patch-3
fix build tracking.c should be tracking.o
2019-07-07 17:49:17 +02:00
2d07883cab fix build tracking.c should be tracking.o
thanks to @rafie
2019-07-07 18:28:15 +03:00
e8ae07aabf Merge pull request #6162 from madolson/dev-unstable-config-refactor
Refactored yesno configs so there is less duplication
2019-07-07 17:13:11 +02:00
677e95e2f5 Merge pull request #6161 from swilly22/redismodule-loading-ctx-flag
Extend REDISMODULE_CTX_FLAGS to indicate if redis is  loading
2019-07-07 17:01:29 +02:00
a714004738 Merge pull request #6022 from itamarhaber/RedisModule_ReplyWithCString
Adds RedisModule_ReplyWithCString
2019-07-05 20:25:18 +02:00