Commit Graph

4786 Commits

Author SHA1 Message Date
aa3de04232 dict.c/dictRehash: check again to update 2015-02-11 11:01:46 +01:00
252ebcd973 dict.c: don't try buckets that are empty for sure in dictGetRandomKey().
This is very similar to the optimization applied to dictGetRandomKeys,
but applied to the single key variant.

Related to issue #2306.
2015-02-11 11:01:46 +01:00
a355568c1a dict.c: dictGetRandomKeys() optimization for big->small table case.
Related to issue #2306.
2015-02-11 11:01:46 +01:00
eb4859db05 dict.c: dictGetRandomKeys() visit pattern optimization.
We use the invariant that the original table ht[0] is never populated up
to the index before the current rehashing index.

Related to issue #2306.
2015-02-11 11:01:46 +01:00
e2ab7be33c dict.c: put a bound to max work dictRehash() call can do.
Related to issue #2306.
2015-02-11 11:01:46 +01:00
40f64bc68a dict.c: prevent useless resize to same size.
Related to issue #2306.
2015-02-11 11:01:46 +01:00
b7d20a2000 Less blocking dictGetRandomKeys().
Related to issue #2306.
2015-02-11 11:01:46 +01:00
3a0e69f391 override histfile from env - fixes #831 and copies #833 2015-02-10 14:52:32 +01:00
c3a5ad2018 redis-cli --lru-test implemented (cache workload simulator). 2015-02-10 14:51:25 +01:00
4793ea982a redis-cli: interactive reconnection for latency modes.
--stat mode already used to reconnect automatically if the server is no
longer available. This is useful since this is an interactive mode used
for debugging, however the same applies to --latency and --latency-dist
modes, so now both use the reconnecting command execution as well.

The reconnection code was modified to use basic VT100 escape sequences
in order to play better with different kinds of output on the screen
when the reconnection happens, and to hide the reconnection attempt
output when finally the reconnection happens.
2015-02-10 14:51:25 +01:00
9a4d71e92e redis-cli --latecy-dist reverted to gray scale.
So far not able to find a color palette within the 256 colors which is
not confusing. However I believe it is a possible task, so will try
better later.
2015-02-10 14:51:25 +01:00
ad9976cd90 redis-cli --latency-dist now uses a color palette.
Still not happy with the result but low grays are hard to see in certain
monitors with a non perfect gamma.
2015-02-10 14:51:25 +01:00
b857578ba2 redis-cli latency dist: add new top HL. 2015-02-10 14:51:25 +01:00
630634a37a Add missing latency-dest legend symbol. 2015-02-10 14:51:25 +01:00
cdf30ba8c7 Initial implementation of redis-cli --latency-dist. 2015-02-10 14:51:25 +01:00
bee03daca7 Faster memory efficiency test.
This test on Linux was extremely slow, since in Tcl we can't enable
easily tcp-nodelay, so the busy loop used to take *a lot* with bigger
writes. Fixed using pipelining.
2015-02-10 14:48:19 +01:00
560823f54c Ignore config.sh inside create-cluster script dir. 2015-02-03 10:11:40 +01:00
7db39b7fd3 Update redis-cli.c
Code was adding '\n'  (line 521) to the end of NIL values exlusively making csv output inconsistent.  Removed '\n'
2015-02-02 22:50:45 +01:00
9b3a1c3c40 Redis 2.9.103 (3.0.0 Release Candidate 3). 3.0.0-rc3 2015-01-30 15:34:16 +01:00
9389d050e3 Merge branch 'testing' into 3.0 2015-01-30 12:18:54 +01:00
19029a4ebc Cluster: Tcl script to check avg pfail->fail time. 2015-01-30 12:18:42 +01:00
55f2bc646a Cluster: some bias towwards FAIL/PFAIL nodes in gossip sections.
This improves PFAIL -> FAIL switch. Too late at this point in the RC
releases to add proper PFAIL/FAIL separate dictionary to do this in a
less randomized way. Tested in practice with experiments that this
helps. PFAIL -> FAIL average with 20 nodes and node-timeout set to 5
seconds takes 2.5 seconds without this commit, 1 second with this
commit.
2015-01-30 12:18:42 +01:00
0f1b9c3db1 More correct wanted / maxiterations values in clusterSendPing(). 2015-01-30 12:18:42 +01:00
96368c2d0b Cluster: create-cluster script improved. 2015-01-30 12:18:42 +01:00
f2cd2fcdb7 Cluster: initialized not used fileds in gossip section.
Otherwise we risk sending not initialized data to other nodes, that may
contain anything. This was actually not possible only because the
initialization of the buffer where the cluster packets header is created
was larger than the 3 gossip sections we use, so the memory was already
all filled with zeroes by the memset().
2015-01-29 15:52:23 +01:00
2553f6c9e5 Cluster: initialized not used fileds in gossip section.
Otherwise we risk sending not initialized data to other nodes, that may
contain anything. This was actually not possible only because the
initialization of the buffer where the cluster packets header is created
was larger than the 3 gossip sections we use, so the memory was already
all filled with zeroes by the memset().
2015-01-29 15:52:17 +01:00
147df6f035 Merge branch 'testing' into 3.0 2015-01-29 15:45:21 +01:00
2616d6f6dc Cluster: magical 10% of nodes explained in comments. 2015-01-29 15:44:54 +01:00
92f29b8904 CLUSTER count-failure-reports command added. 2015-01-29 15:44:49 +01:00
8dd3263216 Cluster: use a number of gossip sections proportional to cluster size.
Otherwise it is impossible to receive the majority of failure reports in
the node_timeout*2 window in larger clusters.

Still with a 200 nodes cluster, 20 gossip sections are a very reasonable
amount of bytes to send.

A side effect of this change is also fater cluster nodes joins for large
clusters, because the cluster layout makes less time to propagate.
2015-01-29 15:44:46 +01:00
5031c2394e create-cluster script: sane default timeout. 2015-01-29 15:44:42 +01:00
75c5229a8b create-cluster script added.
Simple shell script to create / destroy Redis clusters for manual
testing.
2015-01-29 15:44:37 +01:00
267f2ec40b Avoid duplicated instance execution code in Cluster test. 2015-01-22 18:59:44 +01:00
0232eaacdf Avoid duplicated instance execution code in Cluster test. 2015-01-22 18:59:42 +01:00
c7e31c6812 Merge branch 'testing' into 3.0 2015-01-22 16:31:41 +01:00
400ae01a7f Cluster test: when valgrind is enabled, use a larger node-timeout.
Removes some percentage of timing related failures.
2015-01-22 16:31:36 +01:00
096ad2e89d Merge branch 'testing' into 3.0 2015-01-22 11:12:39 +01:00
71da1b15c8 The seed must be static in getRandomHexChars(). 2015-01-22 11:12:28 +01:00
4e73b305f0 counter must be static in getRandomHexChars(). 2015-01-22 11:12:28 +01:00
b25154a387 getRandomHexChars(): use /dev/urandom just to seed.
On Darwin /dev/urandom depletes terribly fast. This is not an issue
normally, but with Redis Cluster we generate a lot of unique IDs, for
example during nodes handshakes. Our IDs need just to be unique without
other strong crypto requirements, so this commit turns the function into
something that gets a 20 bytes seed from /dev/urandom, and produces the
rest of the output just using SHA1 in counter mode.
2015-01-22 11:12:28 +01:00
7e41200b98 Cluster test initialization: use transaction for reset + set-config-epoch.
Otherwise between the two commands other nodes may contact us making the
next SET-CONFIG-EPOCH call impossible.
2015-01-22 11:12:28 +01:00
00e76ae8f9 Merge branch 'testing' into 3.0 2015-01-22 10:35:52 +01:00
ebb07a0b48 Fix cluster migrate memory leak
Fixes valgrind error:
48 bytes in 1 blocks are definitely lost in loss record 196 of 373
   at 0x4910D3: je_malloc (jemalloc.c:944)
   by 0x42807D: zmalloc (zmalloc.c:125)
   by 0x41FA0D: dictGetIterator (dict.c:543)
   by 0x41FA48: dictGetSafeIterator (dict.c:555)
   by 0x459B73: clusterHandleSlaveMigration (cluster.c:2776)
   by 0x45BF27: clusterCron (cluster.c:3123)
   by 0x423344: serverCron (redis.c:1239)
   by 0x41D6CD: aeProcessEvents (ae.c:311)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
2015-01-22 10:35:40 +01:00
98faed3a3f Fix potential invalid read past end of array
If array has N elements, we can't read +1 if we are already at N.

Also, we need to move elements by their storage size in the array,
not just by individual bytes.
2015-01-22 10:35:36 +01:00
97ffeb7c09 Fix cluster reset memory leak
[maybe] Fixes valgrind errors:
32 bytes in 4 blocks are definitely lost in loss record 107 of 228
   at 0x80EA447: je_malloc (jemalloc.c:944)
   by 0x806E59C: zrealloc (zmalloc.c:125)
   by 0x80A9AFC: clusterSetMaster (cluster.c:801)
   by 0x80AEDC9: clusterCommand (cluster.c:3994)
   by 0x80682A5: call (redis.c:2049)
   by 0x8068A20: processCommand (redis.c:2309)
   by 0x8076497: processInputBuffer (networking.c:1143)
   by 0x8073BAF: readQueryFromClient (networking.c:1208)
   by 0x8060E98: aeProcessEvents (ae.c:412)
   by 0x806123B: aeMain (ae.c:455)
   by 0x806C3DB: main (redis.c:3832)

64 bytes in 8 blocks are definitely lost in loss record 143 of 228
   at 0x80EA447: je_malloc (jemalloc.c:944)
   by 0x806E59C: zrealloc (zmalloc.c:125)
   by 0x80AAB40: clusterProcessPacket (cluster.c:801)
   by 0x80A847F: clusterReadHandler (cluster.c:1975)
   by 0x30000FF: ???

80 bytes in 10 blocks are definitely lost in loss record 148 of 228
   at 0x80EA447: je_malloc (jemalloc.c:944)
   by 0x806E59C: zrealloc (zmalloc.c:125)
   by 0x80AAB40: clusterProcessPacket (cluster.c:801)
   by 0x80A847F: clusterReadHandler (cluster.c:1975)
   by 0x2FFFFFF: ???
2015-01-22 10:35:32 +01:00
4a36350d9f Fix sending uninitialized bytes
Fixes valgrind error:
Syscall param write(buf) points to uninitialised byte(s)
   at 0x514C35D: ??? (syscall-template.S:81)
   by 0x456B81: clusterWriteHandler (cluster.c:1907)
   by 0x41D596: aeProcessEvents (ae.c:416)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
 Address 0x5f268e2 is 2,274 bytes inside a block of size 8,192 alloc'd
   at 0x4932D1: je_realloc (jemalloc.c:1297)
   by 0x428185: zrealloc (zmalloc.c:162)
   by 0x4269E0: sdsMakeRoomFor.part.0 (sds.c:142)
   by 0x426CD7: sdscatlen (sds.c:251)
   by 0x4579E7: clusterSendMessage (cluster.c:1995)
   by 0x45805A: clusterSendPing (cluster.c:2140)
   by 0x45BB03: clusterCron (cluster.c:2944)
   by 0x423344: serverCron (redis.c:1239)
   by 0x41D6CD: aeProcessEvents (ae.c:311)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
 Uninitialised value was created by a stack allocation
   at 0x457810: nodeUpdateAddressIfNeeded (cluster.c:1236)
2015-01-22 10:35:27 +01:00
c8f9037477 Cluster test: wait for port to unbound in kill_instance.
Otherwise kill_instance + restart_instance in short succession will
still find the port busy and will fail.
2015-01-22 10:35:23 +01:00
707d97ac4f Cluster/Sentinel test: also pause on abort_sentinel_test call. 2015-01-22 10:35:19 +01:00
4688f60b4b Cluster/Sentinel test: pause on exceptions as well. 2015-01-22 10:35:16 +01:00
0a3edcbe51 Cluster: node deletion cleanup / centralization. 2015-01-22 10:35:12 +01:00