5498 Commits

Author SHA1 Message Date
antirez
fdc8e0bd33 Scripting: execute tests with command replication as well. 2015-10-30 12:09:04 +01:00
antirez
8695d96024 Scripting: ability to turn on Lua commands style replication globally.
Currently this feature is only accessible via DEBUG for testing, since
otherwise depending on the instance configuration a given script works
or is broken, which is against the Redis philosophy.
2015-10-30 12:08:58 +01:00
antirez
f0434cae94 Scripting: test Redis provided Lua functions error reporting. 2015-10-30 12:08:20 +01:00
antirez
61fb05454b Scripting: fix error reporting of many Redis provided functions. 2015-10-30 12:08:20 +01:00
antirez
e9d2329c8e Fix call() FORCE_REPL/AOF flags setting.
This commit also inverts two stanzas of the code just becuase they are
more logical like that, not because currently it makes any difference.
2015-10-30 12:08:20 +01:00
antirez
dfe7f79708 Lua script selective replication fixes. 2015-10-30 12:08:20 +01:00
antirez
a1d1ca1416 Lua script selective replication WIP. 2015-10-30 12:08:20 +01:00
antirez
cbbfaf6ad3 Scripting: single commands replication mode implemented.
By calling redis.replicate_commands(), the scripting engine of Redis
switches to commands replication instead of replicating whole scripts.
This is useful when the script execution is costly but only results in a
few writes performed to the dataset.

Morover, in this mode, it is possible to call functions with side
effects freely, since the script execution does not need to be
deterministic: anyway we'll capture the outcome from the point of view
of changes to the dataset.

In this mode math.random() returns different sequences at every call.

If redis.replicate_commnads() is not called before any other write, the
command returns false and sticks to whole scripts replication instead.
2015-10-30 12:08:20 +01:00
antirez
24cf0f6dcd call(): selective ability to prevent propagation on AOF / slaves. 2015-10-30 12:08:20 +01:00
antirez
0259da8c30 call(): don't inherit CLIENT_PREVENT_PROP + minor refactoring. 2015-10-30 12:08:20 +01:00
antirez
de4d6caf55 CLIENT REPLY command implemented: ON, OFF and SKIP modes.
Sometimes it can be useful for clients to completely disable replies
from the Redis server. For example when the client sends fire and forget
commands or performs a mass loading of data, or in caching contexts
where new data is streamed constantly. In such contexts to use server
time and bandwidth in order to send back replies to clients, which are
going to be ignored, is a shame.

Multiple mechanisms are possible to implement such a feature. For
example it could be a feature of MULTI/EXEC, or a command prefix
such as "NOREPLY SADD myset foo", or a different mechanism that allows
to switch on/off requests using the CLIENT command.

The MULTI/EXEC approach has the problem that transactions are not
strictly part of the no-reply semantics, and if we want to insert a lot
of data in a bulk way, creating a huge MULTI/EXEC transaction in the
server memory is bad.

The prefix is the best in this specific use case since it does not allow
desynchronizations, and is pretty clear semantically. However Redis
internals and client libraries are not prepared to handle this
currently.

So the implementation uses the CLIENT command, providing a new REPLY
subcommand with three options:

    CLIENT REPLY OFF disables the replies, and does not reply itself.
    CLIENT REPLY ON re-enables the replies, replying +OK.
    CLIENT REPLY SKIP only discards the reply of the next command, and
                      like OFF does not reply anything itself.

The reason to add the SKIP command is that it allows to have an easy
way to send conceptually "single" commands that don't need a reply
as the sum of two pipelined commands:

    CLIENT REPLY SKIP
    SET key value

Note that CLIENT REPLY ON replies with +OK so it should be used when
sending multiple commands that don't need a reply. However since it
replies with +OK the client can check that the connection is still
active and all the previous commands were received.

This is currently just into Redis "unstable" so the proposal can be
modified or abandoned based on users inputs.
2015-10-30 12:08:15 +01:00
antirez
600492a7db CONTRIBUTING updated. 2015-10-27 12:08:04 +01:00
David Thomson
b66bf08f4f Add back blank line 2015-10-15 13:05:28 +02:00
David Thomson
47611f7cca Update import command to optionally use copy and replace parameters 2015-10-15 13:05:28 +02:00
antirez
e16c0c19ff Redis.conf example: make clear user must pass its path as argument. 2015-10-15 12:46:14 +02:00
antirez
f28803ce90 Regression test for issue #2813. 2015-10-15 11:25:16 +02:00
antirez
e3344b80cd PR 2813 fix ported to unstable. 2015-10-15 10:21:33 +02:00
antirez
fcb3aef7c4 DEBUG RESTART/CRASH-AND-RECOVER [delay] implemented. 2015-10-15 10:21:33 +02:00
antirez
1db84c21b4 Server: restartServer() API.
This new function is able to restart the server "in place". The current
Redis process executes the same executable it was executed with, using
the same arguments and configuration file.
2015-10-15 10:21:33 +02:00
antirez
2b3f25e00f Cluster: redis-trib fix, coverage for migrating=1 case.
Kinda related to #2770.
2015-10-15 10:21:32 +02:00
antirez
ad8111ba18 Added a README into deps on dependencies and how to upgrade. 2015-10-15 10:21:32 +02:00
antirez
032aeb87dd Test: fix attach_to_replication_stream to handle newlines. 2015-10-09 09:59:20 +02:00
antirez
77df3b06b2 Fix extractLongLatOrReply() sanity check conditionals.
the check for lat/long valid ranges were performed inside the for loop,
two times instead of one, and the first time when the second element of
the array, xy[1], was yet not populated. This resulted into issue #2799.

Close issue #2799.
2015-10-09 09:59:14 +02:00
antirez
7b74d0057b Jemalloc configure script fixed to work nested.
Now way to make unmodified Jemalloc configure to work when the jemalloc
source tree is inside a subdirectory of a different git repository.

Problem signaled here:
http://www.canonware.com/pipermail/jemalloc-discuss/2015-October/001174.html
2015-10-09 09:59:09 +02:00
antirez
5268379e5e Jemalloc updated to 4.0.3. 2015-10-09 09:59:03 +02:00
antirez
589c41e457 Regression test for GEORADIUS COUNT arity check. 2015-10-06 09:27:52 +02:00
antirez
cff2790316 Fix GEORADIUS COUNT option arity checks. 2015-10-06 09:27:52 +02:00
antirez
ded5a248d8 Call writeToClient() directly instead of the write handler. 2015-10-01 11:39:13 +02:00
antirez
9fe23e8f30 Fix processEventsWhileBlocked() to handle PENDING_WRITE clients.
After the introduction of the list with clients with pending writes, to
process clients incrementally outside of the event loop we also need to
process the pending writes list.
2015-10-01 11:39:13 +02:00
antirez
47e6cf119c Refactoring: unlinkClient() added to lower freeClient() complexity. 2015-10-01 11:39:13 +02:00
antirez
2d21af4548 Refactoring: new function to test if client has pending output. 2015-10-01 11:39:13 +02:00
antirez
1c512e4960 Reverse list of clients with pending writes.
May potentially improve locality... not exactly clear if this makes a
difference or not. But for sure is harmless.
2015-10-01 11:39:13 +02:00
antirez
849aa99cb9 writeToClient(): don't remove write handler if not needed. 2015-10-01 11:39:13 +02:00
antirez
37e8b6fcb8 handleClientsWithPendingWrites(): detect dead clients. 2015-10-01 11:39:13 +02:00
antirez
838833cd32 Move handleClientsWithPendingWrites() in networking.c. 2015-10-01 11:39:13 +02:00
antirez
23e7710c28 Avoid installing the client write handler when possible. 2015-10-01 11:39:13 +02:00
antirez
0f81f83ea4 redis-cli pipe mode: don't stay in the write loop forever.
The code was broken and resulted in redis-cli --pipe to, most of the
times, writing everything received in the standard input to the Redis
connection socket without ever reading back the replies, until all the
content to write was written.

This means that Redis had to accumulate all the output in the output
buffers of the client, consuming a lot of memory.

Fixed thanks to the original report of anomalies in the behavior
provided by Twitter user @fsaintjacques.
2015-09-30 16:27:24 +02:00
antirez
9d3f8e197e Release notes for what will become 3.2. 2015-09-29 17:29:39 +02:00
antirez
846da5b22e Test: fix false positive in HSTRLEN test.
HINCRBY* tests later used the value "tmp" that was sometimes generated
by the random key generation function. The result was ovewriting what
Tcl expected to be inside Redis with another value, causing the next
HSTRLEN test to fail.
2015-09-15 09:37:30 +02:00
antirez
3c23b5ffd0 GEORADIUS: Don't report duplicates when radius is huge.
Georadius works by computing the center + neighbors squares covering all
the area of the specified position and radius. Then a distance filter is
used to remove elements which are actually outside the range.

When a huge radius is used, like 5000 km or more, adjacent neighbors may
collide and be the same, leading to the reporting of the same element
multiple times. This only happens in the edge case of huge radius but is
not ideal.

A robust but slow solution would involve qsorting the range to remove
all the duplicates. However since the collisions are only in adjacent
boxes, for the way they are ordered in the code, it is much faster to
just check if the current box is the same as the previous one processed.

This commit adds a regression test for the bug.

Fixes #2767.
2015-09-14 23:10:50 +02:00
antirez
0a91fc459f Test: MOVE expire test improved.
Related to #2765.
2015-09-14 12:35:55 +02:00
antirez
4fec5ee165 MOVE re-add TTL check fixed.
getExpire() returns -1 when no expire exists.

Related to #2765.
2015-09-14 12:34:17 +02:00
antirez
f529a01c1b MOVE now can move TTL metadata as well.
MOVE was not able to move the TTL: when a key was moved into a different
database number, it became persistent like if PERSIST was used.

In some incredible way (I guess almost nobody uses Redis MOVE) this bug
remained unnoticed inside Redis internals for many years.
Finally Andy Grunwald discovered it and opened an issue.

This commit fixes the bug and adds a regression test.

Close #2765.
2015-09-14 12:30:00 +02:00
antirez
33769f840c Sentinel: command arity check added where missing. 2015-09-08 09:27:43 +02:00
Salvatore Sanfilippo
0c62d95538 Merge pull request #2695 from rogerlz/unstable
redis-sentinel crash if ckquorum command is executed without args
2015-09-08 09:24:45 +02:00
antirez
8e55537459 Undo slaves state change on failed rdbSaveToSlavesSockets().
As Oran Agra suggested, in startBgsaveForReplication() when the BGSAVE
attempt returns an error, we scan the list of slaves in order to remove
them since there is no way to serve them currently.

However we check for the replication state BGSAVE_START, which was
modified by rdbSaveToSlaveSockets() before forking(). So when fork fails
the state of slaves remain BGSAVE_END and no cleanup is performed.

This commit fixes the problem by making rdbSaveToSlavesSockets() able to
undo the state change on fork failure.
2015-09-07 16:09:23 +02:00
Salvatore Sanfilippo
5f81303503 Merge pull request #2753 from ofirluzon/unstable
SCAN iter parsing changed from atoi to chartoull
2015-09-07 13:24:43 +02:00
ubuntu
11381b09d9 SCAN iter parsing changed from atoi to chartoull 2015-09-07 11:20:52 +00:00
antirez
467de61c84 Test: print info on HSTRLEN test failure.
This additional info may provide more clues about the test randomly
failing from time to time. Probably the failure is due to some previous
test that overwrites the logical content in the Tcl variable, but this
will make the problem more obvious.
2015-09-07 11:14:52 +02:00
antirez
d036abe27d Log client details on SLAVEOF command having an effect. 2015-08-21 15:29:07 +02:00