Commit Graph

430 Commits

Author SHA1 Message Date
3e6d4d599a Replication: add REPLCONF CAPA EOF support.
Add the concept of slaves capabilities to Redis, the slave now presents
to the Redis master with a set of capabilities in the form:

    REPLCONF capa SOMECAPA capa OTHERCAPA ...

This has the effect of setting slave->slave_capa with the corresponding
SLAVE_CAPA macros that the master can test later to understand if it
the slave will understand certain formats and protocols of the
replication process. This makes it much simpler to introduce new
replication capabilities in the future in a way that don't break old
slaves or masters.

This patch was designed and implemented together with Oran Agra
(@oranagra).
2015-08-06 09:23:23 +02:00
3c8861a73a Support for CLIENT KILL TYPE MASTER. 2015-07-28 17:01:19 +02:00
e6f39338e6 CLIENT_MASTER introduced. 2015-07-28 16:58:35 +02:00
278ea9d16b replicationHandleMasterDisconnection() belongs to replication.c. 2015-07-28 14:36:50 +02:00
32f80e2f1b RDMF: More consistent define names. 2015-07-27 14:37:58 +02:00
40eb548a80 RDMF: REDIS_OK REDIS_ERR -> C_OK C_ERR. 2015-07-26 23:17:55 +02:00
2d9e3eb107 RDMF: redisAssert -> serverAssert. 2015-07-26 15:29:53 +02:00
14ff572482 RDMF: OBJ_ macros for object related stuff. 2015-07-26 15:28:00 +02:00
554bd0e7bd RDMF: use client instead of redisClient, like Disque. 2015-07-26 15:20:52 +02:00
424fe9afd9 RDMF: redisLog -> serverLog. 2015-07-26 15:17:43 +02:00
cef054e868 RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
cb2782c314 SDS: changes to unify Redis SDS with antirez/sds repo. 2015-07-25 17:25:44 +02:00
f15df8ba5d sds size classes - memory optimization 2015-07-14 17:17:06 +02:00
bb3284563c Geo: GEOADD implementation improved, replication fixed
1. We no longer use a fake client but just rewriting.
2. We group all the inserts into a single ZADD dispatch (big speed win).
3. As a side effect of the correct implementation, replication works.
4. The return value of the command is now correct.
2015-06-23 10:20:14 +02:00
6b953a2681 protocol error log should be seen debug/verbose level 2015-05-12 10:04:52 +09:00
49c1b60bd8 Fix Redis server crash when Lua command exceeds client output buffer
limit.
2015-04-26 12:04:16 +03:00
6c60526db9 Net: improve prepareClientToWrite() error handling and comments.
When we fail to setup the write handler it does not make sense to take
the client around, it is missing writes: whatever is a client or a slave
anyway the connection should terminated ASAP.

Moreover what the function does exactly with its return value, and in
which case the write handler is installed on the socket, was not clear,
so the functions comment are improved to make the goals of the function
more obvious.

Also related to #2485.
2015-04-01 10:07:45 +02:00
159875b5a3 fixes to diskless replication.
master was closing the connection if the RDB transfer took long time.
and also sent PINGs to the slave before it got the initial ACK, in which case the slave wouldn't be able to find the EOF marker.
2015-03-31 23:42:08 +03:00
221d2932b5 Ensure array index is in range in addReplyLongLongWithPrefix().
Change done in order to remove a warning and improve code robustness. No
actual bug here.
2015-03-30 11:54:49 +02:00
2b278a3394 Net: processUnblockedClients() and clientsArePaused() minor changes.
1. No need to set btype in processUnblockedClients(), since clients
   flagged REDIS_UNBLOCKED should have it already cleared.
2. When putting clients in the unblocked clients list, clientsArePaused()
   should flag them with REDIS_UNBLOCKED. Not strictly needed with the
   current code but is more coherent.
2015-03-21 09:13:29 +01:00
5fe4a23131 Net: clientsArePaused() should not touch blocked clients.
When the list of unblocked clients were processed, btype was set to
blocking type none, but the client remained flagged with REDIS_BLOCKED.
When timeout is reached (or when the client disconnects), unblocking it
will trigger an assertion.

There is no need to process pending requests from blocked clients, so
now clientsArePaused() just avoid touching blocked clients.

Close #2467.
2015-03-21 09:04:38 +01:00
8e219224b9 CONFIG refactoring: configEnum abstraction.
Still many things to convert inside config.c in the next commits.
Some const safety in String objects creation and addReply() family
functions.
2015-03-11 17:00:13 +01:00
7e6b4ea67b server.current_client fix and minor refactoring.
Thanks to @codeslinger (Toby DiPasquale) for identifying the issue.

Related to issue #2409.
2015-02-27 14:17:46 +01:00
27c30b0e84 Cast sentlen to int before comparison wit bufpos.
This is safe since bufpos is small, inside the range of the local
client buffer.
2015-02-25 10:33:37 +01:00
53c082ec39 Improve networking type correctness
read() and write() return ssize_t (signed long), not int.

For other offsets, we can use the unsigned size_t type instead
of a signed offset (since our replication offsets and buffer
positions are never negative).
2015-01-19 14:10:12 -05:00
ad41a7c404 Add addReplyBulkSds() function
Refactor a common pattern into one function so we don't
end up with copy/paste programming.
2014-12-23 09:31:02 -05:00
bbf0736c4e sdsformatip() removed.
Specialized single-use function. Not the best match for sds.c btw.
Also genClientPeerId() is no longer static: we need symbols.
2014-12-11 18:29:04 +01:00
491881e13b Cleanup all IP formatting code
Instead of manually checking for strchr(n,':') everywhere,
we can use our new centralized IP formatting functions.
2014-12-11 10:12:18 -05:00
1b732c09d0 Network bandwidth tracking + refactoring.
Track bandwidth used by clients and replication (but diskless
replication is not tracked since the actual transfer happens in the
child process).

This includes a refactoring that makes tracking new instantaneous
metrics simpler.
2014-12-03 12:16:25 +01:00
a8f9a989a7 Avoid valgrind memory leak false positive in processInlineBuffer().
zmalloc(0) cauesd to actually trigger a non-zero allocation since with
standard libc malloc we have our own zmalloc header for memory tracking,
but at the same time the returned pointer is at the end of the block and
not in the middle. This triggers a false positive when testing with
valgrind.

When the inline protocol args count is 0, we now avoid reallocating
c->argv, preventing the issue to happen.
2014-11-25 14:48:30 +01:00
bb7fea0d5c Diskless SYNC: fix RDB EOF detection.
RDB EOF detection was relying on the final part of the RDB transfer to
be a magic 40 bytes EOF marker. However as the slave is put online
immediately, and because of sockets timeouts, the replication stream is
actually contiguous with the RDB file.

This means that to detect the EOF correctly we should either:

1) Scan all the stream searching for the mark. Sucks CPU-wise.
2) Start to send the replication stream only after an acknowledge.
3) Implement a proper chunked encoding.

For now solution "2" was picked, so the master does not start to send
ASAP the stream of commands in the case of diskless replication. We wait
for the first REPLCONF ACK command from the slave, that certifies us
that the slave correctly loaded the RDB file and is ready to get more
data.
2014-11-11 17:12:12 +01:00
8a416ca46e Added a function to get slave name for logs. 2014-10-27 11:58:20 +01:00
fa8786f17a Fix typos in comments
Closes #2002
2014-09-29 06:49:09 -04:00
edca2b14d2 Remove warnings and improve integer sign correctness. 2014-08-13 11:44:38 +02:00
59cf0824d9 PubSub clients refactoring and new PUBSUB flag.
The code tested many times if a client had active Pub/Sub subscriptions
by checking the length of a list and dictionary where the patterns and
channels are stored. This was substituted with a client flag called
REDIS_PUBSUB that is simpler to test for. Moreover in order to manage
this flag some code was refactored.

This commit is believed to have no effects in the behavior of the
server.
2014-07-16 17:34:07 +02:00
95b1979c32 No more trailing spaces in Redis source code. 2014-06-26 18:48:40 +02:00
97f1fc65cf CLIENT KILL: don't kill the master as a normal client.
Technically the problem is due to the client type API that does not
return a special value for the master, however fixing it locally in the
CLIENT KILL command is better currently because otherwise we would
introduce a new output buffer limit class as a side effect.
2014-06-26 18:43:09 +02:00
46319094db Old form of CLIENT KILL should still allow suicide. 2014-06-24 12:49:28 +02:00
bb2011d992 CLIENT KILL API modified.
Added a new SKIPME option that is true by default, that prevents the
client sending the command to be killed, unless SKIPME NO is sent.
2014-06-16 14:50:15 +02:00
e06b3819ea CLIENT KILL: fix closing link of the current client. 2014-06-16 14:28:23 +02:00
e7affd266c New features for CLIENT KILL. 2014-06-16 14:24:28 +02:00
f26f79ea37 Assign an unique non-repeating ID to each new client.
This will be used by CLIENT KILL and is also a good way to ensure a
given client is still the same across CLIENT LIST calls.

The output of CLIENT LIST was modified to include the new ID, but this
change is considered to be backward compatible as the API does not imply
you can do positional parsing, since each filed as a different name.
2014-06-16 14:22:55 +02:00
56d26c2380 Client types generalized.
Because of output buffer limits Redis internals had this idea of type of
clients: normal, pubsub, slave. It is possible to set different output
buffer limits for the three kinds of clients.

However all the macros and API were named after output buffer limit
classes, while the idea of a client type is a generic one that can be
reused.

This commit does two things:

1) Rename the API and defines with more general names.
2) Change the class of clients executing the MONITOR command from "slave"
   to "normal".

"2" is a good idea because you want to have very special settings for
slaves, that are not a good idea for MONITOR clients that are instead
normal clients even if they are conceptually slave-alike (since it is a
push protocol).

The backward-compatibility breakage resulting from "2" is considered to
be minimal to care, since MONITOR is a debugging command, and because
anyway this change is not going to break the format or the behavior, but
just when a connection is closed on big output buffer issues.
2014-06-16 10:43:05 +02:00
0bcc7cb4bf CLIENT LIST speedup via peerid caching + smart allocation.
This commit adds peer ID caching in the client structure plus an API
change and the use of sdsMakeRoomFor() in order to improve the
reallocation pattern to generate the CLIENT LIST output.

Both the changes account for a very significant speedup.
2014-04-28 17:36:57 +02:00
f9a4a80f49 Use sdscatfmt() in getClientInfoString() to make it faster. 2014-04-28 16:55:43 +02:00
e29d330724 Process events with processEventsWhileBlocked() when blocked.
When we are blocked and a few events a processed from time to time, it
is smarter to call the event handler a few times in order to handle the
accept, read, write, close cycle of a client in a single pass, otherwise
there is too much latency added for clients to receive a reply while the
server is busy in some way (for example during the DB loading).
2014-04-24 21:44:32 +02:00
3a3458ee7b Accept multiple clients per iteration.
When the listening sockets readable event is fired, we have the chance
to accept multiple clients instead of accepting a single one. This makes
Redis more responsive when there is a mass-connect event (for example
after the server startup), and in workloads where a connect-disconnect
pattern is used often, so that multiple clients are waiting to be
accepted continuously.

As a side effect, this commit makes the LOADING, BUSY, and similar
errors much faster to deliver to the client, making Redis more
responsive when there is to return errors to inform the clients that the
server is blocked in an not interruptible operation.
2014-04-24 21:44:32 +02:00
cac4bae11a AE_ERR -> ANET_ERR in acceptUnixHandler().
No actual changes since the value is the same.
2014-04-24 21:43:22 +02:00
cb4e2ee9e7 Missing return REDIS_ERR added to processMultibulkBuffer().
When we set a protocol error we should return with REDIS_ERR to let the
caller know it should stop processing the client.

Bug found in a code auditing related to issue #1699.
2014-04-23 10:19:43 +02:00
59cf0b1902 Fix return value check for anetTcpAccept
anetTcpAccept returns ANET_ERR, not AE_ERR.

This isn't a physical error since both ANET_ERR
and AE_ERR are -1, but better to be consistent.
2014-03-06 17:55:31 +01:00