Commit Graph

5024 Commits

Author SHA1 Message Date
5171a3ff8e RDB: Apply fix to rdbLoadMillisecondTime() only for new RDB versions.
This way we let big endian systems to still load old RDB versions.
However newver versions will be saved and loaded in a way that make RDB
expires cross-endian again. Thanks to @oranagra for the reporting and
the discussion about this problem, leading to this fix.
2018-06-13 13:26:13 +02:00
af7069627e Streams: generate a few additional events.
Currently it does not look it's sensible to generate events for streams
consumer groups modification, being them metadata, however at least for
key-level events, like the creation or removal of a consumer group, I
added a few events here and there. Later we can evaluate if it makes
sense to add more. From the POV instead of WAIT (in Redis transaciton)
and signaling the key as modified, it looks like that the transaction
should not fail when a stream is modified, so no calls are made in
consumer groups related functions to signalModifiedKey().
2018-06-13 13:26:12 +02:00
3ad68a07ff Fix rdbSaveKeyValuePair() integer overflow.
Again thanks to @oranagra. The object idle time does not fit into an int
sometimes: use the native type that the serialization function will get
as argument, which is uint64_t.
2018-06-13 13:26:12 +02:00
68382091a0 In scanDatabaseForReadyLists() now we need to handle ZSETs as well.
Since the introduction of ZPOP makes this needed. Thanks to @oranagra
for reporting.
2018-06-13 13:26:12 +02:00
6b0cdbd903 RDB: store times consistently in little endian.
I'm not sure how this escaped the attention of Redis users for years,
but finally @oranagra reported this issue... Thanks to Oran.
2018-06-13 13:26:12 +02:00
574017b7d2 Streams: improve type correctness in t_stream.c. 2018-06-13 13:26:12 +02:00
cd2b5a79ff Fix XGROUP help missing space. 2018-06-13 13:26:12 +02:00
8246a38e83 fix typo
fix [#5005](https://github.com/antirez/redis/issues/5005)
2018-06-13 13:26:12 +02:00
b80e4b6925 Security: fix redis-cli buffer overflow.
Thanks to Fakhri Zulkifli for reporting it.

The fix switched to dynamic allocation, copying the final prompt in the
static buffer only at the end.
2018-06-13 12:40:43 +02:00
ce17f76bbd Security: fix redis-cli buffer overflow.
Thanks to Fakhri Zulkifli for reporting it.

The fix switched to dynamic allocation, copying the final prompt in the
static buffer only at the end.
2018-06-13 12:40:33 +02:00
032ea657d7 RDB: Apply fix to rdbLoadMillisecondTime() only for new RDB versions.
This way we let big endian systems to still load old RDB versions.
However newver versions will be saved and loaded in a way that make RDB
expires cross-endian again. Thanks to @oranagra for the reporting and
the discussion about this problem, leading to this fix.
2018-06-12 18:21:39 +02:00
6b8a24a665 Streams: generate a few additional events.
Currently it does not look it's sensible to generate events for streams
consumer groups modification, being them metadata, however at least for
key-level events, like the creation or removal of a consumer group, I
added a few events here and there. Later we can evaluate if it makes
sense to add more. From the POV instead of WAIT (in Redis transaciton)
and signaling the key as modified, it looks like that the transaction
should not fail when a stream is modified, so no calls are made in
consumer groups related functions to signalModifiedKey().
2018-06-12 18:11:15 +02:00
b38682199b Fix rdbSaveKeyValuePair() integer overflow.
Again thanks to @oranagra. The object idle time does not fit into an int
sometimes: use the native type that the serialization function will get
as argument, which is uint64_t.
2018-06-12 17:31:04 +02:00
e534e9aa83 In scanDatabaseForReadyLists() now we need to handle ZSETs as well.
Since the introduction of ZPOP makes this needed. Thanks to @oranagra
for reporting.
2018-06-12 17:28:40 +02:00
f70e88c1f6 RDB: store times consistently in little endian.
I'm not sure how this escaped the attention of Redis users for years,
but finally @oranagra reported this issue... Thanks to Oran.
2018-06-12 17:22:03 +02:00
4774d61691 Streams: improve type correctness in t_stream.c. 2018-06-12 14:12:53 +02:00
6b675b9525 Applies addReplySubSyntaxError to stream commands 2018-06-12 14:52:50 +03:00
bcc42028c1 Fix XGROUP help missing space. 2018-06-12 13:20:46 +02:00
6acd8a31a4 fix active-defrag-threshold value error
The active-defrag-threshold-lower/active-defrag-threshold-upper min/max  value in redis.conf should be consistent with 'config set' command.
2018-06-12 13:15:53 +02:00
20bc3786c0 Streams: fix backward iteration when entry is not flagged SAMEFIELD.
See issue #5006. The comment in the code was also wrong and
was rectified as well.
2018-06-12 13:15:53 +02:00
2cadef46f7 Streams: increment dirty counter for XGROUP SETID/DESTROY.
See issue #5005 comments.
2018-06-12 13:15:53 +02:00
093ec57d06 Use a less aggressive query buffer resize policy.
A user with many connections (10 thousand) on a single Redis server
reports in issue #4983 that sometimes Redis is idle becuase at the same
time many clients need to resize their query buffer according to the old
policy.

It looks like this was created by the fact that we allow the query
buffer to grow without problems to a size up to PROTO_MBULK_BIG_ARG
normally, but when the client is idle we immediately are more strict,
and a query buffer greater than 1024 bytes is already enough to trigger
the resize. So for instance if most of the clients stop at the same time
this issue should be easily triggered.

This behavior actually looks odd, and there should be only a clear limit
after we say, let's look at this query buffer to check if it's time to
resize it. This commit puts the limit at PROTO_MBULK_BIG_ARG, and the
check is performed both if compared to the peak usage the current usage
is too big, or if the client is idle.

Then when the check is performed, to waste just a few kbytes is
considered enough to proceed with the resize. This should fix the issue.
2018-06-12 13:15:52 +02:00
d769ce0a20 Fix client unblocking for XREADGROUP, issue #4978.
We unblocked the client too early, when the group name object was no
longer valid in client->bpop, so propagating XCLAIM later in
streamPropagateXCLAIM() deferenced a field already set to NULL.
2018-06-12 13:15:52 +02:00
28e0ca5a8c Streams: checkType for xread & xinfo 2018-06-12 13:15:52 +02:00
9d2b1f294d Streams: lookupKey[Read->Write]OrReply in xdel and xtrim 2018-06-12 13:15:52 +02:00
3d4ad250cd Abort in XGROUP if the key is not a stream 2018-06-12 13:15:52 +02:00
5a3d490c50 fix integer case error 2018-06-12 13:15:52 +02:00
24fb4e3894 Implement DEBUG htstats-key. 2018-06-12 13:15:52 +02:00
a67ecb91f9 redis-cli inline help updated. 2018-06-12 13:15:52 +02:00
eb26a8174a Typo: entires -> entries in several places. 2018-06-12 13:15:13 +02:00
f2d65c380d Streams: make macro node limits configurable. 2018-06-12 13:15:13 +02:00
b16e5432b6 Streams: max node limits only checked if non zero. 2018-06-12 13:15:13 +02:00
d01af7abe4 Streams: use non static macro node limits.
Also add the concept of size/items limit, instead of just having as
limit the number of bytes.
2018-06-12 13:15:13 +02:00
77acc63eef Fix write() errno error 2018-06-12 13:15:13 +02:00
3ddc537605 Return early in XPENDING if sent a nonexistent consumer group. 2018-06-12 13:15:13 +02:00
b85086c573 Typo in preprocessor condition 2018-06-12 13:15:13 +02:00
4a9670f517 RDB: expand dict if needed when rdb load object 2018-06-12 13:15:13 +02:00
d5712d2151 adjust position of _dictNextPower in dictExpand 2018-06-12 13:15:13 +02:00
3ca02fb939 zset: change the span of zskiplistNode to unsigned long 2018-06-12 13:15:13 +02:00
59e51f92d2 zset: fix the int problem 2018-06-12 13:15:13 +02:00
e916f4519c Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-06-12 13:14:01 +02:00
bcecbebb74 Merge pull request #5012 from shenlongxing/fix-config
fix active-defrag-threshold value error
2018-06-12 13:05:19 +02:00
13957c9d07 fix active-defrag-threshold value error
The active-defrag-threshold-lower/active-defrag-threshold-upper min/max  value in redis.conf should be consistent with 'config set' command.
2018-06-12 17:59:32 +08:00
7cc1312789 Streams: fix backward iteration when entry is not flagged SAMEFIELD.
See issue #5006. The comment in the code was also wrong and
was rectified as well.
2018-06-12 10:22:49 +02:00
7659619824 Update geohash.c
fix geohasEncode bug when step > 31
2018-06-12 15:28:28 +08:00
82661ba329 Merge pull request #5007 from leonchen83/patch-2
fix typo issue #5005
2018-06-12 09:28:26 +02:00
923e63e5ec Streams: increment dirty counter for XGROUP SETID/DESTROY.
See issue #5005 comments.
2018-06-12 09:27:40 +02:00
fac3e8aab5 fix typo
fix [#5005](https://github.com/antirez/redis/issues/5005)
2018-06-12 08:52:18 +08:00
cec404f099 Use a less aggressive query buffer resize policy.
A user with many connections (10 thousand) on a single Redis server
reports in issue #4983 that sometimes Redis is idle becuase at the same
time many clients need to resize their query buffer according to the old
policy.

It looks like this was created by the fact that we allow the query
buffer to grow without problems to a size up to PROTO_MBULK_BIG_ARG
normally, but when the client is idle we immediately are more strict,
and a query buffer greater than 1024 bytes is already enough to trigger
the resize. So for instance if most of the clients stop at the same time
this issue should be easily triggered.

This behavior actually looks odd, and there should be only a clear limit
after we say, let's look at this query buffer to check if it's time to
resize it. This commit puts the limit at PROTO_MBULK_BIG_ARG, and the
check is performed both if compared to the peak usage the current usage
is too big, or if the client is idle.

Then when the check is performed, to waste just a few kbytes is
considered enough to proceed with the resize. This should fix the issue.
2018-06-11 17:12:28 +02:00
5bbb00fbb4 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-06-11 16:52:45 +02:00