Commit Graph

155 Commits

Author SHA1 Message Date
78f35f8d2c Memory related subcommands of DEBUG moved to MEMORY. 2016-09-16 10:26:23 +02:00
123891dbbf Group MEMORY command related APIs together in the source code. 2016-09-16 10:12:04 +02:00
adcfb77b5b objectComputeSize(): skiplist nodes have different sizes.
The size of the node depends on the node level, however it is not stored
into the node itself, is an implicit information, so we use
zmalloc_size() in order to compute the sorted set size.
2016-09-15 17:43:13 +02:00
e9629e148b MEMORY command: HELP + dataset percentage (like in INFO). 2016-09-15 17:33:16 +02:00
5443726d4d MEMORY USAGE: SAMPLES option added + fixes to size computation.
The new SAMPLES option is added, defaulting to 5, and with 0 being a
special value to scan the whole set of elements.

Fixes to the object size computation were made since the original PR
assumed data structures still contaning robj structures, while now after
the lazyfree changes, are all SDS strings.
2016-09-15 15:25:14 +02:00
bf2624ea99 C struct memoh renamed redisMemOverhead. API prototypes added. 2016-09-15 09:44:07 +02:00
be5439bde3 MEMORY OVERHEAD refactored into a generic API. 2016-09-15 09:37:55 +02:00
8c84c962cf MEMORY OVERHEAD implemented (using Oran Agra initial implementation).
This code was extracted from @oranagra PR #3223 and modified in order
to provide only certain amounts of information compared to the original
code. It was also moved from DEBUG to the newly introduced MEMORY
command. Thanks to Oran for the implementation and the PR.

It implements detailed memory usage stats that can be useful in both
provisioning and troubleshooting memory usage in Redis.
2016-09-13 17:39:25 +02:00
89dec6921d objectComputeSize(): estimate collections sampling N elements.
For most tasks, we need the memory estimation to be O(1) by default.
This commit also implements an initial MEMORY command.
Note that objectComputeSize() takes the number of samples to check as
argument, so MEMORY should be able to get the sample size as option
to make precision VS CPU tradeoff tunable.

Related to: PR #3223.
2016-09-13 10:28:23 +02:00
8c24325f8f Adding objectComputeSize() function. 2016-09-12 16:36:59 +02:00
e0582b3598 Fix maxmemory shared integer check bug introduced with LFU. 2016-07-21 11:14:18 +02:00
a8e2d0849e LFU: Initial naive eviction cycle.
It is possible to get better results by using the pool like in the LRU
case. Also from tests during the morning I believe the current
implementation has issues in the frequency decay function that should
decrease the counter at periodic intervals.
2016-07-18 13:50:19 +02:00
5d07984c5d LFU: Redis object level implementation.
Implementation of LFU maxmemory policy for anything related to Redis
objects. Still no actual eviction implemented.
2016-07-15 12:12:58 +02:00
b46239e58b Expire and LRU related code moved into different files. 2016-07-06 15:24:06 +02:00
23791828f1 getLongLongFromObject: use string2ll() instead of strict_strtoll().
strict_strtoll() has a bug that reports the empty string as ok and
parses it as zero.

Apparently nobody ever replaced this old call with the faster/saner
string2ll() which is used otherwise in the rest of the Redis core.

This commit close #3333.
2016-07-06 11:43:33 +02:00
28ea585fce Merge pull request #3336 from yossigo/create_string_from_string
Add RedisModule_CreateStringFromString().
2016-06-23 16:16:28 +02:00
f648c5a70c A string with 21 chars is not representable as a 64-bit integer. 2016-06-23 19:53:56 +08:00
61172ed01e Add RedisModule_CreateStringFromString(). 2016-06-22 21:02:40 +03:00
8f3a4df775 Use const in Redis Module API where possible. 2016-06-20 23:08:06 +03:00
226f679651 Merge pull request #3283 from ideal/unstable
fix mistake in comment in object.c
2016-06-15 15:53:19 +02:00
8ec28002be Modules: support for modules native data types. 2016-06-03 18:14:04 +02:00
4067132092 fix mistake comment in object.c 2016-05-30 16:57:36 +08:00
974514b936 Lazyfree: Hash converted to use plain SDS WIP 4. 2015-10-01 13:02:25 +02:00
777396aeba Lazyfree: Hash converted to use plain SDS WIP 2. 2015-10-01 13:02:24 +02:00
1c247556c6 Lazyfree: Hash converted to use plain SDS WIP 1. 2015-10-01 13:02:24 +02:00
86d48efbfd Lazyfree: Convert Sets to use plains SDS (several commits squashed). 2015-10-01 13:02:24 +02:00
0c05436cef Lazyfree: a first implementation of non blocking DEL. 2015-10-01 13:00:19 +02:00
32f80e2f1b RDMF: More consistent define names. 2015-07-27 14:37:58 +02:00
40eb548a80 RDMF: REDIS_OK REDIS_ERR -> C_OK C_ERR. 2015-07-26 23:17:55 +02:00
2d9e3eb107 RDMF: redisAssert -> serverAssert. 2015-07-26 15:29:53 +02:00
14ff572482 RDMF: OBJ_ macros for object related stuff. 2015-07-26 15:28:00 +02:00
554bd0e7bd RDMF: use client instead of redisClient, like Disque. 2015-07-26 15:20:52 +02:00
cef054e868 RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
f15df8ba5d sds size classes - memory optimization 2015-07-14 17:17:06 +02:00
8e219224b9 CONFIG refactoring: configEnum abstraction.
Still many things to convert inside config.c in the next commits.
Some const safety in String objects creation and addReply() family
functions.
2015-03-11 17:00:13 +01:00
4f56f035a7 String: use new sdigits10() API in stringObjectLen().
Should be much faster, and regardless, the code is more obvious now
compared to generating a string just to get the return value of the
ll2stirng() function.
2015-02-27 16:09:17 +01:00
5e362b84ab Add quicklist implementation
This replaces individual ziplist vs. linkedlist representations
for Redis list operations.

Big thanks for all the reviews and feedback from everybody in
https://github.com/antirez/redis/pull/2143
2015-01-02 11:16:08 -05:00
3632026210 Handle infinite explicitly in createStringObjectFromLongLong(). 2014-12-03 10:37:01 +01:00
92c5ab4029 Use exp format and more precision output for ZSCAN.
Ref: issue #2175
2014-12-02 18:20:09 +01:00
b3c042cdb2 tryObjectEncoding(): use shared objects with maxmemory and non-LRU policy.
In order to make sure every object has its own private LRU counter, when
maxmemory is enabled tryObjectEncoding() does not use the pool of shared
integers. However when the policy is not LRU-based, it does not make
sense to do so, and it is much better to save memory using shared
integers.
2014-07-18 10:09:51 +02:00
ce8a68b1fd Fix OBJECT arity
Previously, the command definition for the OBJECT command specified
a minimum of two args (and that it was variadic), which meant that
if you sent this:

OBJECT foo

When cluster was enabled, it would result in an assertion/SEGFAULT
when Redis was attempting to extract keys.

It appears that OBJECT is not variadic, and only ever takes 3 args.

https://gist.github.com/michael-grunder/25960ce1508396d0d36a
2014-07-11 13:03:26 -07:00
7c4decb101 Fix lack of strtold under Cygwin
Renaming strtold to strtod then casting
the result is the standard way of dealing with
no strtold in Cygwin.
2014-05-12 11:11:09 -04:00
0b0f872f3f REDIS_ENCODING_EMBSTR_SIZE_LIMIT set to 39.
The new value is the limit for the robj + SDS header + string +
null-term to stay inside the 64 bytes Jemalloc arena in 64 bits
systems.
2014-05-07 17:05:09 +02:00
1c12bcbcfb tryObjectEncoding() refactoring.
We also avoid to re-create an object that is already in EMBSTR encoding.
2014-04-04 17:25:35 +02:00
543ede03f2 String value unsharing refactored into proper function.
All the Redis functions that need to modify the string value of a key in
a destructive way (APPEND, SETBIT, SETRANGE, ...) require to make the
object unshared (if refcount > 1) and encoded in raw format (if encoding
is not already REDIS_ENCODING_RAW).

This was cut & pasted many times in multiple places of the code. This
commit puts the small logic needed into a function called
dbUnshareStringValue().
2014-03-30 18:32:17 +02:00
78782ed59f Use LRU_CLOCK() instead of function getLRUClock()
lookupKey() uses LRU_CLOCK(), so it seems object
creation should use LRU_CLOCK() too.
2014-03-24 14:39:26 -04:00
4d2e8fa189 Use getLRUClock() instead of server.lruclock to create objects.
Thanks to Matt Stancliff for noticing this error. It was in the original
code but somehow I managed to remove the change from the commit...
2014-03-21 09:08:20 +01:00
6d5790d682 Fix OBJECT IDLETIME return value converting to seconds.
estimateObjectIdleTime() returns a value in milliseconds now, so we need
to scale the output of OBJECT IDLETIME to seconds.
2014-03-20 11:55:18 +01:00
ad6b0f70b2 Obtain LRU clock in a resolution dependent way.
For testing purposes it is handy to have a very high resolution of the
LRU clock, so that it is possible to experiment with scripts running in
just a few seconds how the eviction algorithms works.

This commit allows Redis to use the cached LRU clock, or a value
computed on demand, depending on the resolution. So normally we have the
good performance of a precomputed value, and a clock that wraps in many
days using the normal resolution, but if needed, changing a define will
switch behavior to an high resolution LRU clock.
2014-03-20 11:47:12 +01:00
d77e231682 Specify LRU resolution in milliseconds. 2014-03-20 11:33:25 +01:00