<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>AppendOnlyFileHowto: Contents</b><br> <ahref="#Append Only File HOWTO">Append Only File HOWTO</a><br> <ahref="#General Information">General Information</a><br> <ahref="#Log rewriting">Log rewriting</a><br> <ahref="#Wait... but how does this work?">Wait... but how does this work?</a><br> <ahref="#How durable is the append only file?">How durable is the append only file?</a>
<b>AppendOnlyFileHowto: Contents</b><br> <ahref="#Append Only File HOWTO">Append Only File HOWTO</a><br> <ahref="#General Information">General Information</a><br> <ahref="#Log rewriting">Log rewriting</a><br> <ahref="#Wait... but how does this work?">Wait... but how does this work?</a><br> <ahref="#How durable is the append only file?">How durable is the append only file?</a><br> <ahref="#What should I do if my Append Only File gets corrupted?">What should I do if my Append Only File gets corrupted?</a>
</div>
<h1class="wikiname">AppendOnlyFileHowto</h1>
...
...
@@ -31,8 +31,7 @@
What is the solution? To use append only file as alternative to snapshotting. How it works?<br/><br/><ul><li> It is an 1.1 only feature.</li><li> You have to turn it on editing the configuration file. Just make sure you have "appendonly yes" somewhere.</li><li> Append only files work this way: every time Redis receive a command that changes the dataset (for instance a SET or LPUSH command) it appends this command in the append only file. When you restart Redis it will first <b>re-play</b> the append only file to rebuild the state.</li></ul>
<h2><aname="Log rewriting">Log rewriting</a></h2>As you can guess... the append log file gets bigger and bigger, every time there is a new operation changing the dataset. Even if you set always the same key "mykey" to the values of "1", "2", "3", ... up to 10000000000 in the end you'll have just a single key in the dataset, just a few bytes! but how big will be the append log file? Very very big.<br/><br/>So Redis supports an interesting feature: it is able to rebuild the append log file, in background, without to stop processing client commands. The key is the command <ahref="BGREWRITEAOF.html">BGREWRITEAOF</a>. This command basically is able to use the dataset in memory in order to rewrite the shortest sequence of commands able to rebuild the exact dataset that is currently in memory.<br/><br/>So from time to time when the log gets too big, try this command. It's safe as if it fails you will not lost your old log (but you can make a backup copy given that currently 1.1 is still in beta!).<h2><aname="Wait... but how does this work?">Wait... but how does this work?</a></h2>Basically it uses the same fork() copy-on-write trick that snapshotting already uses. This is how the algorithm works:<br/><br/><ul><li> Redis forks, so now we have a child and a parent.</li><li> The child starts writing the new append log file in a temporary file.</li><li> The parent accumulates all the new changes in an in-memory buffer (but at the same time it writes the new changes in the <b>old</b> append only file, so if the rewriting fails, we are safe).</li><li> When the child finished to rewrite the file, the parent gets a signal, and append the in-memory buffer at the end of the file generated by the child.</li><li> Profit! Now Redis atomically renames the old file into the new one, and starts appending new data into the new file.</li></ul>
<h2><aname="How durable is the append only file?">How durable is the append only file?</a></h2>Check redis.conf, you can configure how many times Redis will fsync() data on disk. There are three options:<br/><br/><ul><li> Fsync() every time a new command is appended to the append log file. Very very slow, very safe.</li><li> Fsync() one time every second. Fast enough, and you can lose 1 second of data if there is a disaster.</li><li> Never fsync(), just put your data in the hands of the Operating System. The faster and unsafer method.</li></ul>
Warning: by default Redis will fsync() after <b>every command</b>! This is because the Redis authors want to ship a default configuration that is the safest pick. But the best compromise for most datasets is to fsync() one time every second.
The suggested (and default) policy is "everysec". It is both very fast and pretty safe. The "always" policy is very slow in practice, even if it was improved in Redis 2.0.0 there is no way to make fsync() faster than it is.<h2><aname="What should I do if my Append Only File gets corrupted?">What should I do if my Append Only File gets corrupted?</a></h2>It is possible that the server crashes while writing the AOF file (this still should never lead to inconsistencies) corrupting the file in a way that is no longer loadable by Redis. When this happens you can fix this problem using the following procedure:<br/><br/><ul><li> Make a backup copy of your AOF file.</li><li> Fix the original file with: ./redis-check-aof --fix <codename="code"class="python"><filename></code></li><li> Optionally use diff -u to check what is the difference between two files.</li><li> Restart the server with the fixed file.</li></ul>
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>ExpireCommand: Contents</b><br> <ahref="#EXPIRE _key_ _seconds_">EXPIRE _key_ _seconds_</a><br> <ahref="#EXPIREAT _key_ _unixtime_ (Redis >">EXPIREAT _key_ _unixtime_ (Redis ></a><br> <ahref="#PERSIST _key_">PERSIST _key_</a><br> <ahref="#How the expire is removed from a key">How the expire is removed from a key</a><br> <ahref="#Restrictions with write operations against volatile keys">Restrictions with write operations against volatile keys</a><br> <ahref="#Restrictions for write operations with volatile keys as sources">Restrictions for write operations with volatile keys as sources</a><br> <ahref="#Setting the timeout again on already volatile keys">Setting the timeout again on already volatile keys</a><br> <ahref="#Enhanced Lazy Expiration algorithm">Enhanced Lazy Expiration algorithm</a><br> <ahref="#Version 1.0">Version 1.0</a><br> <ahref="#Version 1.1">Version 1.1</a><br> <ahref="#Return value">Return value</a><br> <ahref="#FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?">FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?</a><br> <ahref="#FAQ: How this limitations were solved in Redis versions > 2.1.3?">FAQ: How this limitations were solved in Redis versions > 2.1.3?</a>
<b>ExpireCommand: Contents</b><br> <ahref="#EXPIRE _key_ _seconds_">EXPIRE _key_ _seconds_</a><br> <ahref="#EXPIREAT _key_ _unixtime_ (Redis >">EXPIREAT _key_ _unixtime_ (Redis ></a><br> <ahref="#PERSIST _key_ (Redis >">PERSIST _key_ (Redis ></a><br> <ahref="#How the expire is removed from a key">How the expire is removed from a key</a><br> <ahref="#Restrictions with write operations against volatile keys">Restrictions with write operations against volatile keys</a><br> <ahref="#Restrictions for write operations with volatile keys as sources">Restrictions for write operations with volatile keys as sources</a><br> <ahref="#Setting the timeout again on already volatile keys">Setting the timeout again on already volatile keys</a><br> <ahref="#Enhanced Lazy Expiration algorithm">Enhanced Lazy Expiration algorithm</a><br> <ahref="#Version 1.0">Version 1.0</a><br> <ahref="#Version 1.1">Version 1.1</a><br> <ahref="#Return value">Return value</a><br> <ahref="#FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?">FAQ: Can you explain better why Redis < 2.1.3 deletes keys with an EXPIRE on write operations?</a><br> <ahref="#FAQ: How this limitations were solved in Redis versions > 2.1.3?">FAQ: How this limitations were solved in Redis versions > 2.1.3?</a>
<i>Time complexity: O(1)</i><blockquote>Set a timeout on the specified key. After the timeout the key will beautomatically delete by the server. A key with an associated timeout issaid to be <i>volatile</i> in Redis terminology.</blockquote>
<i>Time complexity: O(1)</i><blockquote>Set a timeout on the specified key. After the timeout the key will beautomatically deleted by the server. A key with an associated timeout issaid to be <i>volatile</i> in Redis terminology.</blockquote>
<blockquote>Voltile keys are stored on disk like the other keys, the timeout is persistenttoo like all the other aspects of the dataset. Saving a dataset containingexpires and stopping the server does not stop the flow of time as Redisstores on disk the time when the key will no longer be available as Unixtime, and not the remaining seconds.</blockquote>
<blockquote>EXPIREAT works exctly like EXPIRE but instead to get the number of secondsrepresenting the Time To Live of the key as a second argument (that is arelative way of specifing the TTL), it takes an absolute one in the form ofa UNIX timestamp (Number of seconds elapsed since 1 Gen 1970).</blockquote>
<blockquote>EXPIREAT was introduced in order to implement <ahref="AppendOnlyFileHowto.html">the Append Only File persistence mode</a>so that EXPIRE commands are automatically translated into EXPIREAT commands for the append only file. Of course EXPIREAT can alsoused by programmers that need a way to simply specify that a given key should expire at a given time in the future.</blockquote>
...
...
@@ -78,7 +78,7 @@ redis> incr a
(integer) 1
</pre>
I set a key to the value of 100, then set an expire of 360 seconds, and then incremented the key (before the 360 timeout expired of course). The obvious result would be: 101, instead the key is set to the value of 1. Why?
There is a very important reason involving the Append Only File and Replication. Let's rework a bit hour example adding the notion of time to the mix:
There is a very important reason involving the Append Only File and Replication. Let's rework a bit our example adding the notion of time to the mix:
Imagine a Redis version that does not implement the "Delete keys with an expire set on write operation" semantic.
Running the above example with the 10 seconds pause will lead to 'a' being set to the value of 1, as it no longer exists when INCR is called 10 seconds later.<br/><br/>Instead if we drop the 10 seconds pause, the result is that 'a' is set to 101.<br/><br/>And in the practice timing changes! For instance the client may wait 10 seconds before INCR, but the sequence written in the Append Only File (and later replayed-back as fast as possible when Redis is restarted) will not have the pause. Even if we add a timestamp in the AOF, when the time difference is smaller than our timer resolution, we have a race condition.<br/><br/>The same happens with master-slave replication. Again, consider the example above: the client will use the same sequence of commands without the 10 seconds pause, but the replication link will slow down for a few seconds due to a network problem. Result? The master will contain 'a' set to 101, the slave 'a' set to 1.<br/><br/>The only way to avoid this but at the same time have reliable non time dependent timeouts on keys is to destroy volatile keys when a write operation is attempted against it.<br/><br/>After all Redis is one of the rare fully persistent databases that will give you EXPIRE. This comes to a cost :)<h2><aname="FAQ: How this limitations were solved in Redis versions > 2.1.3?">FAQ: How this limitations were solved in Redis versions > 2.1.3?</a></h2>Since Redis 2.1.3 there are no longer restrictions in the use you can do of write commands against volatile keys, still the replication and AOF file are guaranteed to be fully consistent.<br/><br/>In order to obtain a correct behavior without sacrificing consistency now when a key expires, a DEL operation is synthesized in both the AOF file and against all the attached slaves. This way the expiration process is centralized in the master instance, and there is no longer a chance of consistency errors.<br/><br/>However while the slaves while connected to a master will not expire keys independently, they'll still take the full state of the expires existing in the dataset, so when a slave is elected to a master it will be able to expire the keys independently, fully acting as a master.
<i>Time complexity: O(1)</i><blockquote>Remove the specified <i>field</i> from an hash stored at <i>key</i>.</blockquote>
<blockquote>If the <i>field</i> was present in the hash it is returned and 1 is returned, otherwise 0 is returned and no operation is performed.</blockquote>
<blockquote>If the <i>field</i> was present in the hash it is deleted and 1 is returned, otherwise 0 is returned and no operation is performed.</blockquote>
<i>Time complexity: O(N), where N is the total number of entries</i><blockquote>HKEYS returns all the fields names contained into a hash, HVALS all the associated values, while HGETALL returns both the fields and values in the form of <i>field1</i>, <i>value1</i>, <i>field2</i>, <i>value2</i>, ..., <i>fieldN</i>, <i>valueN</i>.</blockquote>
<i>Time complexity: O(N), where N is the total number of fields in the hash</i><blockquote>HKEYS returns all the fields names contained into a hash, HVALS all the associated values, while HGETALL returns both the fields and values in the form of <i>field1</i>, <i>value1</i>, <i>field2</i>, <i>value2</i>, ..., <i>fieldN</i>, <i>valueN</i>.</blockquote>
</pre>"3" is a member of the set, while "30" is not. Sets are very good in order to express relations between objects. For instance we can easily Redis Sets in order to implement tags.<br/><br/>A simple way to model this is to have, for every object you want to tag, a Set with all the IDs of the tags associated with the object, and for every tag that exists, a Set of of all the objects tagged with this tag.<br/><br/>For instance if our news ID 1000 is tagged with tag 1,2,5 and 77, we can specify the following two Sets:<br/><br/><preclass="codeblock python python python python python python python"name="code">
</pre>"3" is a member of the set, while "30" is not. Sets are very good in order to express relations between objects. For instance we can easily use Redis Sets in order to implement tags.<br/><br/>A simple way to model this is to have, for every object you want to tag, a Set with all the IDs of the tags associated with the object, and for every tag that exists, a Set of of all the objects tagged with this tag.<br/><br/>For instance if our news ID 1000 is tagged with tag 1,2,5 and 77, we can specify the following two Sets:<br/><br/><preclass="codeblock python python python python python python python"name="code">
</pre><ahref="ZremrangebyscoreCommand.html">ZREMRANGEBYSCORE</a> is not the best command name, but it can be very useful, and returns the number of removed elements.<h3><aname="Back to the reddit example">Back to the reddit example</a></h3>For the last time, back to the Reddit example. Now we have a decent plan to populate a sorted set in order to generate the home page. A sorted set can contain all the news that are not older than a few days (we remove old entries from time to time using ZREMRANGEBYSCORE). A background job gets all the elements from this sorted set, get the user votes and the time of the news, and compute the score to populate the <b>reddit.home.page</b> sorted set with the news IDs and associated scores. To show the home page we have just to perform a blazingly fast call to ZRANGE.<br/><br/>From time to time we'll remove too old news from the <b>reddit.home.page</b> sorted set as well in order for our system to work always against a limited set of news.<h3><aname="Updating the scores of a sorted set">Updating the scores of a sorted set</a></h3>Just a final note before to finish this tutorial. Sorted sets scores can be updated at any time. Just calling again ZADD against an element already included in the sorted set will update its score (and position) in O(log(N)), so sorted sets are suitable even when there are tons of updates.<br/><br/>This tutorial is in no way complete, this is just the basics to get started with Redis, read the <ahref="CommandReference.html">Command Reference</a> to discover a lot more.<br/><br/>Thanks for reading. Salvatore.
== List Commands ==<br/><br/><ul><li><ahref="RpushCommand.html">RPUSH</a></li><li><ahref="RpushCommand.html">LPUSH</a></li><li><ahref="LlenCommand.html">LLEN</a></li><li><ahref="LrangeCommand.html">LRANGE</a></li><li><ahref="LtrimCommand.html">LTRIM</a></li><li><ahref="LindexCommand.html">LINDEX</a></li><li><ahref="LsetCommand.html">LSET</a></li><li><ahref="LremCommand.html">LREM</a></li><li><ahref="LpopCommand.html">LPOP</a></li><li><ahref="LpopCommand.html">RPOP</a></li><li><ahref="BlpopCommand.html">BLPOP</a></li><li><ahref="BlpopCommand.html">BRPOP</a></li><li><ahref="RpoplpushCommand.html">RPOPLPUSH</a></li><li><ahref="SortCommand.html">SORT</a></li></ul>
== List Commands ==<br/><br/><ul><li><ahref="RpushCommand.html">RPUSH</a></li><li><ahref="RpushCommand.html">LPUSH</a></li><li><ahref="LlenCommand.html">LLEN</a></li><li><ahref="LrangeCommand.html">LRANGE</a></li><li><ahref="LtrimCommand.html">LTRIM</a></li><li><ahref="LindexCommand.html">LINDEX</a></li><li><ahref="LsetCommand.html">LSET</a></li><li><ahref="LremCommand.html">LREM</a></li><li><ahref="LpopCommand.html">LPOP</a></li><li><ahref="LpopCommand.html">RPOP</a></li><li><ahref="BlpopCommand.html">BLPOP</a></li><li><ahref="BlpopCommand.html">BRPOP</a></li><li><ahref="RpoplpushCommand.html">RPOPLPUSH</a></li><li><ahref="BrpoplpushCommand.html">BRPOPLPUSH</a></li><li><ahref="SortCommand.html">SORT</a></li></ul>
<h1><aname="EXEC or DISCARD">EXEC or DISCARD</a></h1>MULTI, EXEC, DISCARD and WATCH commands are the fundation of Redis Transactions.
<h1><aname="EXEC or DISCARD">EXEC or DISCARD</a></h1>MULTI, EXEC, DISCARD and WATCH commands are the foundation of Redis Transactions.
A Redis Transaction allows the execution of a group of Redis commands in a single
step, with two important guarantees:<br/><br/><ul><li> All the commands in a transaction are serialized and executed sequentially. It can never happen that a request issued by another client is served <b>in the middle</b> of the execution of a Redis transaction. This guarantees that the commands are executed as a single atomic operation.</li><li> Either all of the commands or none are processed. The EXEC command triggers the execution of all the commands in the transaction, so if a client loses the connection to the server in the context of a transaction before calling the MULTI command none of the operations are performed, instead if the EXEC command is called, all the operations are performed. An exception to this rule is when the Append Only File is enabled: every command that is part of a Redis transaction will log in the AOF as long as the operation is completed, so if the Redis server crashes or is killed by the system administrator in some hard way it is possible that only a partial number of operations are registered.</li></ul>
Since Redis 2.1.0, it's also possible to add a further guarantee to the above two, in the form of optimistic locking of a set of keys in a way very similar to a CAS (check and set) operation. This is documented later in this manual page.<h2><aname="Usage">Usage</a></h2>A Redis transaction is entered using the MULTI command. The command always
replies with OK. At this point the user can issue multiple commands. Instead
to execute this commands Redis will "queue" them. All the commands are
of executing these commands, Redis will "queue" them. All the commands are
executed once EXEC is called.<br/><br/>Calling DISCARD instead will flush the transaction queue and will exit
the transaction.<br/><br/>The following is an example using the Ruby client:
<preclass="codeblock python"name="code">
...
...
@@ -57,7 +57,7 @@ replies, where every element is the reply of a single command in the
transaction, in the same order the commands were queued.<br/><br/>When a Redis connection is in the context of a MULTI request, all the commands
will reply with a simple string "QUEUED" if they are correct from the
point of view of the syntax and arity (number of arguments) of the commaand.
Some command is still allowed to fail during execution time.<br/><br/>This is more clear if at protocol level: in the following example one command
Some commands are still allowed to fail during execution time.<br/><br/>This is more clear on the protocol level; In the following example one command
will fail when executed even if the syntax is right:
<preclass="codeblock python python"name="code">
Trying 127.0.0.1...
...
...
@@ -75,7 +75,7 @@ EXEC
+OK
-ERR Operation against a key holding the wrong kind of value
</pre>
MULTI returned a two elements bulk reply in witch one of this is a +OK
MULTI returned a two elements bulk reply where one is an +OK
code and one is a -ERR reply. It's up to the client lib to find a sensible
way to provide the error to the user.<br/><br/><blockquote>IMPORTANT: even when a command will raise an error, all the other commandsin the queue will be processed. Redis will NOT stop the processing ofcommands once an error is found.</blockquote>
Another example, again using the write protocol with telnet, shows how
...
...
@@ -87,9 +87,8 @@ INCR a b c
-ERR wrong number of arguments for 'incr' command
</pre>
This time due to the syntax error the "bad" INCR command is not queued
at all.<h2><aname="The DISCARD command">The DISCARD command</a></h2>DISCARD can be used in order to abort a transaction. No command will be
executed, and the state of the client is again the normal one, outside
<blockquote>of a transaction. Example using the Ruby client:</blockquote><preclass="codeblock python python python python"name="code">
at all.<h2><aname="The DISCARD command">The DISCARD command</a></h2>DISCARD can be used in order to abort a transaction. No command will be executed, and the state of the client is again the normal one, outside of a transaction. Example using the Ruby client:
<h1><aname="PUNSUBSCRIBE (unsubscribe from all patterns)">PUNSUBSCRIBE (unsubscribe from all patterns)</a></h1>
<h1><aname="PUBLISH channel message">PUBLISH channel message</a></h1>Time complexity: subscribe is O(1), unsubscribe is O(N) where N is the number of clients already subscribed to a channel, publish is O(N+M) where N is the number of clients subscribed to the receiving channel, and M is the total number of subscribed patterns (by any client). Psubscribe is O(N) where N is the number of patterns the Psubscribing client is already subscribed to. Punsubscribe is O(N+M) where N is the number of patterns the Punsubscribing client is already subscribed and M is the number of total patterns subscribed in the system (by any client).<br/><br/><blockquote>SUBSCRIBE, UNSUBSCRIBE and PUBLISH commands implement the<ahref="http://en.wikipedia.org/wiki/Publish/subscribe"target="_blank">Publish/Subscribe messaging paradigm</a> where (citing Wikipedia) senders (publishers) are not programmed to send their messages to specific receivers (subscribers). Rather, published messages are characterized into channels, without knowledge of what (if any) subscribers there may be. Subscribers express interest in one or more channels, and only receive messages that are of interest, without knowledge of what (if any) publishers there are. This decoupling of publishers and subscribers can allow for greater scalability and a more dynamic network topology.</blockquote>
<h1><aname="PUBLISH channel message">PUBLISH channel message</a></h1>Time complexity: subscribe is O(1), unsubscribe is O(N) where N is the number of clients already subscribed to a channel, publish is O(N+M) where N is the number of clients subscribed to the receiving channel, and M is the total number of subscribed patterns (by any client). Psubscribe is O(N) where N is the number of patterns the Psubscribing client is already subscribed to. Punsubscribe is O(N+M) where N is the number of patterns the Punsubscribing client is already subscribed and M is the number of total patterns subscribed in the system (by any client).<br/><br/><b>Note</b>: this commands are available starting form Redis 2.0.0<br/><br/><blockquote>SUBSCRIBE, UNSUBSCRIBE and PUBLISH commands implement the<ahref="http://en.wikipedia.org/wiki/Publish/subscribe"target="_blank">Publish/Subscribe messaging paradigm</a> where (citing Wikipedia) senders (publishers) are not programmed to send their messages to specific receivers (subscribers). Rather, published messages are characterized into channels, without knowledge of what (if any) subscribers there may be. Subscribers express interest in one or more channels, and only receive messages that are of interest, without knowledge of what (if any) publishers there are. This decoupling of publishers and subscribers can allow for greater scalability and a more dynamic network topology.</blockquote>
<blockquote>For instance in order to subscribe to the channels foo and bar the clientwill issue the SUBSCRIBE command followed by the names of the channels.</blockquote><preclass="codeblock python"name="code">
SUBSCRIBE foo bar
</pre>
...
...
@@ -106,7 +106,6 @@ The Publish command is a bulk command where the first argument is the target cla
Pieter Noordhuis provided a great example using Event-machine and Redis to create <ahref="http://chat.redis-db.com"target="_blank">a multi user high performance web chat</a>, with source code included of course!
Because all the messages received contain the original subscription causing the message delivery (the channel in the case of "message" type, and the original pattern in the case of "pmessage" type) clinet libraries may bind the original subscription to callbacks (that can be anonymous functions, blocks, function pointers, and so forth), using an hash table.<br/><br/>When a message is received an O(1) lookup can be done in order to deliver the message to the registered callback.
#sidebar <ahref="StringCommandsSidebar.html">StringCommandsSidebar</a><h1><aname="Redis String Type">Redis String Type</a></h1>Strings are the most basic Redis kind of values. Redis Strings are binary safe, this means a Redis string can contain any kind of data, for instance a JPEG image or a serialized Ruby object, and so forth.<br/><br/>A String value can be at max 1 Gigabyte in length.<br/><br/>Strings are treated as integer values by the <ahref="IncrCommand.html">INCR</a> commands family, in this respect the value of an intger is limited to a singed 64 bit value.<br/><br/>Note that the single elements contained in Redis <ahref="Lists.html">Lists</a>, <ahref="Sets.html">Sets</a> and <ahref="SortedSets.html">Sorted Sets</a>, are Redis Strings.<h1><aname="Implementation details">Implementation details</a></h1>Strings are implemented using a dynamic strings library called <codename="code"class="python">sds.c</code> (simple dynamic strings). This library caches the current length of the string, so to obtain the length of a Redis string is an O(1) operation (but currently there is no such STRLEN command. It will likely be added later).<br/><br/>Redis strings are incapsualted into Redis Objects. Redis Objects use a reference counting memory management system, so a single Redis String can be shared in different places of the dataset. This means that if you happen to use the same strings many times (especially if you have <i>object sharing</i> turned on in the configuration file) Redis will try to use the same string object instead to allocate one new every time.<br/><br/>Starting from version 1.1 Redis is also able to encode in a special way strings that are actually just numbers. Instead to save the string as an array of characters Redis will save the integer value in order to use less memory. With many datasets this can reduce the memory usage of about 30% compared to Redis 1.0.
#sidebar <ahref="StringCommandsSidebar.html">StringCommandsSidebar</a><h1><aname="Redis String Type">Redis String Type</a></h1>Strings are the most basic Redis kind of values. Redis Strings are binary safe, this means a Redis string can contain any kind of data, for instance a JPEG image or a serialized Ruby object, and so forth.<br/><br/>A String value can be at max 512 Megabytes in length.<br/><br/>Strings are treated as integer values by the <ahref="IncrCommand.html">INCR</a> commands family, in this respect the value of an intger is limited to a singed 64 bit value.<br/><br/>Note that the single elements contained in Redis <ahref="Lists.html">Lists</a>, <ahref="Sets.html">Sets</a> and <ahref="SortedSets.html">Sorted Sets</a>, are Redis Strings.<h1><aname="Implementation details">Implementation details</a></h1>Strings are implemented using a dynamic strings library called <codename="code"class="python">sds.c</code> (simple dynamic strings). This library caches the current length of the string, so to obtain the length of a Redis string is an O(1) operation (but currently there is no such STRLEN command. It will likely be added later).<br/><br/>Redis strings are incapsualted into Redis Objects. Redis Objects use a reference counting memory management system, so a single Redis String can be shared in different places of the dataset. This means that if you happen to use the same strings many times (especially if you have <i>object sharing</i> turned on in the configuration file) Redis will try to use the same string object instead to allocate one new every time.<br/><br/>Starting from version 1.1 Redis is also able to encode in a special way strings that are actually just numbers. Instead to save the string as an array of characters Redis will save the integer value in order to use less memory. With many datasets this can reduce the memory usage of about 30% compared to Redis 1.0.
<i>Time complexity: O(start+n) (with start being the start index and n the total length of the requested range). Note that the lookup part of this command is O(1) so for small strings this is actually an O(1) command.</i><blockquote>Return a subset of the string from offset <i>start</i> to offset <i>end</i>(both offsets are inclusive).Negative offsets can be used in order to provide an offset starting fromthe end of the string. So -1 means the last char, -2 the penultimate andso forth.</blockquote>
<i>Time complexity: O(start+n) (with start being the start index and n the total length of the requested range). Note that the lookup part of this command is O(1) so for small strings this is actually an O(1) command.</i><b>Warning:</b> this command was renamed into GETRANGE. SUBSTR will be taken as an alias until the next major release of Redis.<br/><br/><blockquote>Return a subset of the string from offset <i>start</i> to offset <i>end</i>(both offsets are inclusive).Negative offsets can be used in order to provide an offset starting fromthe end of the string. So -1 means the last char, -2 the penultimate andso forth.</blockquote>
<blockquote>The function handles out of range requests without raising an error, butjust limiting the resulting range to the actual length of the string.</blockquote>
</pre>This is the maximum number of threads used in order to perform I/O from/to the swap file. A good value is just to match the number of cores in your system.<br/><br/>However the special value of "0" will enable blocking VM. When VM is configured to be blocking it performs the I/O in a synchronous blocking way. This is what you can expect from blocking VM:
<ul><li> Clients accessing swapped out keys will block other clients while reading from disk, so the latency experimented by clients can be larger, especially if the disk is slow or busy and/or if there are big values swapped on disk.</li><li> The blocking VM performances are <b>overall</b> better, as there is no time lost in synchronization, spawning of threads, resuming blocked clients waiting for values.</li></ul>So if you are willing to accept an higher latency from time to time, blocking VM can be a good pick, especially if swapping happens rarely as most of your often accessed data happens to fit in your memory.<br/><br/>If instead you have a lot of swap in and swap out operations and you have many cores that you want to exploit, and in general when you don't want that clients dealing with swapped values will block other clients for a few milliseconds (or more if the swapped value is very big), then it's better to use threaded VM.<br/><br/>To experiment with your dataset and different configurations is warmly encouraged...<h1><aname="Random things to know">Random things to know</a></h1>
<h2><aname="A good place for the swap file">A good place for the swap file</a></h2>In many configurations the swap file can be fairly large, even 40GB or more.
Not all the kind of file systems are able to deal with large files in a good way, especially Mac OS X file system tends to be really lame about it.<br/><br/>The suggestion is to use Linux ext3 file system, or any other file system with good support for <b>sparse files</b>. What are sparse files?<br/><br/>Sparse files are files where a lot of the content happen to be empty. Advanced file systems like ext2, ext3, ext4, RaiserFS, Raiser4, and many others, are able to encode this files in a more efficient way and will allocate more space for the file when needed, that is, when more actual blocks of the file will be used.<br/><br/>The swap file is obviously pretty sparse, especially if the server is running since little time or it is much bigger compared to the amount of data swapped out. A file system not supporting sparse files can at some point block the Redis process while creating a very big file at once.<br/><br/>For a list of file systems supporting spare files <ahref="http://en.wikipedia.org/wiki/Comparison_of_file_systems"target="_blank">check this Wikipedia page comparing different files systems</a>.<h2><aname="Monitoring the VM">Monitoring the VM</a></h2>Once you have a Redis system with VM enabled up and running, you may be very interested in knowing how it's working: how many objects are swapped in total, the number of objects swapped and loaded every second, and so forth.<br/><br/>There is an utility that is very handy in checking how the VM is working, that is part of <ahref="http://code.google.com/p/redis-tools"target="_blank">Redis Tools</a>. This tool is called redis-stat, and using it is pretty straightforward:<br/><br/><preclass="codeblock python python python"name="code">
Not all the kind of file systems are able to deal with large files in a good way, especially Mac OS X file system tends to be really lame about it.<br/><br/>The suggestion is to use Linux ext3 file system, or any other file system with good support for <b>sparse files</b>. What are sparse files?<br/><br/>Sparse files are files where a lot of the content happen to be empty. Advanced file systems like ext2, ext3, ext4, RaiserFS, Raiser4, and many others, are able to encode this files in a more efficient way and will allocate more space for the file when needed, that is, when more actual blocks of the file will be used.<br/><br/>The swap file is obviously pretty sparse, especially if the server is running since little time or it is much bigger compared to the amount of data swapped out. A file system not supporting sparse files can at some point block the Redis process while creating a very big file at once.<br/><br/>For a list of file systems supporting spare files <ahref="http://en.wikipedia.org/wiki/Comparison_of_file_systems"target="_blank">check this Wikipedia page comparing different files systems</a>.<h2><aname="Monitoring the VM">Monitoring the VM</a></h2>Once you have a Redis system with VM enabled up and running, you may be very interested in knowing how it's working: how many objects are swapped in total, the number of objects swapped and loaded every second, and so forth.<br/><br/>There is an utility that is very handy in checking how the VM is working, that is part of <ahref="http://github.com/antirez/redis-tools"target="_blank">Redis Tools</a>. This tool is called redis-stat, and using it is pretty straightforward:<br/><br/><preclass="codeblock python python python"name="code">
</pre>The above output is about a redis-server with VM enable, around 1 million of keys inside, and a lot of simulated load using the redis-load utility.<br/><br/>As you can see from the output a number of load-in and swap-out operations are happening every second. Note that the first line reports the actual values since the server was started, while the next lines are differences compared to the previous reading.<br/><br/>If you assigned enough memory to hold your working set of data, probably you should see a lot less dramatic swapping happening, so redis-stat can be a really valuable tool in order to understand if you need to shop for RAM ;)<h2><aname="Redis with VM enabled: better .rdb files or Append Only File?">Redis with VM enabled: better .rdb files or Append Only File?</a></h2>When VM is enabled, saving and loading the database are <b>much slower</b> operations. A DB that usually loads in 2 seconds takes 13 seconds with VM enabled if the server is configured to use the smallest memory possible (that is, vm-max-memory set to 0).<br/><br/>So you probably want to switch to a configuration using the Append Only File for persistence, so that you can perform the BGREWRITEAOF from time to time.<br/><br/>It is important to note that while a BGSAVE or BGREWRITEAOF is in progress Redis does <b>not</b> swap new values on disk. The VM will be read-only while there is another child accessing it. So if you have a lot of writes while there is a child working, the memory usage may grow.<h2><aname="Using as little memory as possible">Using as little memory as possible</a></h2>An interesting setup to turn Redis into an on-disk DB with just keys in memory is setting vm-max-memory to 0. If you don't mind some latency more and poorer performances but want to use very little memory for very big values, this is a good setup.<br/><br/>In this setup you should first try setting the VM as blocking (vm-max-threads 0) as with this configuration and high traffic the number of swap in and swap out operations will be huge, and threading will consume a lot of resources compared to a simple blocking implementation.<h2><aname="VM Stability">VM Stability</a></h2>VM is still experimental code, but in the latest weeks it was tested in many ways in development environments, and even in some production environment. No bugs were noticed during this testing period. Still the more obscure bugs may happen in non controlled environments where there are setups that we are not able to reproduce for some reason.<br/><br/>In this stage you are encouraged to try VM in your development environment, and even in production if your DB is not mission critical, but for instance just a big persistent cache of data that may go away without too much problems.<br/><br/>Please report any problem you will notice to the Redis Google Group or by IRC joining the #redis IRC channel on freenode.
<i>Time complexity: O(log(N))</i><blockquote>ZRANK returns the rank of the member in the sorted set, with scores ordered from low to high. ZREVRANK returns the rank with scores ordered from high to low. When the given member does not exist in the sorted set, the special value 'nil' is returned. The returned rank (or index) of the member is 0-based for both commands.</blockquote>
<!-- This is a (PRE) block. Make sure it's left aligned or your toc title will be off. -->
<b>index: Contents</b><br> <ahref="#HOWTOs about selected features">HOWTOs about selected features</a><br> <ahref="#Hacking">Hacking</a><br> <ahref="#Videos">Videos</a>
<b>index: Contents</b><br> <ahref="#Getting started">Getting started</a><br> <ahref="#Full programming examples">Full programming examples</a><br> <ahref="#FAQs and benchmarks">FAQs and benchmarks</a><br> <ahref="#HOWTOs about selected features">HOWTOs about selected features</a><br> <ahref="#Hacking">Hacking</a><br> <ahref="#Videos">Videos</a><br> <ahref="#Recipes and books">Recipes and books</a>
</div>
<h1class="wikiname">index</h1>
...
...
@@ -26,12 +26,14 @@
</div>
<divclass="narrow">
= Redis Documentation =<br/><br/><ahref="http://pyha.ru/wiki/index.php?title=Redis:index"target="_blank">Russian Translation</a>Hello! The followings are pointers to different parts of the Redis Documentation.<br/><br/><ul><li> New! You can now <ahref="http://try.redis-db.com"target="_blank">Try Redis directly in your browser!</a>.</li><li><ahref="README.html">The README</a> is the best starting point to know more about the project.</li><li><ahref="QuickStart.html">This short Quick Start</a> provides a five minutes step-by-step istructions on how to download, compile, run and test the basic workings of a Redis server.</li><li><ahref="CommandReference.html">The command reference</a> is a description of all the Redis commands with links to command specific pages. You can also download the <ahref="http://go2.wordpress.com/?id=725X1342&site=masonoise.wordpress.com&url=http%3A%2F%2Fmasonoise.files.wordpress.com%2F2010%2F03%2Fredis-cheatsheet-v1.pdf"target="_blank">Redis Commands Cheat-Sheet</a> provided by Mason Jones (btw some command may be missing, the primary source is the wiki).</li><li><ahref="TwitterAlikeExample.html">This is a tutorial about creating a Twitter clone using *only* Redis as database, no relational DB at all is used</a>, it is a good start to understand the key-value database paradigm.</li><li><ahref="IntroductionToRedisDataTypes.html">A Fifteen Minutes Introduction to the Redis Data Types</a> explains how Redis data types work and the basic patterns of working with Redis.</li><li><ahref="http://simonwillison.net/static/2010/redis-tutorial/"target="_blank">the Simon Willison Redis Tutorial</a> is a <b>must read</b>, very good documentation where you will find a lot of real world ideas and use cases.</li><li><ahref="Features.html">The features page</a> (currently in draft) is a good start to understand the strength and limitations of Redis.</li><li><ahref="Benchmarks.html">The benchmark page</a> is about the speed performances of Redis.</li><li><ahref="FAQ.html">Our FAQ</a> contains of course some answers to common questions about Redis.</li><li><ahref="http://www.rediscookbook.org/"target="_blank">The Redis Cookbook</a> is a collaborative effort to provide some good recipe ;)</li></ul>
<h1><aname="HOWTOs about selected features">HOWTOs about selected features</a></h1><ul><li><ahref="ReplicationHowto.html">The Redis Replication HOWTO</a> is what you need to read in order to understand how Redis master <codename="code"class="python"><-></code> slave replication works.</li><li><ahref="AppendOnlyFileHowto.html">The Append Only File HOWTO</a> explains how the alternative Redis durability mode works. AOF is an alternative to snapshotting on disk from time to time (the default).</li><li><ahref="VirtualMemoryUserGuide.html">Virtual Memory User Guide</a>. A simple to understand guide about using and configuring the Redis Virtual Memory.</li></ul>
= Redis Documentation =<br/><br/><ahref="http://pyha.ru/wiki/index.php?title=Redis:index"target="_blank">Russian Translation</a>Hello! The followings are pointers to different parts of the Redis Documentation.<h1><aname="Getting started">Getting started</a></h1><ul><li> New! You can now <ahref="http://try.redis-db.com"target="_blank">Try Redis directly in your browser!</a>.</li><li><ahref="README.html">The README</a> is a good starting point to know more about the project.</li><li><ahref="QuickStart.html">This short Quick Start</a> provides a five minutes step-by-step istructions on how to download, compile, run and test the basic workings of a Redis server.</li><li><ahref="CommandReference.html">The command reference</a> is a description of all the Redis commands with links to command specific pages. You can also download the <ahref="http://go2.wordpress.com/?id=725X1342&site=masonoise.wordpress.com&url=http%3A%2F%2Fmasonoise.files.wordpress.com%2F2010%2F03%2Fredis-cheatsheet-v1.pdf"target="_blank">Redis Commands Cheat-Sheet</a> provided by Mason Jones (btw some command may be missing, the primary source is the wiki).</li><li><ahref="IntroductionToRedisDataTypes.html">A Fifteen Minutes Introduction to the Redis Data Types</a> explains how Redis data types work and the basic patterns of working with Redis.</li><li><ahref="http://simonwillison.net/static/2010/redis-tutorial/"target="_blank">the Simon Willison Redis Tutorial</a> is a <b>must read</b>, very good documentation where you will find a lot of real world ideas and use cases.</li><li><ahref="http://blog.mjrusso.com/2010/10/17/redis-from-the-ground-up.html"target="_blank">Redis from the ground up</a> is an impressive article to know more about Redis if you are a newcomer.</li><li><ahref="http://playnice.ly/blog/2010/10/19/getting-started-redis-python/"target="_blank">Getting started with Redis and Python</a></li></ul>
<h1><aname="Full programming examples">Full programming examples</a></h1><ul><li><ahref="TwitterAlikeExample.html">This is a tutorial about creating a Twitter clone using *only* Redis as database, no relational DB at all is used</a>, it is a good start to understand the key-value database paradigm.</li></ul>
<h1><aname="FAQs and benchmarks">FAQs and benchmarks</a></h1><ul><li><ahref="Benchmarks.html">The benchmark page</a> is about the speed performances of Redis.</li><li><ahref="FAQ.html">Our FAQ</a> contains of course some answers to common questions about Redis. Not very up-to-date.</li></ul>
<h1><aname="HOWTOs about selected features">HOWTOs about selected features</a></h1><ul><li><ahref="ReplicationHowto.html">The Redis Replication HOWTO</a> is what you need to read in order to understand how Redis master <codename="code"class="python"><-></code> slave replication works.</li><li><ahref="AppendOnlyFileHowto.html">The Append Only File HOWTO</a> explains how the alternative Redis durability mode works. AOF is an alternative to snapshotting on disk from time to time (the default).</li><li><ahref="VirtualMemoryUserGuide.html">Virtual Memory User Guide</a>. A simple to understand guide about using and configuring the Redis Virtual Memory.</li><li><ahref="RedisPipelining.html">Redis Pipelining Guide</a>.</li><li> Memory optimization: Full of keys, a <ahref="http://antirez.com/post/redis-weekly-update-7.html"target="_blank">blog post at antirez.com</a> showing how to use Redis 2.0 and hashes to create a setup where you can store 5 times more data in your Redis instance.</li><li><ahref="http://antirez.com/post/autocomplete-with-redis.html"target="_blank">Implementing auto complete with Redis</a>.</li></ul>
<h1><aname="Hacking">Hacking</a></h1>
<ul><li><ahref="ProtocolSpecification.html">The Protocol Specification</a> is all you need in order to implement a Redis client library for a missing language. PHP, Python, Ruby and Erlang are already supported.</li></ul>
<ul><li> Look at <ahref="RedisInternals.html">Redis Internals</a> if you are interested in the implementation details of the Redis server.</li></ul>
<h1><aname="Videos">Videos</a></h1><ul><li><ahref="http://mwrc2009.confreaks.com/13-mar-2009-19-24-redis-key-value-nirvana-ezra-zygmuntowicz.html"target="_blank">watch the Ezra Zygmuntowicz talk about Redis</a> to know the most important Redis ideas in few minutes.</li><li><ahref="http://www.ustream.tv/recorded/7855635"target="_blank">Salvatore Sanfilippo and Pieter Noordhuis at the SF Redis Meetup</a></li></ul>
<ul><li><ahref="ProtocolSpecification.html">The Protocol Specification</a> is all you need in order to implement a Redis client library for a missing language. PHP, Python, Ruby and Erlang are already supported.</li><li> Look at <ahref="RedisInternals.html">Redis Internals</a> if you are interested in the implementation details of the Redis server.</li></ul>
<h1><aname="Videos">Videos</a></h1><ul><li><ahref="http://confreaks.net/videos/62-mwrc2009-redis-key-value-nirvana"target="_blank">Ezra Zygmuntowicz at the Mountain West Ruby Conference 2009</a> to know the most important Redis ideas in few minutes.</li><li><ahref="http://www.ustream.tv/recorded/7855635"target="_blank">Salvatore Sanfilippo and Pieter Noordhuis at the SF Redis Meetup</a></li></ul>
<h1><aname="Recipes and books">Recipes and books</a></h1><ul><li><ahref="http://www.rediscookbook.org/"target="_blank">The Redis Cookbook</a> is a collaborative effort to provide some good recipe.</li><li> There is an ongoing effort to write a Redis book for O'Reilly</li></ul>
