Using Redis cache backend and session storage in Magento

Combined with the power of Zend Framework, Magento currently supports many cache backends with file system, APC and Memcached being the most widely used. Every supported cache backend brings it’s own set of upsides and downsides, but since there’s a lot to choose from, one could easily assume that as far as cache backends are concerned, Magento is very well covered. To some degree this might be true, but when you seriously up the number of requests, none of the cache backends scale well. Besides poor scaling, some of the cache backend implementations also suffer from serious limitations like not having support for grouping of related cache entries, a.k.a. tagging. With this in mind, and in order to provide support for stable and scalable cache backend, latest versions of Magento are turning to Redis key-value store. In this article I’ll do my best to describe process of implementing Redis as cache and session backend with recent versions of Magento Community and Enterprise Edition.

Installing Redis

As far as Magento is concerned, Redis consists of two key parts, first being the server it self and second PhpRedis extension that allows PHP to communicate with Redis. I’ll show you how to install both parts on top of both server and workstation Linux configurations. For the sake of keeping this article concise, I’ll make the assumption that the server configuration is being powered by CentOS in it’s version 6, and that the developer configuration is using latest Ubuntu version at the time of writing, Ubuntu 13.10. Although following procedure doesn’t cover every Linux distribution out there, it does generally cover two main camps, RedHat and Debian based Linux distributions.

Server configuration – CentOS 6.0 or later

Easiest way to install Redis on RedHat based Linux distributions like CentOS is to use EPEL (Extra Packages for Enterprise Linux) repository. This comes down to installing one package and importing EPEL repository GPG key. I must point out that there’s high probability that your CentOS server already has this repository added (you can use yum repolist to test). First step, download package that adds the repository to yum package manager. If you’re using i386 kernel and environment (command uname -r to test) use following line:

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

On the other hand if you are using x86_64 kernel, you should execute following line from your shell:

wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Next step is to install repository package and import GPG key used by yum to authenticate packages (root access required):

rpm -Uvh epel-release-6*.rpm
rpm --import http://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-6

What’s left to do is install Redis server and PhpRedis extension trough yum, enable the Redis server to be started at boot, start Redis service and restart Apache service:

su
yum install redis php-pecl-redis
chkconfig redis on
service redis start
service httpd restart

Workstation configuration – Ubuntu 13.10 or later

Good news, Debian has Redis server in it’s repository since Wheezy (in previous Squeeze version you had to use backports repository), so we do not have to add new repositories to our Ubuntu system. Bad news, there is no PhpRedis package in any of the repositories, so we must compile the thing with a little help from our good old friend PECL. So let’s proceed with installing the Redis server (administrative user access required):

sudo -i
apt-get install redis-server php5-dev php5-cli php-pear

Next, we install PhpRedis extension trough PECL, inform PHP about it’s existence, enable the PHP extension and restart Apache service:

pecl install redis
echo "extension=redis.so" > /etc/php5/mods-available/redis.ini
php5enmod redis
service apache2 restart

Verifying Redis installation

At this point we should be ready to test basic operations of Redis as well as it’s PHP extension.

Verifying Redis server

To test Redis server, we simply ping it using:

redis-cli ping

If you have received your PONG, you can proceed.

Verifying PhpRedis extension

To see if Redis extension is being loaded by PHP we can grep the output of php -m shell command:

php -m | grep redis

We expect string redis as a result.

Redis shell tools

Redis comes with a few tools, most useful one being it’s client software accessible trough following shell command:

redis-cli

After starting Redis client, you are presented with Redis console where you can enter supported commands. Some of the commands you’ll be using most of the time are definitely:

FLUSHALL – clear all databases
SELECT # – select database under index #
FLUSHDB – empty currently selected database
KEYS * – list all keys from currently selected

Good thing to point out regarding Redis is that databases are created on demand and that they can only be indexed by integers. Default number of databases is 16, but this and many other parameters can be configured trough Redis main configuration file located at /etc/redis/redis.conf for CentOS and /etc/redis/redis.conf for Debian based distributions.

Redis support across different Magento versions

Since Redis joined Magento cache and session backends not so long ago after being community project for a while, level of support Magento provides out the box varies with Magento version.

Here are the links to community projects that brought Redis support to Magento:

In case your Magento version is missing support for Redis cache or session backend (or both), you can freely download and use community extensions. Here’s short overview of Redis cache and session backends out the box support for recent Magento CE and EE versions.

Magento CE >= 1.7.0.0 and < 1.8.0.0

Session storage – not included
Cache backend – not included, after installation available as Cm_Cache_Backend_Redis

Magento CE >= 1.8.0.0

Session storage – included
Cache backend – included, available as Mage_Cache_Backend_Redis

Magento EE >= 1.13.0.0 and < 1.13.1.0

Session storage – not included
Cache backend – included, available as Mage_Cache_Backend_Redis

Magento EE >= 1.13.1.0

Session storage – included
Cache backend – included, available as Mage_Cache_Backend_Redis

Using Redis cache backend

When placed inside app/etc/local.xml, following code configures Redis cache backend to use database with index 0 to hold cache:

<config>
    <global>
 
        <!-- other configuration nodes -->
 
        <cache>
          <backend>CACHE_BACKEND_CLASS_NAME</backend>
          <backend_options>
            <server>127.0.0.1</server>              <!-- or absolute path to unix socket -->
            <port>6379</port>
            <persistent></persistent>               <!-- Specify a unique string like "cache-db0" to enable persistent connections. -->
            <database>0</database>
            <password></password>
            <force_standalone>0</force_standalone>  <!-- 0 for phpredis, 1 for standalone PHP -->
            <connect_retries>1</connect_retries>    <!-- Reduces errors due to random connection failures -->
            <read_timeout>10</read_timeout>         <!-- Set read timeout duration -->
            <automatic_cleaning_factor>0</automatic_cleaning_factor> <!-- Disabled by default -->
            <compress_data>1</compress_data>        <!-- 0-9 for compression level, recommended: 0 or 1 -->
            <compress_tags>1</compress_tags>        <!-- 0-9 for compression level, recommended: 0 or 1 -->
            <compress_threshold>20480</compress_threshold>  <!-- Strings below this size will not be compressed -->
            <compression_lib>gzip</compression_lib> <!-- Supports gzip, lzf and snappy -->
          </backend_options>
        </cache>
 
        <!-- other configuration nodes -->
 
    </global>
</config>

Replace CACHE_BACKEND_CLASS_NAME with relevant cache backend class name as specified under “Redis support across different Magento versions” heading.

After enabling Redis as cache backend, var/cache directory of your Magento installation can be emptied and should stay empty. If you are using Magento Enterprise edition and cache backend that supports tagging for full page cache, similar thing goes for core_cache/core_cache_tag tables from your Magento database because in this case these tables can be truncated and should stay empty. To check is Redis backend being used for storing cache you can run redis-cli tool and query database 1 using following Redis commands:

SELECT 0
KEYS *

Using Redis as FPC backend (Magento Enterprise Edition only)

When placed inside app/etc/local.xml, following code configures Redis cache backend to use database with index 1 to hold full page cache:

<config>
    <global>
 
        <!-- other configuration nodes -->
 
        <full_page_cache>
            <backend>CACHE_BACKEND_CLASS_NAME</backend>
            <backend_options>
              <server>127.0.0.1</server>  <!-- or absolute path to unix socket -->
              <port>6379</port>
              <persistent></persistent>   <!-- Specify a unique string like "cache-db0" to enable persistent connections. -->
              <database>1</database>      <!-- Separate database 1 to keep FPC separately -->
              <password></password>
              <force_standalone>0</force_standalone>  <!-- 0 for phpredis, 1 for standalone PHP -->
              <connect_retries>1</connect_retries>    <!-- Reduces errors due to random connection failures -->
              <lifetimelimit>57600</lifetimelimit>    <!-- 16 hours of lifetime for cache record -->
              <compress_data>0</compress_data>        <!-- DISABLE compression for EE FPC since it already uses compression -->
            </backend_options>
        </full_page_cache>
 
        <!-- other configuration nodes -->
 
    </global>
</config>

Replace CACHE_BACKEND_CLASS_NAME with relevant cache backend class name as specified under “Redis support across different Magento versions” heading.

After enabling Redis as full page cache backend, var/full_page_cache directory of your Magento Enterprise Edition installation can be emptied and should stay empty. If you are using cache backend that supports tagging for cache, similar thing goes for core_cache/core_cache_tag tables from your Magento database because in this case these tables can be truncated and should stay empty. To check is Redis backend being used for storing full page cache you can run redis-cli tool and query database 1 using following Redis commands:

SELECT 1
KEYS *

Cache tags cleanup cron

Redis solves most problems that affect other cache backends, but cleaning up old cache tags manually is still required. Luckily, Magento provides garbage cleanup PHP script designed to be run by cron daily to remedy this issue.

Step one, clone the GitHub repository that hosts garbage cleanup PHP script on path accessible to user that will be running this script from it’s crontab (/some/path/ in this example):

su
cd /some/path/
git clone git@github.com:samm-git/cm_redis_tools.git
cd cm_redis_tools
git submodule update --init --recursive

Next step, edit crontab as this user:

crontab -e

and add this script to be started every night:

30 2 * * * /usr/bin/php /some/path/cm_redis_tools/rediscli.php -s 127.0.0.1 -p 6379 -d 0,1

where -s parameter is Redis server host, -p port and -d coma separated list of databases to clean. If you remember, we’ve used 0 for cache and 1 for full page cache so adjust as necessary. Also, don’t forget to adjust /some/path/ into whatever path you’ve chosen before cloning the repository.

Using Redis session storage

As first step you should make sure that CM_RedisSession module is active by editing app/etc/modules/CM_RedisSession.xml, because this isn’t the case in Magento versions that support Redis as session storage out the box.

Another thing worth mentioning is that following code snippet sets session storage to db. Reason for this is that Redis session works by rewriting Mage_Core_Model_Mysql4_Session.

With this in mind, here’s code that configures Redis session model to use database with index 2, when placed inside app/etc/local.xml.

<config>
    <global>
 
        <!-- other configuration nodes -->
        <session_save>db</session_save>
        <redis_session>                       <!-- All options seen here are the defaults -->
            <host>127.0.0.1</host>            <!-- Specify an absolute path if using a unix socket -->
            <port>6379</port>
            <password></password>             <!-- Specify if your Redis server requires authentication -->
            <timeout>2.5</timeout>            <!-- This is the Redis connection timeout, not the locking timeout -->
            <persistent></persistent>         <!-- Specify unique string to enable persistent connections. -->
            <db>2</db>                        <!-- Redis database number; protection from accidental loss is improved by using a unique DB number for sessions -->
            <compression_threshold>2048</compression_threshold>  <!-- Set to 0 to disable compression (recommended when suhosin.session.encrypt=on); known bug with strings over 64k: https://github.com/colinmollenhour/Cm_Cache_Backend_Redis/issues/18 -->
            <compression_lib>gzip</compression_lib>              <!-- gzip, lzf or snappy -->
            <log_level>4</log_level>               <!-- 0 (emergency: system is unusable), 4 (warning; additional information, recommended), 5 (notice: normal but significant condition), 6 (info: informational messages), 7 (debug: the most information for development/testing) -->
            <max_concurrency>6</max_concurrency>                 <!-- maximum number of processes that can wait for a lock on one session; for large production clusters, set this to at least 10% of the number of PHP processes -->
            <break_after_frontend>5</break_after_frontend>       <!-- seconds to wait for a session lock in the frontend; not as critical as admin -->
            <break_after_adminhtml>30</break_after_adminhtml>
            <bot_lifetime>7200</bot_lifetime>                    <!-- Bots get shorter session lifetimes. 0 to disable -->
        </redis_session>
 
        <!-- other configuration nodes -->
 
    </global>
</config>

Migrate sessions to Redis

If you are implementing Redis on a live store, you should import current sessions before you proceed. This should be done with your store in maintenance mode. Cm_RedisSession project provides two scripts for this purpose. Which one you use depends on whether you’ve been using files or db session backend before implementing Redis.

Here’s procedure if you were previously using files for storing session data:

cd /path/to/magento/root
touch maintenance.flag
sleep 10
wget https://raw2.github.com/colinmollenhour/Cm_RedisSession/master/migrateSessions.php
php migrateSessions.php -y
rm maintenance.flag

And here’s procedure if you were previously using database for storing session data:

cd /path/to/magento/root
touch maintenance.flag
sleep 10
wget https://raw2.github.com/colinmollenhour/Cm_RedisSession/master/migrateSessions_mysql_redis.php
php migrateSessions_mysql_redis.php -y
rm maintenance.flag

After enabling Redis as session backend, var/sessions directory of your Magento installation can be emptied and should stay empty. Similar thing goes for core_session table from your Magento database because in this case this table can be truncated and should stay empty. To check is Redis model being used for storing sessions you can run redis-cli tool and query database 2 using following Redis commands:

SELECT 2
KEYS *

That’s all there is to implementing Redis with Magento. I hope you found this article useful and until next time, I wish you happy coding! In case you’re going to need any help regarding your Magento development, we would be happy to assist.