2General

Want to know more about us? Visit 2general.com »

Changing Django cache backend between test cases

It’s a good practice to run tests for a Django project with a dummy cache backend. This eliminates side effects of one test from affecting the results of other tests.

Here’s how to activate the dummy backend in a Django settings file:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.dummy.DummyCache'
    }
}

However, sometimes it’s also necessary to test how an application uses the cache. In this article, we’ll show how to replace the dummy cache with a real cache backend separately for individual test cases.

We’ll use the trivial myapp/mymodule.py below as example code under test:

1
2
3
4
5
from django.core.cache import cache


def myfunc(key):
    cache.set(key, 'myvalue')

The obvious (but alas, wrong) way

One solution is to create a separate test suite with its own test settings. The drawback is that it must be run separately even if the tests belong logically together with the rest of the tests.

Django 1.4 makes it easy to override settings on a test-by-test basis (see Overriding settings in the testing section of Django’s documentation). It feels natural that the @override_settings decorator should do the trick of enabling caching separately for individual test cases:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
from django.core import cache
from django.test import TestCase
from django.test.utils import override_settings
from myapp import mymodule


class RealCacheTestCase(TestCase):

    # this does not work:
    @override_settings(
        CACHES={
            'default': {
                'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'
            }
        }
    )
    def test_with_cache(self):
        mymodule.myfunc('mykey')  # tries to set 'mykey' in the cache
        self.assertEqual('myvalue', cache.cache.get('mykey'))  # FAILS

Don’t do that. It won’t work.

The django.core.cache module has most probably been already imported when entering the test case. The global cache object is created at import time, so overriding settings after the fact won’t change which cache backend is used.

How to actually make it work

It turns out that it’s not even necessary to override Django’s settings to switch cache backends. Since the module being tested holds a reference to the global cache object, a new cache object can be created on the fly and injected into the module using a mock library. In the example below, Michael Foord’s Mock is used for this purpose.

This myapp/tests.py file changes the dummy cache backend to the local memory cache backend on the fly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
from django.core import cache
from django.test import TestCase
from mock import patch

from myapp import mymodule


class MyTestCase(TestCase):
    def test_caching(self):
        locmem_cache = cache.get_cache(
            'django.core.cache.backends.locmem.LocMemCache')
        locmem_cache.clear()
        with patch.object(mymodule, 'cache', locmem_cache):

            mymodule.myfunc('mykey')  # sets 'mykey' in the cache

        self.assertEqual('myvalue', mymodule.cache.get('mykey'))
        # or:
        self.assertEqual('myvalue', locmem_cache.get('mykey'))
Note that only mymodule.cache is patched, while in tests.py, the reference cache.cache still points to the original dummy cache. Be careful to verify cached values with mymodule.cache.get() instead of cache.cache.get() (or just use locmem_cache.get() instead).

If multiple tests use the cache, patching can be refactored into separate setup and teardown:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
class MyTestCase(TestCase):
    def setUp(self):
        self.locmem_cache = cache.get_cache(
            'django.core.cache.backends.locmem.LocMemCache')
        self.locmem_cache.clear()
        self.patch = patch.object(mymodule, 'cache', self.locmem_cache)
        self.patch.start()

    def tearDown(self):
        self.patch.stop()

    def test_caching(self):
        mymodule.myfunc('mykey')  # sets 'mykey' in the cache

        self.assertEqual('myvalue', mymodule.cache.get('mykey'))
        self.assertEqual(None, mymodule.cache.get('otherkey'))

    def test_caching_again(self):
        mymodule.myfunc('otherkey')  # sets 'otherkey' in the cache

        self.assertEqual(None, mymodule.cache.get('mykey'))
        self.assertEqual('myvalue', mymodule.cache.get('otherkey'))
(Here, too, you may substitute mymodule.cache.get() with locmem_cache.get())

Note that it’s actually essential to call locmem_cache.clear() since all LocMemCache instances share a common cache dictionary unless a different LOCATION setting is specified.

Packaging it all up

For convenience, we can encapsulate the temporary cache replacement as a function which can be used both as a decorator and a context manager. Here we use to our advantage the clever mechanisms already in place in Mock’s patch machinery:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
## test_utils.py
from django.core import cache
from mock import patch


def patch_local_cache(module, varname):
    locmem_cache = cache.get_cache(
        'django.core.cache.backends.locmem.LocMemCache')
    locmem_cache.clear()
    return patch.object(module, varname, locmem_cache)

The test case can now be expressed more succintly using patch_local_cache as either a decorator or a context manager:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
## myapp/tests.py
from django.core import cache
from django.test import TestCase
from test_utils import patch_local_cache

from myapp import mymodule


class MyTestCase(TestCase):
    @patch_local_cache(mymodule, 'cache')
    def test_decorator(self):
        mymodule.myfunc('mykey')  # sets 'mykey' in the cache

        self.assertEqual('myvalue', mymodule.cache.get('mykey'))

    def test_context_manager(self, locmem_cache):
        with patch_local_cache(mymodule, 'cache'):
            mymodule.myfunc('mykey')  # sets 'mykey' in the cache

            self.assertEqual('myvalue', mymodule.cache.get('mykey'))
Here locmem_cache isn’t available to the test case, so we have to use mymodule.cache.get().

So there you have it, another handy tool to help you reach 100% test coverage :)