baseplate.clients.memcache

Memcached is a high-performance in-memory key value store frequently used for caching. Pymemcache is a Python client library for it.

Example

To integrate pymemcache with your application, add the appropriate client declaration to your context configuration:

baseplate.configure_context(
   app_config,
   {
      ...
      "foo": MemcacheClient(),
      ...
   }
)

configure it in your application’s configuration file:

[app:main]

...

# required: the host and port to connect to
foo.endpoint = localhost:11211

# optional: the maximum size of the connection pool (default 2147483648)
foo.max_pool_size = 99

# optional: how long to wait for connections to establish
foo.connect_timeout = .5 seconds

# optional: how long to wait for a memcached command
foo.timeout = 100 milliseconds

...

and then use the attached PooledClient-like object in request:

def my_method(request):
    request.foo.incr("bar")

Configuration

class baseplate.clients.memcache.MemcacheClient(serializer=None, deserializer=None)[source]

Configure a Memcached client.

This is meant to be used with baseplate.Baseplate.configure_context().

See pool_from_config() for available configuration settings.

Parameters
baseplate.clients.memcache.pool_from_config(app_config, prefix='memcache.', serializer=None, deserializer=None)[source]

Make a PooledClient from a configuration dictionary.

The keys useful to pool_from_config() should be prefixed, e.g. memcache.endpoint, memcache.max_pool_size, etc. The prefix argument specifies the prefix used to filter keys. Each key is mapped to a corresponding keyword argument on the PooledClient constructor.

Supported keys:

  • endpoint (required): a string representing a host and port to connect

    to memcached service, e.g. localhost:11211 or 127.0.0.1:11211.

  • max_pool_size: an integer for the maximum pool size to use, by default

    this is 2147483648.

  • connect_timeout: how long (as

    Timespan()) to wait for a connection to memcached server. Defaults to the underlying socket default timeout.

  • timeout: how long (as Timespan()) to

    wait for calls on the socket connected to memcache. Defaults to the underlying socket default timeout.

Parameters
Return type

PooledClient

Returns

pymemcache.client.base.PooledClient

Classes

class baseplate.clients.memcache.MemcacheContextFactory(pooled_client)[source]

Memcache client context factory.

This factory will attach a MonitoredMemcacheConnection to an attribute on the RequestContext. When memcache commands are executed via this connection object, they will use connections from the provided PooledClient and automatically record diagnostic information.

Parameters

pooled_client (PooledClient) – A pooled client.

make_object_for_context(name, span)[source]

Return an object that can be added to the context object.

Parameters
  • name (str) – The name assigned to this object on the context.

  • span (Span) – The current span this object is being made for.

Return type

MonitoredMemcacheConnection

class baseplate.clients.memcache.MonitoredMemcacheConnection(context_name, server_span, pooled_client)[source]

Memcache connection that collects diagnostic information.

This connection acts like a PooledClient except that operations are wrapped with diagnostic collection. Some methods may not yet be wrapped with monitoring. Please request assistance if any needed methods are not being monitored.

Serialization/deserialization helpers

Memcache serialization/deserialization (and compression) helper methods.

Memcached can only store strings, so to store arbitrary objects we need to serialize them to strings and be able to deserialize them back to their original form.

New services should use dump_and_compress() and decompress_and_load().

Services that need to read and write to the same memcache instances as r2 should use pickle_and_compress() and decompress_and_unpickle().

baseplate.clients.memcache.lib.decompress_and_load(key, serialized, flags)[source]

Deserialize data.

This should be paired with make_dump_and_compress_fn().

Parameters
  • key (str) – the memcached key.

  • serialized (bytes) – the serialized object returned from memcached.

  • flags (int) – value stored and returned from memcached for the client to use to indicate how the value was serialized.

Return type

Any

Returns

The deserialized value.

baseplate.clients.memcache.lib.make_dump_and_compress_fn(min_compress_length=0, compress_level=1)[source]

Make a serializer.

This should be paired with decompress_and_load().

The resulting method is a chain of json.loads() and zlib compression. Values that are not JSON serializable will result in a TypeError.

Parameters
  • min_compress_length (int) – the minimum serialized string length to enable zlib compression. 0 disables compression.

  • compress_level (int) – zlib compression level. 0 disables compression and 9 is the maximum value.

Return type

Callable[[str, Any], Tuple[bytes, int]]

Returns

The serializer.

baseplate.clients.memcache.lib.decompress_and_unpickle(key, serialized, flags)[source]

Deserialize data stored by pylibmc.

Warning

This should only be used when sharing caches with applications using pylibmc (like r2). New applications should use the safer and future proofed decompress_and_load().

Parameters
  • key (str) – the memcached key.

  • serialized (bytes) – the serialized object returned from memcached.

  • flags (int) – value stored and returned from memcached for the client to use to indicate how the value was serialized.

Return type

Any

Returns

the deserialized value.

baseplate.clients.memcache.lib.make_pickle_and_compress_fn(min_compress_length=0, compress_level=1)[source]

Make a serializer compatible with pylibmc readers.

The resulting method is a chain of pickle.dumps() and zlib compression. This should be paired with decompress_and_unpickle().

Warning

This should only be used when sharing caches with applications using pylibmc (like r2). New applications should use the safer and future proofed make_dump_and_compress_fn().

Parameters
  • min_compress_length (int) – the minimum serialized string length to enable zlib compression. 0 disables compression.

  • compress_level (int) – zlib compression level. 0 disables compression and 9 is the maximum value.

Return type

Callable[[str, Any], Tuple[bytes, int]]

Returns

the serializer method.