Why our own memcache client? By Michael Barton
python-memcached doesn’t use consistent hashing, so adding or removing a memcache server from the pool invalidates a huge percentage of cached items.
If you keep a pool of python-memcached client objects, each client object has its own connection to every memcached server, only one of which is ever in use. So you wind up with n * m open sockets and almost all of them idle. This client effectively has a pool for each server, so the number of backend connections is hopefully greatly reduced.
python-memcache uses pickle to store things, and there was already a huge stink about Swift using pickles in memcache (http://osvdb.org/show/osvdb/86581). That seemed sort of unfair, since nova and keystone and everyone else use pickles for memcache too, but it’s hidden behind a “standard” library. But changing would be a security regression at this point.
Also, pylibmc wouldn’t work for us because it needs to use python sockets in order to play nice with eventlet.
Lucid comes with memcached: v1.4.2. Protocol documentation for that version is at:
http://github.com/memcached/memcached/blob/1.4.2/doc/protocol.txt
Bases: eventlet.pools.Pool
Connection pool for Memcache Connections
The server parameter can be a hostname, an IPv4 address, or an IPv6 address with an optional port. If an IPv6 address is specified it must be enclosed in [], like [::1] or [::1]:11211. This follows the accepted prescription for IPv6 host literals.
Examples:
memcache.local:11211
127.0.0.1:11211
[::1]:11211
[::1]
Bases: object
Simple, consistent-hashed memcache client.
Decrements a key which has a numeric value by delta. Calls incr with -delta.
Parameters: |
|
---|---|
Returns: | result of decrementing |
Raises MemcacheConnectionError: | |
Deletes a key/value pair from memcache.
Parameters: | key – key to be deleted |
---|
Gets the object specified by key. It will also unserialize the object before returning if it is serialized in memcache with JSON, or if it is pickled and unpickling is allowed.
Parameters: | key – key |
---|---|
Returns: | value of the key in memcache |
Gets multiple values from memcache for the given keys.
Parameters: |
|
---|---|
Returns: | list of values |
Increments a key which has a numeric value by delta. If the key can’t be found, it’s added as delta or 0 if delta < 0. If passed a negative number, will use memcached’s decr. Returns the int stored in memcached Note: The data memcached stores as the result of incr/decr is an unsigned int. decr’s that result in a number below 0 are stored as 0.
Parameters: |
|
---|---|
Returns: | result of incrementing |
Raises MemcacheConnectionError: | |
Set a key/value pair in memcache
Parameters: |
|
---|---|
Min_compress_len: | |
minimum compress length, this parameter was added to keep the signature compatible with python-memcached interface. This implementation ignores it. |
Sets multiple key/value pairs in memcache.
Parameters: |
|
---|---|
Min_compress_len: | |
minimum compress length, this parameter was added to keep the signature compatible with python-memcached interface. This implementation ignores it |
Sanitize a timeout value to use an absolute expiration time if the delta is greater than 30 days (in seconds). Note that the memcached server translates negative values to mean a delta of 30 days in seconds (and 1 additional second), client beware.