lrucache_client - foundation package for VoltDB LRU Cache

This repo contains the core classes used by VoltDB's LRU cache implementation.

You will need the latest VoltDB client library.

voltseutil

To use the lrucache you will also need an implementation project, such as lrucache_client_cassandra . To add a new database you will find [lrucache_sdk]https://bitbucket.org/sr_mad_science/lrucache_sdk) very useful.

Both voltDB and the target database will have copies of the same table. So for example:

VoltDB:

CREATE TABLE subscriber  ( 
  s_id    BIGINT NOT NULL PRIMARY KEY
, sub_nbr VARCHAR(80) NOT NULL
, f_tinyint tinyint
, f_smallint smallint
, f_integer integer
, f_bigint bigint
, f_float float
, f_decimal decimal
, f_geography geography
, f_geography_point GEOGRAPHY_POINT
, f_varchar varchar(80)
, f_varbinary varbinary(1024)
, flush_date TIMESTAMP 
, last_use_date TIMESTAMP NOT NULL) ;

Cassandra:

CREATE TABLE vtest.subscriber  (
  s_id    BIGINT
, sub_nbr text
, f_tinyint tinyint
, f_smallint smallint
, f_integer int
, f_bigint bigint
, f_float float
, f_decimal decimal
, f_geography text
, f_geography_point text
, f_varchar varchar
, f_varbinary blob
, flush_date TIMESTAMP
, last_use_date TIMESTAMP
, primary key (s_id)) ;

Make sure you have JAR files for

• lrucache_client
• voltseutil - general utilities
• Your chosen implementation (e.g. lrucache_client_cassandra)
• Database support classes (e.g. voltseutil_cassandra)

Use LRUCacheClient

Use a rematerializer class such as CassandraRematerializerImpl that implements LruFillMissingCallbackIFace.

Note that each class will have its own config information:

Properties config = new Properties();
		
config.put(AbstractSqlRematerializer.DATABASE_HOSTNAME, "192.168.0.20");
config.put(AbstractSqlRematerializer.DATABASE_PORT, "9042");

The class name and Properties object ('m_rematConfig') are passed in to the constructor for LRUCacheClient, along with a config for a VoltDB client:

client = new LRUCacheClient(config, m_classname, schema, tableName, m_rematConfig);

Your read procedure must follow the following rules:

• Must take PK as leading arguments
• Must call 'setAppStatus' with known values for FOUND and NOT_FOUND

An example:

	private static final byte NOT_FOUND = 42;  
	private static final byte FOUND = 0;  

	private static final byte NOT_FOUND = 42;  
	private static final byte FOUND = 0;  

	...	

	public VoltTable[] run(long subscriberId, long maxSizePerPart) throws VoltAbortException {

	setAppStatusCode(FOUND);

or

	setAppStatusCode(NOT_FOUND);

The most efficient way to purge data is to add the logic to an existing procedure that is called all the time, with a read procedure being an ideal candidate.

Use the additional 'callProcedure' in LRUCacheClient to call your FOUND-enabled procedure:

Object[] procArgs = { param, limit };
GetByIdCallback callback = new GetByIdCallback(System.currentTimeMillis());

client.callProcedure(callback, theMissingDataCallback, (byte) 42, "GetById", 1,procArgs);

Behind the scenes we:

• Call 'GetById'

• Check the status code - if it's '42' we assume the PK is the first field (the '1'in the call) and that we need to ask the rematerializer to replace the missing data. We ask the other DB for a value and if it's found we insert it into VoltDB and they retry the transction

• Your callback 'callback' 's 'clientCallback' method is invoked.

Copious stats for how this is working are available from 'client.getStatsInstance().toString()'

Use:

• The testing harness : lrucache_sdk

• The test DB schema lrucache_server

• drolfe@voltdb.com