Terracotta Tuning: Part 1

One of the major features of Terracotta 3.0 was the support for Terracotta Striped Array. This removed the limitation of having only 1 Active server in a cluster. This would enable achieve linear scalability in the throughput on increasing the server count.

To get the best out of the Active servers present in the cluster, the load should be distributed among the different Active servers. So the aim of this blog is to show with an example as to how we can distribute the most used shared objects equally among the different mirror groups in the Terracotta cluster if not already happening.

How the objects are distributed?
To begin with Terracotta has the concept of the transactions, so when we try to create some shared objects it happens within the scope of these transactions. Currently what happens is that the default round robin strategy for object creation makes all the objects created within a single transaction go to a single server. That is all the objects created within a single transaction will reside only on one of the mirror groups.

When will this problem arise and how to verify?
So this issue which I am trying to address in the blog will only arise if the most used shared objects are created within a single transaction. To verify this one can launch the dev console present in the bin bin directory of Terracotta installation. Then the object browser will tell the gid(group id) in which the object is residing. This has been explained in a little more detail later in the blog.

However there is a very simple workaround for this issue. Let us take an example here:
Consider we have a map like Store which supports the following operations:
1. put(key, value)
2. get(key)
3. remove(key)
The way this Store works is similar to that as a Map would. Thus when a put operation is performed then an appropriate Bucket is selected based on the hash code. Each Bucket hold a pointer to the head of a linked list and the new key is inserted as the head of the list. So this test program aims at sharing an instance of such a store. The test class has an instance store which will shared as a root in the cluster using Terracotta:

private final Store store = new Store();

The Test class has a run method which just does some puts and gets on this store.
So when this store will be initialized, then all buckets will be created in a single transaction and hence all of the buckets will reside on a single mirror group.

public Store() {
buckets = new Bucket[NO_OF_BUCEKTS];
for (int i = 0; i < NO_OF_BUCEKTS; i++) {
buckets[i] = new Bucket();
}
}

So if we start 4 mirror groups and start a client. Then all the buckets will be formed on a single mirror group. We can confirm this by launching dev-console provided by Terracotta. The dev console will be present in the bin directory of Terracotta installation.
On launching dev-console and clicking on the object browser in the left panel, you will see something like shown below:

So if see on the right panel, you will see something like:
0(blog.store.Bucket() [@11011, gid=0]
This implies that this particular Bucket is being stored in a mirror group with group id = 0.
You will also notice that all the Buckets have group id = 0.

This kind of scenario won't be able to extract the best out of the Terracotta striped Array.
So to remove this problem we just need to make Bucket implement a marker interface. When Terracotta encounters this marker interface then it automatically takes care of uniformly distributing this objects of this class across the different mirror groups.

So we just need to add tc.jar present in the lib directory to the classpath and make Bucket implement the interface AAFairDistributionPolicyMarker, something like shown below:

public class Bucket implements AAFairDistributionPolicyMarker

On running the tests again, the Bucket will be distributed uniformly among the different mirror groups. We can verify this by launching dev-console again.

The source code along with the tc-config.xml can be downloaded from:

http://www.easy-share.com/1907575717/src.tar