This site hosts historical documentation. Visit www.terracotta.org for recent product information.
Tuning BigMemory Go often involves sizing the data storage tiers appropriately. BigMemory Go provides a number of ways to size the different data tiers using simple sizing attributes. These sizing attributes affect memory and disk resources.
The following table summarizes the sizing attributes in BigMemory Go's Ehcache API.
| Tier | Attribute | Pooling available at CacheManager Level? | Description |
|---|---|---|---|
| Memory Store (Heap) | maxEntriesLocalHeap
maxBytesLocalHeap |
maxBytesLocalHeap only |
The maximum number of entries or bytes a data set can use in local heap memory, or when set at the CacheManager level (maxBytesLocalHeap only), as a pool available to all data sets under that CacheManager. This setting is required for every cache or at the CacheManager level. |
| Off-heap Store | maxBytesLocalOffHeap |
Yes | The maximum number of bytes a data set can use in off-heap memory, or when set at the CacheManager level, as a pool available to all data sets under that CacheManager. |
| Disk Store | maxEntriesLocalDisk
maxBytesLocalDisk |
maxBytesLocalDisk only |
The maximum number of entries or bytes a data set can use on the local disk, or when set at the CacheManager level (maxBytesLocalDisk only), as a pool available to all data sets under that CacheManager. Note that these settings apply to temporary disk storage ("localTempSwap"); these settings do not apply to disk persistence. |
Attributes that set a number of entries or elements take an integer. Attributes that set a memory size (bytes) use the Java -Xmx syntax (for example: "500k", "200m", "2g") or percentage (for example: "20%"). Percentages, however, can be used only in the case where a CacheManager-level pool has been configured.
You can constrain the size of any data set on a specific tier in that data set's configuration. You can also constrain the size of all of a CacheManager's data sets in a specific tier by configuring an overall size at the CacheManager level.
If there is no CacheManager-level pool specified for a tier, an individual data set claims the amount of that tier specified in its configuration. If there is a CacheManager-level pool specified for a tier, an individual data set claims that amount from the pool. In this case, data sets with no size configuration for that tier receive an equal share of the remainder of the pool (after data sets with explicit sizing configuration have claimed their portion).
For example, if a CacheManager with eight data sets pools one gigabyte of heap, and two data sets each explicitly specify 200MB of heap while the remaining data sets do not specify a size, the remaining data sets will share 600MB of heap equally. Note that data sets must use bytes-based attributes to claim a portion of a pool; entries-based attributes such as maxEntriesLocal cannot be used with a pool.
On startup, the sizes specified by data sets are checked to ensure that any CacheManager-level pools are not over-allocated. If over-allocation occurs for any pool, an InvalidConfigurationException is thrown. Note that percentages should not add up to more than 100% of a single pool.
If the sizes specified by data sets for any tier take exactly the entire CacheManager-level pool specified for that tier, a warning is logged. In this case, data sets that do not specify a size for that tier cannot use the tier, as nothing is left over.
A size must be provided for the heap, either in the CacheManager (maxBytesLocalHeap only) or in each individual cache (maxBytesLocalHeap or maxEntriesLocalHeap). Not doing so causes an InvalidConfigurationException.
If a pool is configured, it can be combined with a heap setting in an individual cache. This allows the cache to claim a specified portion of the heap setting configured in the pool. However, in this case the cache setting must use maxBytesLocalHeap (same as the CacheManager).
In any case, every cache must have a heap setting, either configured explicitly or taken from the pool configured in the CacheManager.
Off-heap sizing can be configured in bytes only, never by entries.
If a CacheManager has a pool configured for off-heap, your application cannot add caches dynamically that have off-heap configuration — doing so generates an error. In addition, if any caches that used the pool are removed programmatically or through the Terracotta Management Console (TMC), other caches in the pool cannot claim the unused portion. To allot the entire off-heap pool to the remaining caches, remove the unwanted cache from the Ehcache configuration and then reload the configuration.
To use off-heap as a data tier, a cache must have overflowToOffHeap set to "true". If a CacheManager has a pool configured for using off-heap, the overflowToOffHeap attribute is automatically set to "true" for all caches. In this case, you can prevent a specific cache from overflowing to off-heap by explicitly setting its overflowToOffHeap attribute to "false".
Note that an exception is thrown if any cache using an off-heap store attempts to put an element that will cause the off-heap store to exceed its allotted size. The exception will contain a message similar to the following:
The element '[ key = 25274, value=[B@3ebb2a91, version=1, hitCount=0,
CreationTime = 1367966069431, LastAccessTime = 1367966069431 ]'
is too large to be stored in this offheap store.
The local disk can be used as a data tier, either for temporary storage or for disk persistence, but not both at once.
To use the disk as a temporary tier during BigMemory operation, set the persistenceStrategy to "localTempSwap" (refer to Temporary Disk Storage), and use the maxBytesLocalDisk setting to configure the size of this tier.
To use the disk for disk persistence, refer to Disk Persistence Implementation.
The following examples illustrate both pooled and individual cache-sizing configurations.
The following configuration sets pools for all of this CacheManager's caches:
<ehcache xmlns...
Name="CM1"
maxBytesLocalHeap="100M"
maxBytesLocalOffHeap="10G"
maxBytesLocalDisk="50G">
...
<cache name="Cache1" ... </cache>
<cache name="Cache2" ... </cache>
<cache name="Cache3" ... </cache>
</ehcache>
CacheManager CM1 automatically allocates these pools equally among its three caches. Each cache gets one third of the allocated heap, off-heap, and local disk. Note that at the CacheManager level, resources can be allocated in bytes only.
You can explicitly allocate resources to specific caches:
<ehcache xmlns...
Name="CM1"
maxBytesLocalHeap="100M"
maxBytesLocalOffHeap="10G"
maxBytesLocalDisk="60G">
...
<cache name="Cache1" ...
maxBytesLocalHeap="50M"
...
</cache>
<cache name="Cache2" ...
maxBytesLocalOffHeap="5G"
...
</cache>
<cache name="Cache3" ... </cache>
</ehcache>
In the example above, Cache1 reserves 50Mb of the 100Mb local-heap pool; the other caches divide the remaining portion of the pool equally. Cache2 takes half of the local off-heap pool; the other caches divide the remaining portion of the pool equally. Cache3 receives 25Mb of local heap, 2.5Gb of off-heap, and 20Gb of the local disk.
Caches that reserve a portion of a pool are not required to use that portion. Cache1, for example, has a fixed portion of the local heap but may have any amount of data in heap up to the configured value of 50Mb.
Note that caches must use the same sizing attributes used to create the pool. Cache1, for example, cannot use maxEntriesLocalHeap to reserve a portion of the pool.
If a CacheManager does not pool a particular resource, that resource can still be allocated in cache configuration, as shown in the following example.
<ehcache xmlns...
Name="CM2"
maxBytesLocalHeap="100M">
...
<cache name="Cache4" ...
maxBytesLocalHeap="50M"
maxEntriesLocalDisk="100000"
...
</cache>
<cache name="Cache5" ...
maxBytesLocalOffHeap="10G"
...
</cache>
<cache name="Cache6" ... </cache>
</ehcache>
CacheManager CM2 creates one pool (local heap). Its caches all use the local heap and are constrained by the pool setting, as expected. However, cache configuration can allocate other resources as desired. In this example, Cache4 allocates disk space for its data, and Cache5 allocates off-heap space for its data. Cache6 gets 25Mb of local heap only.
The following configuration sets pools for each tier:
<ehcache xmlns...
Name="CM1"
maxBytesLocalHeap="1G"
maxBytesLocalOffHeap="10G"
maxBytesLocalDisk="50G">
...
<!-- Cache1 gets 400Mb of heap, 2.5Gb of off-heap, and 5Gb of disk. -->
<cache name="Cache1" ...
maxBytesLocalHeap="40%">
</cache>
<!-- Cache2 gets 300Mb of heap, 5Gb of off-heap, and 5Gb of disk. -->
<cache name="Cache2" ...
maxBytesLocalOffHeap="50%">
</cache>
<!-- Cache2 gets 300Mb of heap, 2.5Gb of off-heap, and 40Gb of disk. -->
<cache name="Cache3" ...
maxBytesLocalDisk="80%">
</cache>
</ehcache>
| You can use a percentage of the total JVM heap for the CacheManager maxBytesLocalHeap. The CacheManager percentage, then, is a portion of the total JVM heap, and in turn, the Cache percentage is the portion of the CacheManager pool for that tier. |
The CacheManager in this example does not pool any resources.
<ehcache xmlns...
Name="CM3"
... >
...
<cache name="Cache7" ...
maxBytesLocalHeap="50M"
maxEntriesLocalDisk="100000"
...
</cache>
<cache name="Cache8" ...
maxEntriesLocalHeap="1000"
maxBytesLocalOffHeap="10G"
...
</cache>
<cache name="Cache9" ...
maxBytesLocalHeap="50M"
...
</cache>
</ehcache>
Caches can be configured to use resources as necessary. Note that every cache in this example must declare a value for local heap. This is because no pool exists for the local heap; implicit (CacheManager configuration) or explicit (cache configuration) local-heap allocation is required.
Caches that do not specify overflow will overflow if a pool is set for off-heap and disk.
<ehcache maxBytesLocalHeap="1g" maxBytesLocalOffHeap="4g"
maxBytesLocalDisk="100g" >
<cache name="explicitlyAllocatedCache1"
maxBytesLocalHeap="50m"
maxBytesLocalOffHeap="200m"
timeToLiveSeconds="100">
</cache>
<!-- Does not overflow to disk because overflow has been explicitly
disabled. -->
<cache name="explicitlyAllocatedCache2"
maxLocalHeap="10%"
maxBytesLocalOffHeap="200m"
timeToLiveSeconds="100">
<persistence strategy="none"/>
</cache>
<!-- Overflows automatically to off-heap and disk because no specific override and resources
are set at the CacheManager level -->
<cache name="automaticallyAllocatedCache1"
timeToLiveSeconds="100">
</cache>
<!-- Does not use off-heap because overflow has been explicitly
disabled. -->
<cache name="automaticallyAllocatedCache2"
timeToLiveSeconds="100"
overflowToOffHeap="false">
</cache>
</ehcache>
Pinned caches can override the limits set by cache-configuration sizing attributes, potentially causing OutOfMemory errors. This is because pinning prevents flushing of cache entries to lower tiers. For more information on pinning, see Pinning, Eviction, and Expiration.
Internal Ehcache mechanisms track data-element sizes and enforce the limits set by CacheManager sizing pools.
Elements put in a memory-limited cache will have their memory sizes measured. The entire Element instance added to the cache is measured, including key and value, as well as the memory footprint of adding that instance to internal data structures. Key and value are measured as object graphs – each reference is followed and the object reference also measured. This goes on recursively.
Shared references will be measured by each class that references it. This will result in an overstatement. Shared references should therefore be ignored.
For the purposes of measurement, references can be ignored using the @IgnoreSizeOf annotation. The annotation may be declared at the class level, on a field, or on a package. You can also specify a file containing the fully qualified names of classes, fields, and packages to be ignored.
This annotation is not inherited, and must be added to any subclasses that should also be excluded from sizing.
The following example shows how to ignore the Dog class.
@IgnoreSizeOf
public class Dog {
private Gender gender;
private String name;
}
The following example shows how to ignore the sharedInstance field.
public class MyCacheEntry {
@IgnoreSizeOf
private final SharedClass sharedInstance;
...
}
Packages may be also ignored if you add the @IgnoreSizeOf annotation to appropriate package-info.java of the desired package. Here is a sample package-info.java for and in the com.pany.ignore package:
@IgnoreSizeOf package com.pany.ignore; import net.sf.ehcache.pool.sizeof.filter.IgnoreSizeOf;
Alternatively, you may declare ignored classes and fields in a file and specify a net.sf.ehcache.sizeof.filter system property to point to that file.
# That field references a common graph between all cached entries com.pany.domain.cache.MyCacheEntry.sharedInstance # This will ignore all instances of that type com.pany.domain.SharedState # This ignores a package com.pany.example
Note that these measurements and configurations apply only to on-heap storage. Once Elements are moved to off-heap memory or disk, they are serialized as byte arrays. The serialized size is then used as the basis for measurement.
As noted above, sizing caches involves traversing object graphs, a process that can be limited with annotations. This process can also be controlled at both the CacheManager and cache levels.
Control how deep the size-of engine can go when sizing on-heap elements by adding the following element at the CacheManager level:
<sizeOfPolicy maxDepth="100" maxDepthExceededBehavior="abort"/>
This element has the following attributes
maxDepth – Controls how many linked objects can be visited before the size-of engine takes any action. This attribute is required.maxDepthExceededBehavior – Specifies what happens when the max depth is exceeded while sizing an object graph:
Ehcache.hasAbortedSizeOf() returns true.The SizeOf policy can be configured at the CacheManager level (directly under <ehcache>) and at
the cache level (under <cache> or <defaultCache>). The cache policy always overrides the CacheManager if both are set.
Use the <sizeOfPolicy> as a subelement in any <cache> block to control how deep the size-of engine can go when sizing on-heap elements belonging to the target cache. This cache-level setting overrides the CacheManager size-of setting.
If warnings or errors appear that seem related to size-of measurement (usually caused by the size-of engine walking the graph), generate more log information on sizing activities:
net.sf.ehcache.sizeof.verboseDebugLogging system property to true.net.sf.ehcache.pool.sizeof in your chosen implementation of SLF4J.When a CacheManager-level storage pool is exhausted, a cache is selected on which to perform eviction to recover pool space. The eviction from the selected cache is performed using the cache's configured eviction algorithm (LRU, LFU, etc...). The cache from which eviction is performed is selected using the "minimal eviction cost" algorithm described below:
eviction-cost = mean-entry-size * drop-in-hit-rate
Eviction cost is defined as the increase in bytes requested from the underlying SOR (System of Record, e.g., database) per unit time used by evicting the requested number of bytes from the cache.
If we model the hit distribution as a simple power-law then:
P(hit n'th element) ~ 1/n^{alpha}
In the continuous limit, this means the total hit rate is proportional to the integral of this distribution function over the elements in the cache. The change in hit rate due to an eviction is then the integral of this distribution function between the initial size and the final size. Assuming that the eviction size is small compared to the overall cache size, we can model this as:
drop ~ access * 1/x^{alpha} * Delta(x)
where "access" is the overall access rate (hits + misses), and x is a unit-less measure of the "fill level" of the cache. Approximating the fill level as the ratio of hit rate to access rate, and substituting in to the eviction-cost expression, we get:
eviction-cost = mean-entry-size * access * 1/(hits/access)^{alpha}
* (eviction / (byteSize / (hits/access)))
Simplifying:
eviction-cost = (byteSize / countSize) * access * 1/(h/A)^{alpha}
* (eviction * hits)/(access * byteSize)
eviction-cost = (eviction * hits) / (countSize * (hits/access)^{alpha})
Removing the common factor of "eviction", which is the same in all caches, lead us to evicting from the cache with the minimum value of:
eviction-cost = (hits / countSize) / (hits/access)^{alpha}
When a cache has a zero hit-rate (it is in a pure loading phase), we deviate from this algorithm and allow the cache to occupy 1/n'th of the pool space, where "n" is the number of caches using the pool. Once the cache starts to be accessed, we re-adjust to match the actual usage pattern of that cache.