Terracotta EhCache

What is L1 & L2?

L1 is a client (or your application) that connects to terracotta caching server
L2 is caching server.

EhCache Start and Stop scripts

Here are the custom scripts for start and stop operations.

gethostname.sh
HOST_NAME=`cat /proc/sys/kernel/hostname | cut -d'.' -f1`
echo $HOST_NAME

profile_terracotta
TERRACOTTA_HOME=/opt/terracotta-3.4/terracotta-ee-3.4.0
JAVA_HOME=/opt/jdk1.6.0_22
export TERRACOTTA_HOME JAVA_HOME
export PATH=$PATH:$JAVA_HOME/bin:$SCRIPTS:$TERRACOTTA_HOME/bin
host=`gethostname.sh`
node=""
usr=admin
pwd=""
if [ "$host" = myhost1 ]
then
        node=primary
        pwd=test1
elif [ "$host" = myhost2 ]
then
        node=secondary
        pwd=test2
fi

start_terracotta.sh
source profile_terracotta
nohup $TERRACOTTA_HOME/bin/start-tc-server.sh -f /opt/terracotta-3.4/config/tc-config.xml > /opt/terracotta-3.4/startuplogs/startup.log &
sleep 10
cat /opt/terracotta-3.4/startuplogs/startup.log
ps -ef | grep java

stop_terracotta.sh
source profile_terracotta
$TERRACOTTA_HOME/bin/stop-tc-server.sh -f /opt/terracotta-3.4/config/tc-config.xml -n $node -u $usr -w $pwd
rm -Rf /opt/terracotta-3.4/server-logs/.terra*
rm -Rf /opt/terracotta-3.4/server-logs
rm -Rf /opt/terracotta-3.4/server-data
rm -Rf /opt/terracotta-3.4/server-statistics
rm -Rf /opt/terracotta-3.4/startuplogs/*
sleep 10
ps -ef | grep java

1) Repeat the 3.4 installation steps on each participating server in the cluster.
2) Create a file /opt/terracotta-3.4/config/tc-config.xml.

Here is the production recommended version of tc-config.xml.
Following is a sample for two node cluster.

<?xml version="1.0" encoding="UTF-8" ?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-6.xsd">
<tc-properties>
<property name="l2.nha.dirtydb.autoDelete" value="true"/>
<property name="l1.cachemanager.enabled" value="true"/>
<property name="logging.maxLogFileSize" value="1024"/>
<property name="l2.l1reconnect.enabled" value="true" />
    <property name="l2.l1reconnect.timeout.millis" value="5000"/>
<property name="l2.healthcheck.l2.ping.idletime" value="5000" />
<property name="l2.healthCheck.l2.ping.interval" value="1000" />
<property name="l2.healthCheck.l2.ping.probes" value="3" />
<property name="l2.healthCheck.l2.socketConnect" value="true" />
<property name="l2.healthCheck.l2.socketConnectCount" value="5" />
<property name="l1.healthCheck.l2.ping.idletime" value="5000" />
<property name="l1.healthCheck.l2.ping.interval" value="1000" />
<property name="l1.healthCheck.l2.ping.probes" value="3" />
<property name="l1.healthCheck.l2.socketConnect" value="true" />
<property name="l1.healthCheck.l2.socketConnectCount" value="8" />
<property name="l2.healthCheck.l1.ping.idletime" value="5000" />
<property name="l2.healthCheck.l1.ping.interval" value="1000" />
<property name="l2.healthCheck.l1.ping.probes" value="3" />
<property name="l2.healthCheck.l1.socketConnect" value="true" />
<property name="l2.healthCheck.l1.socketConnectCount" value="5" />
</tc-properties>
<servers>
<server host="myhost1" name="primary" bind="172.x.x.1">
  <data>/opt/terracotta-3.4/server-data</data>
  <logs>/opt/terracotta-3.4/server-logs</logs>
  <statistics>/opt/terracotta-3.4/server-statistics</statistics>
  <dso-port bind="172.x.x.1">9540</dso-port>
  <jmx-port bind="172.x.x.1">9550</jmx-port>
  <l2-group-port bind="172.x.x.1">9560</l2-group-port>
  <authentication>
   <mode>
     <password-file>/home/cacheadm/terracotta-access/jmxremote.password</password-file>
   </mode>
   <access-file>/home/cacheadm/terracotta-access/jmxremote.access</access-file>
  </authentication>
  <dso>
   <client-reconnect-window>120</client-reconnect-window>
   <persistence>
    <mode>permanent-store</mode>
    <offheap>
     <enabled>false</enabled>
     <maxDataSize>5g</maxDataSize>
    </offheap>
   </persistence>
   <garbage-collection>
    <enabled>true</enabled>
    <verbose>true</verbose>
    <interval>3600</interval>
   </garbage-collection>
  </dso>
</server>
<server host="myhost2" name="secondary" bind="172.x.x.2">
  <data>/opt/terracotta-3.4/server-data</data>
  <logs>/opt/terracotta-3.4/server-logs</logs>
  <statistics>/opt/terracotta-3.4/server-statistics</statistics>
  <dso-port bind="172.x.x.2">9540</dso-port>
  <jmx-port bind="172.x.x.2">9550</jmx-port>
  <l2-group-port bind="172.x.x.2">9560</l2-group-port>
  <authentication>
   <mode>
     <password-file>/home/cacheadm/terracotta-access/jmxremote.password</password-file>
   </mode>
   <access-file>/home/cacheadm/terracotta-access/jmxremote.access</access-file>
  </authentication>
  <dso>
   <client-reconnect-window>120</client-reconnect-window>
   <persistence>
    <mode>permanent-store</mode>
    <offheap>
     <enabled>false</enabled>
     <maxDataSize>5g</maxDataSize>
    </offheap>
   </persistence>
   <garbage-collection>
    <enabled>true</enabled>
    <verbose>true</verbose>
    <interval>3600</interval>
   </garbage-collection>
  </dso>
</server>
<mirror-groups>
  <mirror-group group-name="group1">
   <members>
    <member>primary</member>
    <member>secondary</member>
   </members>
  </mirror-group>
</mirror-groups>

<ha>
  <mode>networked-active-passive</mode>
   <networked-active-passive>
   <election-time>5</election-time>
  </networked-active-passive>
</ha>
<update-check>
  <enabled>false</enabled>
  <period-days>10</period-days>
</update-check>
</servers>
<clients>
<logs>/opt/terracottalogs</logs>
</clients>
</tc:tc-config>

Note:
Authentication is enabled. You can disable it if you don't need them

Default ports are modified. It is a good security practice to modify default vendor supplied ports for production environments.

The above config is just a sample. You should tune the environment based on your specific needs

3) Copy tc_config.xml to each participatinng server in a cluster.
4) Start server with option -f tc_config.xml (Refer to scripts posting)

Terracotta EhCache 3.4 installation

The installation steps are for RHEL 5 64 bit

1) Create user
#useradd -m cacheadm
#passwd cacheadm
Login an cacheadm

2) Install Java environment
Download JDK 1.6 update latest from http://java.sun.com/javase/downloads/index.jsp
I downloaded 64-bit JDK.
Copy to /opt or install folder
#./jdk-6u22-linux-x64.bin
This should install Java under directory jdk1.6.0_22
Note: You can also install JRE alone the same way. JDK comes with utilities such as jar, jconsole, jstack are not available in JRE

3) Set Java runtime
Edit your profile file to set system runtime variables.
# vi /home/cacheadm/.bash_profile
Update your profile file to read as the following.
# cat /home/jbossadm/.bash_profile
PATH=$PATH:$HOME/bin
JAVA_HOME=/opt/jdk1.6.0_22
export JAVA_HOME
export PATH=$JAVA_HOME/bin:$PATH

Exit the shell and re-login for the changes to take effect

4) Verify Java path and version
#which java
#java -version

5) Install Terracotta Distribute Cache
Download terracotta-3.3.0-ee-installer.jar from http://www.terracotta.org/
Copy to /opt or install folder
# export DISPLAY={ip}:0.0
# java -jar terracotta-ee-3.4.0-installer.jar
Set the installation path as /opt/terracotta-3.4/terracotta-ee-3.4.0
Complete the installation
Download the license key from Terracotta website and copy it to /opt/terracotta-3.4/terracotta-ee-3.4.0 directory. The license file is terracotta-license.key

6) Update ~/.bash_profile file
JAVA_HOME=/opt/jdk1.6.0_22
TERRACOTTA_HOME=/opt/terracotta-3.4/terracotta-ee-3.4.0
export JAVA_HOME
export PATH=$JAVA_HOME/bin:$TERRACOTTA_HOME/bin:$PATH

7) Start Terracotta
Execute ./start-tc-server.sh
On successful start, the following message will be written.
INFO - Terracotta Server instance has started up as ACTIVE node on 0:0:0:0:0:0:0:0:9510 successfully, and is now ready for work.

8) Launch dev console
# export DISPLAY={ip}:0.0
$TERRACOTTA_HOME/bin/dev-console.sh
Right click on Terracotta cluster and connect
If you see console with login window, please specify the following
Server host: localhost or server name
JMX port: 9520

Terracotta Remote Connection

If you have problem connecting to remote Terracotta servers running on Linux, check /etc/hosts file on the server. Make sure it has the ipadress and FQDN/hostname specified.

Sample hosts file
171.67.89.212 hostname.domain.com hostname
127.0.0.1 hostname.domain.com hostname localhost.

Terrocotta does a host lookup when JMX/RMI remote connections are requested. It is important your hosts file return correct address. If the loopback address is listed on the first line, the remote connection will not work properly.

Terracotta HA and health checker for production environments

Here is a sample file with Terracotta HA and health checker recommended for production environments.

<?xml version="1.0" encoding="UTF-8"?>

<tc:tc-config xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-5.xsd"
xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<tc-properties>
  <property name="l2.l1reconnect.enabled" value="true" />
  <property name="l2.healthcheck.l2.ping.idletime" value="5000" />
  <property name="l2.healthCheck.l2.ping.interval" value="1000" />
  <property name="l2.healthCheck.l2.ping.probes" value="3" />
  <property name="l2.healthCheck.l2.socketConnect" value="true" />
  <property name="l2.healthCheck.l2.socketConnectCount" value="5" />
  <property name="l1.healthCheck.l2.ping.idletime" value="5000" />
  <property name="l1.healthCheck.l2.ping.interval" value="1000" />
  <property name="l1.healthCheck.l2.ping.probes" value="3" />
  <property name="l1.healthCheck.l2.socketConnect" value="true" />
  <property name="l1.healthCheck.l2.socketConnectCount" value="8" />
  <property name="l2.healthCheck.l1.ping.idletime" value="5000" />
  <property name="l2.healthCheck.l1.ping.interval" value="1000" />
  <property name="l2.healthCheck.l1.ping.probes" value="3" />
  <property name="l2.healthCheck.l1.socketConnect" value="true" />
  <property name="l2.healthCheck.l1.socketConnectCount" value="5" />
</tc-properties>

<servers>
  <server host="tc-server1" name="tc-server1">
   <data>/opt/terracotta/server-data</data>
   <logs>/opt/terracotta/server-logs</logs>
  </server>
  <server host="tc-server2" name="tc-server2">
   <data>/opt/terracotta/server-data</data>
   <logs>/opt/terracotta/server-logs</logs>
  </server>

  <mirror-groups>
   <mirror-group group-name="group1">
    <members>
     <member>tc-server1</member>
     <member>tc-server2</member>
    </members>
   </mirror-group>
  </mirror-groups>
  <ha>
   <mode>networked-active-passive</mode>
   <networked-active-passive>
   <election-time>10</election-time>
   </networked-active-passive>
  </ha>
</servers>

<clients>
  <logs>/opt/terracotta/client-logs</logs>
</clients>
</tc:tc-config>

Terracotta File Based Authentication

1) Include the authentication section in tc-config.xml.
  <server host="{hostname}" name="{servername}">
   <data>/opt/terracotta/server-data</data>
   <logs>/opt/terracotta/server-logs</logs>
   <authentication>
    <mode>
     <password-file>/opt/jmxremote.password</password-file>
    </mode>
    <access-file>/opt/jmxremote.access</access-file>
   </authentication>
  </server>

2) Create two files with the following content
jmxremote.access
admin readwrite
monitor readonly
jmxremote.password
admin password1
monitor password2

Note: You can keep these files in any location and adjust the tc-config accordingly.

3) Restart the server. Access devconsole.
You will be prompted for user name and password.
Enter admin as user and password as password1

Terracotta Monitoring

1) Export your display
export DISPLAY={ip_address}:0.0

2) Launch dev console
/opt/terracotta/terracotta-3.3.0-ee/bin/dev-console.sh
Connect with default parameters

3) Launch operations console
/opt/terracotta/terracotta-3.3.0-ee/bin/ops-center.sh
Connect with default parameters

Terracotta EhCache 3.3 Installation

The installation steps are for RHEL 5

1) Create user
#useradd -m cacheadm
#passwd cacheadm
Login an cacheadm

2) Install Java environment
Download JDK 1.6 update latest from http://java.sun.com/javase/downloads/index.jsp
Copy to /opt or install folder
#./jdk-6u21-linux-i586.bin
This should install Java under directory jdk1.6.0_21
Note: You can also install JRE alone the same way. JDK comes with utilities such as jar, jconsole, jstack are not available in JRE

3) Set Java runtime
Edit your profile file to set system runtime variables.
# vi /home/cacheadm/.bash_profile
Update your profile file to read as the following.
# cat /home/jbossadm/.bash_profile
PATH=$PATH:$HOME/bin
JAVA_HOME=/opt/jdk1.6.0_21
export PATH=$JAVA_HOME/bin:$PATH

Exit the shell and re-login for the changes to take effect

4) Verify Java path and version
#which java
#java -version

5) Install Terracotta Distribute Cache
Download terracotta-3.3.0-ee-installer.jar from http://www.terracotta.org/
Copy to /opt or install folder
# export DISPLAY={ip}:0.0
# java -jar terracotta-3.3.0-ee-installer.jar
Set the installation path as /opt/terracotta/terracotta-3.3.0-ee
Complete the installation
Download the license key from Terracotta website and copy it to /opt/terracotta/terracotta-3.3.0-ee directory.

6) Update ~/.bash_profile file
JAVA_HOME=/opt/jdk1.6.0_21
TERRACOTTA_HOME=/opt/terracotta/terracotta-3.3.0-ee
export PATH=$JAVA_HOME/bin:$TERRACOTTA_HOME/bin:$PATH
7) Start Terracotta
Java home is not properly recoginzed the start script. I edited $TERRACOTTA_HOME/bin/start-tc-server.sh and added JAVA_HOME at the beginning.
JAVA_HOME=/opt/jdk1.6.0_21

Execute ./start-tc-server.sh

If the license file (product.key) does not exist, the following error will be reported.
2010-09-23 14:13:34,732 INFO - Terracotta Enterprise 3.3.0, as of 20100716-160738 (Revision 7871-15922 by cruise@su10mo5 from 3.3)
2010-09-23 14:13:35,144 INFO - Successfully loaded base configuration from Java resource at '/com/tc/config/schema/setup/default-config.xml', relative to class com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.
2010-09-23 14:13:35,351 INFO - Log file: '/home/cacheadm/terracotta/server-logs/terracotta-server.log'.
2010-09-23 14:13:35,355 FATAL - Product key does not exist at /opt/terracotta/terracotta-3.3.0-ee/bin/./../product.key

On successful start, the following message will be written.
INFO - Terracotta Server instance has started up as ACTIVE node on 0:0:0:0:0:0:0:0:9510 successfully, and is now ready for work.

8) Launch console
Java home is not properly recoginzed the start script. I edited $TERRACOTTA_HOME/bin/dev-console.sh and added JAVA_HOME at the beginning.
# export DISPLAY={ip}:0.0
Right click on Terracotta cluster and connect/
If you see console with login window, please specify the following
Server host: localhost or server name
JMX port: 9520

9) Setup Cluster Config
Replicate the above steps on each participating server .
Create a file tc-config.xml and copy to /opt/terracotta/config on each server.

<?xml version="1.0" encoding="UTF-8"?>

<tc:tc-config xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-5.xsd"
xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

<servers>
    
    <server host="{ip}" name="{hostname}">
      <data>/opt/terracotta/server-data</data>
      <logs>/opt/terracotta/server-logs</logs>
    </server>
    
    <server host="{ip}" name="{hostname}">
      <data>/opt/terracotta/server-data</data>
      <logs>/opt/terracotta/server-logs</logs>
</server>

    <ha>
      <mode>networked-active-passive</mode>
      <networked-active-passive>
        <election-time>5</election-time>
      </networked-active-passive>
    </ha>
</servers>

<clients>
    <logs>/opt/terracotta/client-logs</logs>
</clients>
</tc:tc-config>

10) Setup ehcache config
Update /opt/terracotta/terracotta-3.3.0-ee/ehcache/ehcache.xml with the following.
     <terracottaConfig url="{server1-ip}:9510,{server2-ip}:9510"/>
11) Start the server
$TERRACOTTA_HOME/bin/start-tc-server.sh -f /opt/terracotta/config/tc-config.xml -n {hostname or alias specified in tc-confg.xml}

Terracotta EhCache

Saturday, November 13, 2010

What is L1 & L2?

EhCache Start and Stop scripts

EhCache Servers Clustering

Terracotta EhCache 3.4 installation

Thursday, November 11, 2010

Terracotta Remote Connection

Monday, November 8, 2010

Terracotta HA and health checker for production environments

Terracotta File Based Authentication

Monday, October 4, 2010

Terracotta Monitoring

Thursday, September 23, 2010

Terracotta EhCache 3.3 Installation

About Me

Posts

Archives