Database Sharding

Database sharding is a way of scaling your database horizontally. For a given table or set of tables, you split up the data that is stored and fetched based on a given hash or something like that. Google, Facebook, and Wikipedia all use database sharding.

Table of Contents [-+]

• 1 Benefits
• 2 Sharded Portal Instances
• 3 Additional Settings

 

Benefits

• One database does not get overloaded
• Smaller queries (since each table has less data now)
• Better overall throughput under load because all your IO is not going through one database server.

Sharded Portal Instances

At present, sharding exists in Liferay from version 5.2.3+ for handling data across multiple portal instances. Here's what you do to get it up.

1. After you have a normally working development environment, make sure your hosts file is setup to allow virtual hosting (for our convenience, I will call the hosts abc1.com, abc2.com, abc3.com, etc.).
2. Use the create-minimal SQL script (from our Downloads section) to create three database schemas: lportal, lportal1, and lportal2. Be careful and use the scripts for your liferay portal version, otherwise, it won't work. By default, the configuration files are setup for three schemas called default, one and two, but you can configure it for more. All this configuration is set up in the file portal-impl/src/META-INF/shard-data-source-spring.xml . As you will see, we will include this file in the property spring.configs in the next step. In case we want to modify it we just should write our own file in the Extension Environment and include it in this property instead of the default one.
3. In your portal-ext.properties, you will need to set the following:
   1. Enable META-INF/shard-data-source-spring.xml under spring.configs.
   2. Configure the jdbc schema settings for jdbc.default.*, jdbc.one.*, and jdbc.two.*.
   3. Enable the shard names: shard.available.names=default,one,two
4. Don't forget to set the appropriate username and password for each schema.
5. Startup the server and create several (2-3) portal instances (e.g., abc1.com, abc2.com, abc3.com).
6. Using your favorite database browser, do a query on each of your schemas for the User_ table, and you will notice that the data is now distributed across different schemas. That's it!

Additional Settings

• Remember that default, one, and two are default values defined in portal-impl/src/META-INF/shard-data-source-spring.xml and portal-ext.properties. If you want to change it, make sure to propagate the changes to the spring XML file as well as portal-ext.properties.
• If you want to manually select the shard (via the GUI when creating portal instances), you need to enable in your portal-ext.properties file: shard.selector=com.liferay.portal.dao.shard.ManualShardSelector. Otherwise, the shard for the data will be chosen using a round robin technique.

======================================================================

Hi,

Multiple tenants are using one portal, but each tenant requires a separate datasource (database). So upon each incoming request we need to perform a context switch each time a Hibernate Session is needed. I have browsed the source fairly extensively and found what I believe is the best place to hook in for this context switch:

com.liferay.portal.dao.orm.hibernate.SessionFactoryImpl#openSession() {
...
if (PropsValues.SPRING_HIBERNATE_SESSION_DELEGATED) {
session = _sessionFactoryImplementor.getCurrentSession();
}
else {
session = _sessionFactoryImplementor.openSession();
}

I found that SPRING_HIBERNATE_SESSION_DELEGATED maps to the following property:

# Set this property to false to manually open and close sessions
spring.hibernate.session.delegated=true


Per what I read in docs and code I can set this property to false, which I have done:

spring.hibernate.session.delegated=false

Docs say to also change the following property to reference my replacement class:

# Set the bean name for the Liferay session factory.
spring.hibernate.session.factory=com.blah.blah.BlahSessionFactoryImplementor

I have done this in portal-ext.properties in:

tomcat-6.0.18/webapps/ROOT/WEB-INF/classes/portal-ext.properties

I have also placed my class here:

tomcat-6.0.18/webapps/ROOT/WEB-INF/classes/com/blah/blah/BlahSessionFactoryImplementor.class

When I start the portal I see that the ext properties get loaded. But my class is apparently not loaded. I have tried to log output when openSession() is called but I see no logging to the console. I also have added an explicit exception throw from the method in case logging is not setup correctly. I see no indication that an exception is thrown. The portal runs normally. I am sure my class is not being used.

1. Is this the correct way to override the session factory? If so, what could be the problem?

2. If not, what should I do?

Thanks!

Vaughn