One of the important thing that must be supported by the production XACML PDP is that horizontal scaling with high availability and reliability. WSO2 Identity server’s PDP achieves the horizontal scaling using clustering mechanism. In this post i am going to explain deploying of PDP cluster using WSO2 Identity Server.
Here following three main points are very important
1. Point all cluster nodes to same user store
User store can be LDAP, AD or JDBC base. All nodes must be pointed to same user store. By default, WSO2 Identity Server uses ApacheDS LDAP directory as its user store and all node need to point to one ApacheDS instance that has been started with one Identity server.
2. Point all cluster nodes to same database
There are three logical databases can be found in Identity server. All these three databases must be shared by each node. You do not need three databases to configure them. You can use one physical database and it can be configured using one database configuration.
a). User management database –
Identity Server persists user management data ( such as internal roles. role permission, claim management data and etc) in user management database.
b). Identity database —
Identity Server persists meta data that is related to OAuth, SAML, STS and etc in Identity database.
c). Registry database –
XACML Policies, configurations and meta data related to server are stored in registry database.
But as we are more consider about XACML Policies, It is fine to point all these logical databases in to one physical database.
3. Enable distributed caching
Identity Server keeps the user permissions, user roles mapping, policies , decisions and attribute values in-memory in order to increase performance. Therefore these in-memory data need to shared with other cluster nodes. Identity Server uses Hazelcast based distributed caching technology.
How to Setup
Lets assume that we are going configure two Identity Server nodes in a cluster using a MSSQL database and embedded ApacheDS user store. Both nodes are in same machine and also all three logical databases are point to one database.
Step 1. Download WSO2Identity Server distribution and Extract it to two locations
Lets assume IS_NODE1 and IS_NODE2 as where we extracted the distribution
Step 2. Create database
You need to create a new database in MSSQL server . Let says “IdentityDB”
Step 3. Configure database
You can configure all three database configuration using one data source configuration. You need to edit the default database configuration defined in the master-datasources.xml file located at IS_NODE1/repository/conf/datasources directory. Database configurations in registry.xml , user-mgt.xml and identity.xml refer this data source by default. Please follow same step for IS_NODE2 as well
Please find the sample configuration for MSSQL database
<datasource> <name>WSO2_CARBON_DB</name> <description>The datasource used for registry and user manager</description> <jndiConfig> <name>jdbc/WSO2CarbonDB</name> </jndiConfig> <definition type="RDBMS"> <configuration> <url>jdbc:jtds:sqlserver://10.100.3.251:1433/identityDB</url> <username>isadmin</username> <password>isadmin</password> <driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName> <maxActive>80</maxActive> <maxWait>60000</maxWait> <minIdle>5</minIdle> <testOnBorrow>true</testOnBorrow> <validationQuery>SELECT 1</validationQuery> <validationInterval>30000</validationInterval> </configuration> </definition> </datasource>
Step 4. Configuring port offset
This is an optional step, Only if you need to run more than one Identity Server in same machine. When you configure the port offset, Server would be start with a port value i.e (default port + offset value). Therefore you change the port offset value in IS_NODE2 and it would start in a different port.
You can find port offset configuration in carbon.xml which can be found in IS_NODE2/repository/conf
<Ports> <Offset>1</Offset> </Ports>
Step 5. Configuring user store to share an one LDAP directory
By default, Identity Server is started with embedded LDAP . Except the one node(IS_NODE1), embedded can be disabled by modifying embedded-ldap.xml which can be found in IS_NODE2/repository/conf directory
<Property name="enable">false</Property>
Then point to the ApacheDS LDAP directory instance that would be started by IS_NODE1
You need to configure its url in user-mgt.xml of IS_NODE2 as following (default port is 10389)
<UserStoreManager class="org.wso2.carbon.user.core.ldap.ApacheDSUserStoreManager"> <Property name="ConnectionURL"> ldap://localhost:10389</Property> </UserStoreManager >
Step 6. Enable Axis2 Clustering
To enable the invalidation cache you need to enable Axis2 clustering. You can simply enable axis2 clustering by configuring axis2.xml file which can be found in IS_NODE1/repository/conf/axis2 directory
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true">
Also make sure to use same domain name across your cluster
<parameter name="domain">wso2.is.domain.1</parameter>
Please follow same step for IS_NODE2 as well
Step 7. Copy JDBC driver
You need to copy the of JDBC driver of MSSQL (Ex-jtds-1.2.2.jar) in to IS_NODE1/repository/component/lib directory . Please follow same step for IS_NODE2 as well
Step 8. Start Server by creating database;
You can start WSO2 Identity Server with “-Dsetup” option. “-Dsetup” option is used to create database tables while server is started. In IS_NODE1, you can start server by running wso2server script which can be found in IS_NODE1/bin.
./wso2server.sh -Dsetup
./wso2server.bat -Dsetup
Then other nodes (IS_NODE2) can be started without -Dsetup option. you only have to run first time with -Dsetup option. Because the next time the created database is already there.
Please note -Dsetup is an optional one. Database tables can be created either manually by running scripts. There are three script files that you need to execute. You can find the from <IS_NODE1>/dbscripts /mssql.sql and <IS_NODE1>/dbscripts/identity/mssql.sql and <IS_NODE1>/dbscripts/service-provider/mssql.sql