Understand WSO2 API Manager’s new Configuration Model

Mapping between old configuration files and new deployment.toml file

Understand wso2 api manager 3.x new configuration model

The new configuration model based on the toml format is introduced from API Manager 3.0.0 onwards. In older versions of the product, users had to modify different configuration files depending on the components related to the specific feature they were configuring.

With new configuration model all configuration files have been merged to make configurations easier. Therefore, the <API-M_HOME>/repository/conf/deployment.toml file is the single source used to configure and tune the various features in API Manager.

You can follow below documentation to understand the configuration parameters that can be used in deployment.toml file.

Configuration Catalog - WSO2 API Manager Documentation 3.2.0

But IMHO, this documentation doesn’t cover everything. Even though most of the configurations are added to this new configuration model, this configuration catalog doesn’t contain all the configurations.

Therefore I thought of writing this article, just to share the knowledge on how to find the new configuration parameters for the deployment.toml for any configuration.

Understanding this, will be really helpfull when migrating from old APIM versions to new APIM 3.x versions

.j2 template files

If you go in to

AM_HOME/repository/resources/conf/templates/repository/conf

you may find files that are exactly in the same structure as AM_HOME/repository/conf directory.

Each configuration file that is inside repository/conf folder has a .j2 template file. These .j2 files contain the parameterisations.

Lets see it by an example. Lets assume that we want to change the ServerURL value inside APIKeyValidator section of the api-manager.xml file

<APIKeyValidator>
<!-- Server URL of the API key manager --><ServerURL>https://localhost:${mgt.transport.https.port}${carbon.context}services/</ServerURL>
<!-- Admin username for API key manager. --><Username>${admin.username}</Username>
<!-- Admin password for API key manager. --><Password>${admin.password}</Password><KeyValidationHandlerClassName>org.wso2.carbon.apimgt.keymgt.handlers.DefaultKeyValidationHandler</KeyValidationHandlerClassName><DefaultKeyManagerType>default</DefaultKeyManagerType><EnableKeyManagerConfigurationRetriever>true</EnableKeyManagerConfigurationRetriever><EnableProvisionedAppValidation>true</EnableProvisionedAppValidation>
</APIKeyValidator>

Lets check the respective .j2 file. Which is api-manager.xml.j2 file inside AM_HOME/repository/resources/conf/templates/repository/conf

<APIKeyValidator>
<!-- Server URL of the API key manager -->
<ServerURL>{{apim.key_manager.service_url}}</ServerURL>
<!-- Admin username for API key manager. -->
<Username>{{apim.key_manager.username}}</Username>
<!-- Admin password for API key manager. -->
<Password>{{apim.key_manager.password}}</Password><KeyValidationHandlerClassName>{{apim.key_manager.key_validation_handler_impl}}</KeyValidationHandlerClassName>
<DefaultKeyManagerType>{{apim.key_manager.type}}</DefaultKeyManagerType>
{% if apim.jwt.expiry_time is defined %}
<JWTExpiryTime>{{apim.jwt.expiry_time}}</JWTExpiryTime>
{% endif %}
<EnableKeyManagerConfigurationRetriever>{{apim.key_manager.enable_retriever}}</EnableKeyManagerConfigurationRetriever>
{% if apim.key_manager.enable_provisioned_app_validation is defined %}
<EnableProvisionedAppValidation>{{apim.key_manager.enable_provisioned_app_validation}}</EnableProvisionedAppValidation>
{% endif %}
</APIKeyValidator>

You can see that the parameter for setting up the value to ServerURL in APIKeyValidator is apim.key_manager.service_url. Therefore the configuration for the deployment.toml will be as below.

[apim.key_manager]
service_url = "https://my_keymanager.com:${mgt.transport.https.port}/services/"

default.json file

All the parameters inside .j2 files have default values.

We need to add the values in deployment.toml file only if we need to modify the default value. If the default value is fine with us, then we don’t need to add a configuration block inside deployment.toml file.

This default.json file can be found in below path.

AM_HOME/repository/resources/conf/default.json

This file has all the parameter names and the default values for those.

Let’s look at another example. Let’s say that we want to Enable the Gateway Token Cache. Relevent configuration can be found in api-manager.xml and inside <CacheConfigurations>

<CacheConfigurations>
<!-- Enable/Disable token caching at the Gateway-->
<EnableGatewayTokenCache>{{apim.cache.gateway_token.enable}}</EnableGatewayTokenCache>
.......
</CacheConfigurations>

Lets check the default.json file and look for the default value set for apim.cache.gateway_token.enable

default.json file

Its set to true. Which means, by default the gateway token cache is enabled.

Therefore we don’t need to add any configuration in to deployment.toml file.

Repetitive Configurations

There are some important points that I felt its worth mentioning here.

Arrays

If you look at the .j2 template files, you may notice in some places there are for loops. Those for loops are there to add multiple/repetitive configurations. For example if you have more than one Gateway in your environment, then how we can config that inside deployment.toml ? In old configuration file its just a matter of adding multiple <APIGateway> sections. But in deployment.toml you will have to add it as arrays/groups.

First let’s look at APIGateway section in the api-manager.xml.j2 file.

<APIGateway>
........
{% for environment_name in apim.gateway.environment%}
<Environment type="{{environment_name.type}}" api-console="{{environment_name.display_in_api_console}}" isDefault="{{environment_name.show_as_token_endpoint_url}}">
<Name>{{environment_name.name}}</Name>
.........
</APIGateway>

In above section you can see that there is a for loop to iterate through each section called apim.gateway.environment. Therefore when you configure in the deployment.toml file you need to add as grouped sections like below.

[[apim.gateway.environment]]
name = "Internal Gateway"
type = "hybrid"
display_in_api_console = true
......
[[apim.gateway.environment]]
name = "External Gateway"
type = "hybrid"
display_in_api_console = true
......

KeyValue Pairs

If you look at the .j2 template files, you may notice in some places there is another set of for-loops for adding key value pairs. Most of the time this is there for adding multiple properties.

For example let’s look at how we can set the properties in http transport sender configuration in axis2_client.xml. This is how the section for http transport sender in axis2_client.xml.j2 file configuration looks like

<transportSender name="https"
class="org.apache.axis2.transport.http.CommonsHTTPTransportSender">{% for name,value in transport.client.https.properties.items()%}<parameter name="{{name}}">{{value}}</parameter>
{% endfor %}
</transportSender>

In above section you can see that there is a for loop to iterate through transport.client.https.properties.

First let’s look at default.json file to see what are the default properties getting added to this transport sender.

"transport.client.http.properties.PROTOCOL": "HTTP/1.1",
"transport.client.http.properties.Transfer-Encoding": "chunked",
"transport.client.http.properties.SO_TIMEOUT": "60000",
"transport.client.http.properties.CONNECTION_TIMEOUT": "60000",

Let’s say we want to increase the timeout values. Which means we have to override from deployment.toml. We have to add those as key value pairs.

[transport.client.https.properties]
SO_TIMEOUT = 120000
CONNECTION_TIMEOUT = 120000

What Happens

During the server startup, the WSO2 API Manager will assign the values inside default.json file to its respective parameters inside multiple configuration files (api-manager.xml, carbon.xml, user-mgt.xml, registry.xml ,pass-through.properties, axis2.xml etc)

If we have set any values for any of those parameters(inside deployment.toml), then the server will assign the value from deployment.toml instead of the default value in default.json file.

Summary

In summary, from APIM 3.x onwards,

There is only one configuration file to modify. That is deployment.toml file.

Its not recommended to modify the .xml files directly.

You don’t have to add a configuration to deployment.toml file unless you want to modify the server’s default value.

Default values for all the configurations can be found in default.json file.

You always can find the required parameter name to add in deployment.toml by looking at the respective .j2 template file of that configuration file.