On-Premise Agent Clustering

Multiple Network Setups

In distributed environments, Agents may live in isolated networks and have different application access. Without proper configuration, operations could fail when an Agent cannot reach its target system. You may need to configure multiple Agents connecting to your Lumos tenant that exist in separate networks.

A simple example of this setup may look like this:

By default, Agents indiscriminately pick up operations from the Lumos Platform. They do not validate network reachability ahead of time. In this case, either Agent 1 or Agent 4 could pick up an operation meant for Postgres DB A, but Agent 4 will not be able to reach that database over the network.

Clustering Agents

In Lumos, a Cluster is a group of Agents that have access to the same set of applications within a shared network environment.

To control which Agents handle specific integration connections, you assign each Agent a Cluster ID and configure your integrations to target the appropriate cluster.

A Cluster ID is simply a name you choose to identify a group of Agents. We recommend using clear and descriptive names, as users setting up integrations will need to select the correct Cluster ID when connecting applications.

Agent Configuration

To indicate an Agent is part of a cluster, it must be assigned a Cluster ID. This can be done via an environment variable or the settings.toml file.

settings.toml file

Within the settings.toml file that exists in the decompressed Lumos On-Premise Agent files, set the key cluster_id to your desired value:

[agent-settings]
cluster_id="test_cluster"

Restart the Lumos On Premise Agent to pick up the configuration change

Environment Variable

For both Windows and Linux Agents, the environment variable is LUMOS_ON_PREMISE_AGENT_CLUSTER_ID

Windows

  1. In the System Properties window, go to the Advanced tab.
  2. Click the Environment Variables button near the bottom.
  3. In the Environment Variables window, you can choose to set the variable:
    1. System-wide (available to all users and services): Under System variables, click New.
    2. User-specific (available only to the logged-in user, use this option if you are running the Lumos On Prem agent as a specific user): Under User variables, click New.
  4. In the New Variable dialog:
    1. Enter the Variable name (e.g., LUMOS_ON_PREMISE_AGENT_CLUSTER_ID).
    2. Enter the Variable value (e.g., network_a_cluster).
  5. Click OK to save.
  6. Restart your Lumos On Premise Agent Service via the Windows Service Manager

Linux/Docker

This may vary depending on how you are deploying your Lumos On-Prem Agents.

If you are running the Docker Linux Agent manually or through a script, set the environment variable during the docker run command like so:

docker run -e LUMOS_ON_PREMISE_AGENT_CLUSTER_ID=network_a_cluster ...

If you are deploying your Lumos On-Prem Agents through AWS ECS or another container hosting service, follow that service’s instructions for setting environment variables.

If configured correctly, the assigned Cluster ID should appear alongside the Agent ID in the Agents tab of the Integrations page within Lumos.

Lumos Integration Configuration

Now you should have an on premise agent setup that looks like this:

Note: It is not necessary to assign a Cluster ID to all of your Agents. An Agent without a Cluster ID is considered part of the default cluster.

Connecting a new integration

When connecting a new integration that exists within the same network as an Agent cluster, select the cluster in the integration connection modal when entering your connection settings.

Updating an existing integration

To update an existing integration, navigate to the Integration tab of the App you wish to change:

At the bottom of the connection settings, there will be an option to set the cluster:

Set the cluster here, re-enter any secret settings (password, API token, etc) and then click "Reconnect"

Troubleshooting

  • If you have set a cluster id on an agent but do not see the option to select a cluster when connecting a new integration
    • Check the Agents tab of the Integrations page within Lumos. If the cluster id is not appearing alongside the Agent id on that page, Lumos was not able to detect the cluster id. Check that your configuration is set correctly. Note: It may take several minutes for Agent changes to be reflected in Lumos