Lumos provides a way to manage hundreds of apps for a company. To build a platform across hundreds of different applications with different use cases, data models, and API protocols, we have a standard connector interface into which we translate all the app APIs. ![](https://files.readme.io/b6149571855df0a2038f3ca18b5c7232e6bfc868fc9f2f1a9ebeee8a1c326315-image.png)
We have a connector SDK to make it easier to build these connectors. [https://pypi.org/project/connector-py/](https://pypi.org/project/connector-py/)
## Terminology * `connector`: A package or program Lumos can run, to read data from and write to a third party app. * `custom connector`: A connector that is built and operated by one of Lumos’ customers; usually to talk to their internally built apps, or apps that aren’t accessible via the public internet. * `capability`: a standardized Lumos API on a connector. Example: `list_accounts` returns a list of app user accounts, in standardized Lumos Account format, and enough data to request the next page if applicable. See [Connector capabilities](ref:connector-capabilities) for details. * `On-premise` (aka `on prem`): a way for a customer to run a Lumos connector on their own hardware, in their own network. * `SDK`: Our set of libraries for making it easier to build connectors. Currently only in Python. * `account`: The Lumos term for a user or service account in an underlying app * `entitlement`: Membership in a group, assignment to a role, or some other access-granting permission in the underlying app. Usually something assignable directly to an account. * `resource`: Something in an underlying app that an account can have an entitlement to. In the case of a global entitlement, like a tenant-wide role of Admin, there may be no resource.
# How Lumos connectors are used in production ![](https://files.readme.io/899c4ea8c0ab3d2a832d1b99be26b15c9985a93a8b37768503a67d0e07184130-image.png)
## Connectors describe themselves When Lumos discovers a new connector, our first job is to learn what the connector supports, and how to call it. We do this via the `info` capability. Info tells us… 1. A name, logo, and description of the app in Lumos 2. How to authenticate against the underlying app 3. What settings are available or required for the connector 4. What capabilities the connector supports 5. What resource and entitlement types the connector provides access to 6. An app ID
*Part of the`info` response* ```json { "response": { "user_friendly_name": "Pretend App", "description": "A pretend app for these examples", "logo_url": "https://some-logo-hosting-site.com/pretend_app", "authentication_schema": { ... } "request_settings_schema": { ... }, "capabilities": [ ... ], "resource_types": [ ... ], "entitlement_types": [ ... ], "app_id": "pretend_app", } } ``` For more details on how connectors self-describe, see [Connector capabilities](ref:connector-capabilities#learning-about-connectors) ## The end-user configures a connector Connectors have two avenues for configuration: auth and settings. These are separate keys on the request envelope, with slightly different semantics. The flow here is 1. **Connector Discovery:** The connector describes what kind of auth and non-auth settings are required or available in the `info` response. 2. **Customer configuration:** Lumos uses the settings schema and auth type to render a webform for the customer IT admin, to allow them to establish the connection with a specific integration tenant. 3. **Connector execution:** Each call to a connector capability includes the auth + settings collected in the Customer Configuration phase.

### Auth Lumos assumes that all connectors require some credential materials to successfully talk to the underlying app. Token and Basic credentials require the user to pass one or two credentials, respectively, in the auth section of the request envelope. OAuth requires an access token for most capabilities, and requires a variety of different request credentials for OAuth-specific capabilities. Example: to refresh the access token, you must call the connector with the OAuth refresh token. The auth model for a connector is declared in the `"authentication_schema"` field in the Info response. * To learn more about using OAuth in the connector, see the section on [OAuth capabilities](connector-capabilities#oauth-capabilities) . * To see how the auth settings are passed to the caller, see the [Info response](connector-capabilities#info) .
### Settings There are non-sensitive (usually) settings fields that a connector may require to connect to the underlying app, or may optionally accept to adjust behavior. Some examples might be * The specific DNS name to use to connect to the underlying service. * Which role to assume when connecting to the service. * Whether to include groups when reading entitlements. The settings for a connector are defined as a JSON schema in the `"request_settings_schema"` field in the Info response.
## Connectors are “connected” when their credentials are validated A specific capability, `validate_credentials`, is the last step after a customer has configured and authenticated the connector, to ensure Lumos can call other capabilities successfully.
# Lumos connector capabilities
## Capabilities can be called via Python API or CLI When running connectors inside Lumos, we import and run them directly in a Python VM. ```python from pretend_app import integration await integration.dispatch(CapabilityName.LIST_ACCOUNTS, '{"request":{}}') ``` When running them as in an on-prem configuration, we call them via CLI. ```bash pretend_app list_accounts --json '{"request":{}}' ```
## Capabilities are invoked with request and response envelopes The top-level JSON objects passed to and from capabilities are considered envelopes. Inside the envelope is the capability-specific data, at `"request"` and `"response"` fields respectively. Other notable fields in these envelopes are * `"auth"` (see next section) * `"settings"` (see next section) * `"page"` (see section “Pagination”)
## Types of capabilities Capabilities can be roughly clustered into four sections. * [Metadata: How we learn about the connector.](connector-capabilities#learning-about-connectors) * [OAuth: Used to establish and maintain a good connection to an OAuth app.](connector-capabilities#oauth-capabilities) * [Read: Used to read users, resources, entitlements, and activity.](connector-capabilities#read-capabilities) * [Write: Used to update users and entitlements.](connector-capabilities#write-capabilities)
## Pagination Read capabilities can read large sets of data; sometimes in the hundreds of thousands or millions of rows, e.g. associations between users and groups. Read capabilities are all expected to support a `"page"` field on their requests and responses. A response can have a `"token"` string attached; if it is, that token can be used by the caller on another request to get the next page of results.
# Deploying Lumos connectors There are two models for deploying connectors that Lumos can talk to; Lumos-hosted and Customer-hosted (on-prem) connectors. The connector SDK and connector capabilities are the same for different hosting models. The biggest difference is how the request for the capability gets to the connector, and how the response gets back to Lumos. ![](https://files.readme.io/8e1ea7d1042e60aa2fd58e89c89b30029887d7e75e9d7e746bef94002fe88d48-image.png)
The biggest difference, from the connector’s perspective, is that it gets called via CLI instead of directly loaded in a running Python process.
### How to run an on-prem agent locally See [On-Premise Agent](https://developers.lumos.com/docs/on-premise-agent-1) for details on running an on-prem agent on WIndows or Linux.
## Best Practises * Make use of the scaffolding and prepared class systems * When available, feel free to use external SDKs and libraries to connect with the end system * Connectors are idempotent by nature, and simple by design. Make sure you are within the context of a capability and only modifying necessary parameters for the capability to function. * Define your entitlements and resources clearly * Utilize pagination and consider using it as a minimal state store * Use pydantic for all models and DTOs, due to the describable nature of the connectors it is perfect for validation and documentation * Explore and make use of the SDKs utilities and helpers * Read the entire guide before beginning development ## Troubleshooting Tips 1. Test out each capability individually 1. You can call the capability through HTTP or CLI, make sure to set a proper log level if using the latter. 2. Check agent logs for errors 3. Run the capability through Swagger UI / Postman 4. Consult AI on the application docs and code to check problems 1. One of the great ways to provide context to an LLM is exporting the OpenAPI Specification of your connector, either using the `/docs` endpoint, or by calling `abc app_info --json '{"request": {}}'` and copying the schema in response. 2. Provide additional context from Lumos's developer documentation for the areas that are causing problems