SemiLayerDocs

Connect a Source

A Source is a connection to an existing data source — a database, API, file store, or anything a Bridge can read from. Sources are scoped to an Environment and their credentials are stored encrypted per-organization.


How should SemiLayer reach your database?

Before you fill in connection details, decide how SemiLayer will get to your database. There are two answers, and the right one depends on whether your database is reachable from the public internet.

ℹ️

If your database is not publicly reachable — sitting behind a corporate VPN, in a private VPC subnet, or in any environment that won't allowlist external IPs — use a runner. It's the dramatically simpler answer for the security review and the simpler answer operationally. See Runners — Overview.

Your DB situationPick this
Publicly reachable (or behind a firewall you can allowlist)Managed credentials — fastest setup. SemiLayer connects outbound from a small set of static egress IPs you allowlist. Connection string lives encrypted on the SemiLayer side.
Not publicly reachable — VPN, private VPC, on-premManaged credentials + runner — SemiLayer holds the (encrypted) connection string and dispatches jobs to a runner you run inside your network. Outbound-only WebSocket; no inbound to your network ever.
Credentials must never leave your machine — security review wants ironclad proof, regulated industry, etc.Runner-local credentials + runner (the "airgap" mode) — SemiLayer stores {} for the source. Your runner reads the DB URL from its own env. Even our database has no copy of your connection string. See Airgap mode.

A runner is one small process — a Docker container or an npm install -g. It opens one outbound WebSocket and stays idle until you query. You can switch between modes later by toggling Credentials location on the source.


What You'll Need

  • A running SemiLayer environment (see Organization Setup)
  • A bridge package for your data source type (e.g. @semilayer/bridge-postgres)
  • Connection credentials (URL, API key, etc.)

Connect via the CLI

semilayer sources connect

The wizard prompts for:

  1. Bridge — the package name (e.g. @semilayer/bridge-postgres)
  2. Source name — the identifier used in sl.config.ts (e.g. main-db)
  3. Connection details — URL or other bridge-specific config

After connecting, SemiLayer introspects the source and lists available tables, collections, or endpoints.


Connect via the Console

Settings → Sources → Connect source

Fill in the same fields. The dialog asks you to pick a Credentials location:

  • Managed — paste the connection details; SemiLayer encrypts them at rest and connects on your behalf (or via an assigned runner).
  • Runner-local — no fields to fill in. SemiLayer stores {} for the source. Set SEMILAYER_SOURCE_<NAME>_URL on your runner instead. See Airgap mode.

After saving, the Console shows the introspected targets and their schema (column names, types, estimated row count). For runner-local sources, introspection runs through your runner — make sure the runner is online and assigned to the source first.


Reference in sl.config.ts

Once a source is connected, reference it by name in your config:

import { defineConfig } from '@semilayer/core'

export default defineConfig({
  stack: 'my-app',
  sources: {
    'main-db': {
      bridge: '@semilayer/bridge-postgres',
    },
  },
  lenses: {
    products: {
      source: 'main-db',      // ← matches the source name above
      table: 'public.products',
      // ...
    },
  },
})

The connection string itself lives in the service (stored encrypted) — not in sl.config.ts. Your config only references the bridge type and source name.


Available Bridges

23 first-party bridges ship today from the bridges monorepo. Every major relational store, every popular document/key-value system, the columnar warehouses, and the edge-native platforms:

Relational
Postgres
@semilayer/bridge-postgres
MySQL
@semilayer/bridge-mysql
MariaDB
@semilayer/bridge-mariadb
Oracle
@semilayer/bridge-oracle
SQLite
@semilayer/bridge-sqlite
SQL Server
@semilayer/bridge-mssql
CockroachDB
@semilayer/bridge-cockroachdb
Neon
@semilayer/bridge-neon
Turso
@semilayer/bridge-turso
PlanetScale
@semilayer/bridge-planetscale
D1
@semilayer/bridge-d1
Supabase
@semilayer/bridge-supabase
Document
MongoDB
@semilayer/bridge-mongodb
Firestore
@semilayer/bridge-firestore
Key-value
Redis
@semilayer/bridge-redis
Upstash
@semilayer/bridge-upstash
Columnar / analytical
ClickHouse
@semilayer/bridge-clickhouse
BigQuery
@semilayer/bridge-bigquery
Snowflake
@semilayer/bridge-snowflake
DuckDB
@semilayer/bridge-duckdb
Wide-column
Cassandra
@semilayer/bridge-cassandra
DynamoDB
@semilayer/bridge-dynamodb
Search
Elasticsearch
@semilayer/bridge-elasticsearch

Bridges are installed as npm packages under @semilayer/bridge-<name>. The worker resolves them at runtime via @semilayer/bridge-resolver. Missing one? The Bridge SDK is how you ship a new adapter.


Multiple Sources

A single environment can have multiple sources. Each lens declares which source it reads from:

sources: {
  'postgres-main': { bridge: '@semilayer/bridge-postgres' },
  'postgres-analytics': { bridge: '@semilayer/bridge-postgres' },
},
lenses: {
  products:  { source: 'postgres-main',      table: 'products', /* ... */ },
  pageviews: { source: 'postgres-analytics', table: 'pageviews', /* ... */ },
},

Managing Sources

semilayer sources list           # list all sources in the current environment
semilayer sources remove --name main-db

Or from the Console: Settings → Sources.


Next Steps

With a source connected:

  1. Schema (Config) — define lenses over your source
  2. Push & Ingest — push the config and start indexing