Schema (Config)

The sl.config.ts file is the single source of truth for your SemiLayer setup. It declares your sources (which databases SemiLayer reads from), lenses (intelligent collections over a table or collection), the relations between them, and your auth configuration.

The platform stores credentials, status, and indexes; your sl.config.ts declares shape and is pushed up via semilayer push. Either side of that boundary can be edited through the Console — but the config file is what your generated Beam client is built from.

File location

SemiLayer searches for sl.config.ts (or sl.config.js) starting from the current working directory and walking up to the filesystem root. Place it at the root of your project.

Top-level shape

import { defineConfig } from '@semilayer/core'

export default defineConfig({
  stack: 'my-app',          // unique name for this stack — surfaced in generated code
  sources: { ... },         // database connections (declared here, credentialed in the platform)
  lenses: { ... },          // intelligent collections, with optional relations between them
  auth?: { ... },           // JWKS / access rule config for end-user JWTs
})

Sources

A Source declares a database connection. The key ('main-db' below) is the source name that lenses reference.

sources: {
  'main-db': {
    bridge: '@semilayer/bridge-postgres',  // required — the bridge package name
    timeout: 30000,                         // optional — query timeout in ms (default 30s)
  },
}

That's all sl.config.ts needs. Connection credentials live on the SemiLayer platform, not in this file — they're either stored encrypted (managed mode) or held by your runner (runner-local mode). See Connecting a source below.

Connecting a source — Console (recommended)

Open Console → Sources → Connect source. Pick your bridge, then choose a credentials location:

• Managed — paste the connection details. SemiLayer encrypts them at rest with AES-256-GCM and connects on your behalf (or via an assigned runner).
• Runner-local — no credentials stored. Set SEMILAYER_SOURCE_<NAME>_<KEY> env vars on the runner instead — _URL for URL-style bridges (Postgres, MySQL, MongoDB, Redis), or one var per config field (_HOST, _PORT, _DATABASE, _ACCESS_KEY_ID, …) for structured-config bridges. SemiLayer holds nothing. See Airgap mode.

After saving, the Console runs introspection and lists the available targets (tables / collections) so you can spot-check the connection.

Connecting a source — CLI

semilayer sources connect --name main-db --bridge @semilayer/bridge-postgres

You'll be prompted for the connection URL (or the per-field config the bridge declares). Pass --url "postgres://..." to skip the prompts in CI.

semilayer sources list
semilayer sources delete --source-id <id>

Multiple sources

Cross-source joins (a lens with a relation to another lens on a different source) are fully supported — the planner stitches results across bridges. Declare each source by name and reference both from the lens config:

sources: {
  'recipes-db':  { bridge: '@semilayer/bridge-postgres' },
  'reviews-db':  { bridge: '@semilayer/bridge-postgres' },
}

Lenses

A Lens is the unit of intelligence over a table. The key becomes the lens name (and the property name on your generated Beam client).

lenses: {
  products: {
    source: 'main-db',                   // which source to read from
    table: 'public.products',             // fully-qualified table / collection
    primaryKey: 'id',                     // optional shorthand — see Fields below
    fields: { ... },                      // field declarations
    feeds?: { ... },                      // named feed declarations
    grants?: { ... },                     // access control (see /guides/auth-and-rbac)
    relations?: { ... },                  // joins to other lenses
    syncInterval?: '15m',                 // periodic incremental sync
    smartSyncInterval?: '24h',            // scheduled smart sync (full scan + tombstones)
    changeTrackingColumn?: 'updated_at',  // column used to detect changes
    nullValues?: [],                      // values to treat as null at ingest time
  },
}

Fields

Fields declare what to read from the source and how to expose it. The key becomes the output field name; the value declares its type, mapping, and any transforms.

fields: {
  id:          { type: 'number', primaryKey: true },
  name:        { type: 'text', searchable: true },
  description: { type: 'text', searchable: true },
  tags:        { type: 'json' },
  category:    { type: 'text' },
  price:       { type: 'number' },
  createdAt:   { type: 'date' },
  isActive:    { type: 'boolean' },
}

Field types

Primary key

You can declare the primary key in two equivalent ways:

// At the field — modern, recommended
fields: {
  id: { type: 'number', primaryKey: true },
  // ...
}

// At the lens — legacy shorthand
primaryKey: 'id',
fields: {
  id: { type: 'number' },
  // ...
}

If neither is specified, SemiLayer falls back to a column named id.

Marking fields for embedding

Add searchable: true to include a field in the semantic search embedding:

name:        { type: 'text', searchable: true },               // weight 1 (default)
description: { type: 'text', searchable: { weight: 2 } },      // 2x weight in the embedding

Higher-weighted fields are repeated in the embedding input, boosting their relevance signal in search results.

Column mapping (from)

When the source column name differs from your desired output field name:

displayName: { type: 'text', from: 'product_name', searchable: true },

When a field is built from multiple source columns, declare a merge strategy:

fullAddress: {
  type: 'text',
  from: ['address_line1', 'address_line2', 'city'],
  merge: 'concat',
  separator: ', ',
  searchable: true,
}

Transforms

Apply a transform chain to the resolved value before indexing:

priceDollars: {
  type: 'number',
  from: 'price_cents',
  transform: { type: 'round', decimals: 2 },
}

slug: {
  type: 'text',
  from: 'title',
  transform: [
    { type: 'lowercase' },
    { type: 'replace', pattern: '\\s+', replacement: '-' },
    { type: 'truncate', length: 100 },
  ],
  searchable: true,
}

Available transforms

Null handling

nullAs and undefinedAs per-field, plus nullValues at the lens level, control how SemiLayer treats missing data at ingest time.

fields: {
  rating: { type: 'number', nullAs: 0 },               // null  -> 0
  notes:  { type: 'text',   undefinedAs: 'omit' },     // missing -> drop the key
}
nullValues: ['', 'NULL', 'N/A'],                       // treat these source values as null

Operations

A lens exposes a fixed set of operations: search, similar, query, and any named feeds you declare under feeds. Each operation is enabled by adding a grant under grants.<op>. The embedding inputs for search and similar come from the searchable flag on each field — per-field weighting via searchable: { weight: N }. Search mode is a request-time choice:

await beam.products.search({ query: '...', mode: 'hybrid' })
// or the sugar method:
await beam.products.searchHybrid({ query: '...' })

Post-mapping projection is also request-time — pass fields: ['title', 'body'] to search, similar, query, feed, or their stream variants to shape the response payload.

Relations

Lenses can declare relations to other lenses, including across different sources. The join planner resolves them at request time when callers pass an include clause to search, query, similar, stream.search, or stream.query.

lenses: {
  recipes: {
    source: 'recipes-db',
    table: 'recipes',
    // ... fields, grants ...
    relations: {
      reviews: { lens: 'reviews', kind: 'hasMany',   on: { id: 'recipe_id' }, defaultIncludeLimit: 50 },
      author:  { lens: 'authors', kind: 'belongsTo', on: { author_id: 'id' } },
    },
  },
}

hasMany returns an array per parent row; belongsTo returns at most one. The on mapping is { [localColumn]: foreignColumn }. See Joins for the full reference including cross-source patterns and access-rule composition.

Sync configuration

Keep the index fresh as your source data changes.

syncInterval — Periodic incremental sync (fast, cheap, no deletes). Valid values: '1m' | '5m' | '15m' | '30m' | '1h' | '6h' | '24h'.

syncInterval: '5m',  // fast incremental refresh every 5 minutes

smartSyncInterval — Scheduled smart sync with tombstone detection. Same 7 values. Runs a full-table scan with content-hash dedup on top of the same handler the Console "Sync now" button uses. Independent of syncInterval — pair them for comprehensive auto-freshness:

syncInterval: '5m',           // fast partial refresh
smartSyncInterval: '24h',     // nightly tombstone sweep

Subject to tier floor and monthly cap — see Keeping data fresh.

changeTrackingColumn — The column used to detect new/updated rows during incremental sync. Defaults to updated_at.

changeTrackingColumn: 'modified_at',

For change-driven (rather than poll-driven) ingestion, see the ingest webhook flow under Push & Ingest — push records as they change with an ik_ ingest key.

Grants (access rules)

See Auth & RBAC for the full reference. Quick example:

grants: {
  search:  'public',             // anyone can search
  similar: 'public',
  query:   'authenticated',      // user JWT required for direct queries
  stream: { enabled: true, modes: ['chunked', 'live'] },
}

query is off by default — set a grant to enable the /v1/query/:lens route.

Auth

Configure JWKS validation for end-user JWTs. This enables 'authenticated' grants and function grants that receive user claims.

auth: {
  client: {
    jwt: {
      jwksUrl: 'https://your-issuer/.well-known/jwks.json',
      issuer:  'https://your-issuer/',
      audience: 'https://your-api/',  // optional
    }
  }
}

The same fields can be configured per-environment via the Console / CLI without touching sl.config.ts. JWKS pulled from the file is the developer-experience path; JWKS pulled from the platform is the production-rotation path.

Complete example

import { defineConfig } from '@semilayer/core'

export default defineConfig({
  stack: 'storefront',

  sources: {
    'main-db': { bridge: '@semilayer/bridge-postgres' },
  },

  lenses: {
    products: {
      source: 'main-db',
      table: 'public.products',
      syncInterval: '15m',
      changeTrackingColumn: 'updated_at',

      fields: {
        id:           { type: 'number', primaryKey: true },
        name:         { type: 'text', searchable: { weight: 2 } },
        description:  { type: 'text', searchable: true },
        brand:        { type: 'text', searchable: true },
        category:     { type: 'text' },
        priceDollars: {
          type: 'number',
          from: 'price_cents',
          transform: { type: 'round', decimals: 2 },
        },
        tags:         { type: 'json' },
        isActive:     { type: 'boolean' },
        createdAt:    { type: 'date', from: 'created_at' },
      },

      grants: {
        search:  'public',
        similar: 'public',
        query:   'authenticated',
      },
    },
  },

  auth: {
    client: {
      jwt: {
        jwksUrl: 'https://your-issuer/.well-known/jwks.json',
        issuer:  'https://your-issuer/',
      },
    },
  },
})

Drift detection

When you run semilayer push, SemiLayer hashes your lens config and compares it against the last pushed hash. If the config drifted, the CLI flags it:

semilayer push
# ! Lens "products" config has drifted from last push
# -> run with --rebuild to re-index all records with the new config

Changes to fields (adding, removing, or changing a searchable field) require a rebuild because the embedding text changed. Changes to grants, feeds, syncInterval, or smartSyncInterval take effect immediately without a rebuild — the tick handler picks them up on the next pass.