Concepts

SemiLayer is an intelligence layer that sits between your application and your database. It reads your existing data, embeds it into vectors, and exposes semantic operations — search, similarity, feeds, and direct queries — through a typed client, REST, or WebSocket.

Your database stays where it is. SemiLayer never writes to it.

How it fits together

Search, similar, feed go through SemiLayer's vector index — your source is NOT hit at query time.

Query goes through the bridge — it reads directly from your source with the where predicates you pass. Opt in with grants.query on the lens.

Analyze goes through the bridge for aggregation (or reduces in-process when the bridge can't push down) and through the index for vector-narrowed candidates. The result is tidy buckets ready for any chart library, with drill-down rows replaying through the same RBAC + mapping pipeline query uses. Opt in with grants.analyze.<name> on the lens.

Organization · Project · Environment

Three nested scopes:

• Organization — your company or team. Billing, members, and encryption keys live here.
• Project — a logical application within an org. Namespaces lenses and API keys. One project per product or per microservice is common.
• Environment — a deployment stage within a project. Default is development. Production apps typically run development, staging, and production — each with its own sources, lenses, and API keys.

semilayer init stores your selection in .semilayerrc so subsequent CLI commands know where to operate.

→ See Environments for the full model, promotion flow, and diff tooling.

Sources & Bridges

A Source is a connection to an existing data source — a database, an API, a file store. A Bridge is the adapter that knows how to read from a specific source type. Your config names the bridge; the service holds the credentials.

sources: {
  'main-db': { bridge: '@semilayer/bridge-postgres' },
}

First-party bridge: @semilayer/bridge-postgres. Others (MySQL, MongoDB, SQLite, and more) are available as separate packages; community bridges can be built on the Bridge SDK.

→ See Connect a Source for every connection mode (managed, runner-local, airgap). → See Bridge SDK to build your own.

Lenses

A Lens is the core concept — a declaration of intelligence over a table. You define which fields to embed, which operations to expose, and who can access them.

products: {
  source: 'main-db',
  table: 'public.products',
  fields: {
    id:          { type: 'number',  primaryKey: true },
    name:        { type: 'text',    searchable: { weight: 2 } },
    description: { type: 'text',    searchable: true },
    category:    { type: 'enum',    values: ['footwear', 'apparel', 'accessories'] },
    price:       { type: 'number' },
  },
  grants: {
    search:  'public',
    similar: 'public',
  },
}

Field types: text, number, boolean, date, json, enum, relation.

searchable is opt-in. Only fields with searchable: true (or { weight: N }) are embedded. Others are stored as metadata — returned in results, filterable in query(), invisible to search().

Lens status moves through paused → indexing → ready → error.

→ See Schema (Config) for the full field/mapping/transform reference.

Operations

Operations are what a lens exposes. Each one is gated by a grant under grants.<op> on the lens — declare the grant and the operation is live.

Search and similar share the same vectors — both are powered by the top-level searchable flag on each FieldConfig, so there's no double-embed.

The Beam client

Beam is a typed client generated from your config. Run semilayer generate to produce a semilayer/ folder with per-lens classes.

import { createBeam } from './semilayer'

const beam = createBeam({
  baseUrl: 'https://api.semilayer.com',
  apiKey: process.env.SEMILAYER_API_KEY!,
})

// Fully typed — results[i].metadata.name is string, .price is number
const { results } = await beam.products.search({ query: 'shoes', limit: 10 })

// Live tail every insert/update/delete
for await (const event of beam.products.stream.subscribe()) {
  console.log(event.kind, event.record)
}

No codegen? @semilayer/client exposes the same methods on a BeamClient class — you pass lens as a string argument.

→ See Beam Client for the full API.

API keys

API keys authenticate requests. Each environment mints its own set. The environment slug is literal in the key prefix — no hardcoded dev/live.

So an environment named production mints keys like sk_production_<random>; an environment named staging mints sk_staging_<random>. The slug lets ops see at a glance which environment a leaked key came from.

Per-user scoping layers on top: pass an end-user JWT in the X-User-Token header (or via beam.withUser(token)) and the service enforces the lens's grants against that user's identity — see Auth & RBAC.

Ingest

Ingest is a background job coordinated through a durable job queue.

1. semilayer push --resume-ingest enqueues an ingest job.
2. The worker opens the bridge, reads the source in cursor-paged batches.
3. Each batch is embedded, indexed, and written to the vector store.
4. Lens status goes paused → indexing → ready.

Full (push --rebuild) re-indexes from scratch. Incremental (push --resume-ingest) resumes from the last cursor. For ongoing freshness, combine with syncInterval (cheap incremental poll), smartSyncInterval (scheduled full scan with tombstone detection), or webhook ingest triggered by your own changefeed.

→ See Push & Ingest for sync options, the dead- letter queue, and manual re-indexing.