Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

alecgrieser/fdb-record-layer

Open more actions menu
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5,223 Commits
5,223 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

FoundationDB logo

FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as an ordered key-value store and employs ACID transactions for all operations. It is especially well-suited for read/write workloads but also has excellent performance for write-intensive workloads. Users interact with the database using API language binding.

To learn more about FoundationDB, visit foundationdb.org

FoundationDB Record Layer (FRL)

FRL provides a relational database with SQL support built on top of FoundationDB, featuring:

  • SQL Database - SQL support with JDBC connectivity for defining schemas, querying data, and managing tables using familiar SQL syntax. The SQL API is under active development with frequent enhancements.
  • Advanced Data Types - Beyond standard SQL types (STRING, INTEGER, FLOAT, BOOLEAN), FRL supports:
    • Nested Structures - User-defined struct types that can be nested arbitrarily deep
    • Arrays - Collections of primitives or complex types
    • Vectors - Fixed-dimension numerical vectors for ML embeddings and similarity search
  • Schema Templates - Reusable schema definitions that enable multi-tenant architectures where each tenant gets their own database instance with a shared, evolvable schema.
  • Intelligent Query Planning - Automatic index selection and query optimization with support for JOINs, aggregations (COUNT, SUM, etc.), GROUP BY, and ORDER BY. Queries are efficiently executed using index-backed operations without in-memory sorting.
  • Indexes - Rich indexing capabilities including value indexes, rank indexes, aggregate indexes, and indexes on nested fields. Indexes are materialized views that update incrementally.
  • Scalable Architecture - Designed for distributed, stateless environments with millisecond-level store initialization. Perfect for applications managing thousands of discrete database instances.
  • ACID Transactions - Full transactional semantics inherited from FoundationDB, with support for continuations for efficiently paging through large result sets.

Quick Start with SQL

// Connect via JDBC
String url = "jdbc:embed:/__SYS?schema=CATALOG";
Connection conn = DriverManager.getConnection(url);

// Define a schema template with tables and indexes
conn.createStatement().execute("""
    CREATE SCHEMA TEMPLATE my_template
        CREATE TABLE customers (
            customer_id BIGINT,
            name STRING,
            email STRING,
            PRIMARY KEY(customer_id)
        )
        CREATE INDEX email_idx AS
            SELECT email FROM customers ORDER BY email
    """);

// Create a database and schema
conn.createStatement().execute(
    "CREATE DATABASE /my_app/production");
conn.createStatement().execute(
    "CREATE SCHEMA /my_app/production/main WITH TEMPLATE my_template");

// Insert and query data
PreparedStatement insert = conn.prepareStatement(
    "INSERT INTO customers VALUES (?, ?, ?)");
insert.setLong(1, 1);
insert.setString(2, "Alice");
insert.setString(3, "alice@example.com");
insert.executeUpdate();

ResultSet rs = conn.createStatement().executeQuery(
    "SELECT * FROM customers WHERE email = 'alice@example.com'");

Key Features

Multi-Tenant Schema Templates

Schema templates enable efficient multi-tenant architectures:

-- Define the template once
CREATE SCHEMA TEMPLATE user_data_template
    CREATE TABLE documents (id BIGINT, content STRING, PRIMARY KEY(id))
    CREATE INDEX content_idx AS SELECT content FROM documents ORDER BY content;

-- Create separate database instances for each tenant
CREATE DATABASE /tenant/user_1;
CREATE SCHEMA /tenant/user_1/data WITH TEMPLATE user_data_template;

CREATE DATABASE /tenant/user_2;
CREATE SCHEMA /tenant/user_2/data WITH TEMPLATE user_data_template;

Each tenant's data is completely isolated with its own database, yet all share the same schema definition for easy management and evolution.

Advanced Type System

Define complex, nested data structures:

-- Define custom struct types
CREATE TYPE AS STRUCT address (
    street STRING,
    city STRING,
    postal_code STRING
)

CREATE TYPE AS STRUCT contact_info (
    email STRING,
    phone STRING,
    mailing_address address
)

-- Use in tables with arrays and nesting
CREATE TABLE users (
    user_id BIGINT,
    name STRING,
    contacts contact_info ARRAY,
    PRIMARY KEY(user_id)
)

Vector Support for ML Applications

Store and query high-dimensional vectors for embeddings and similarity search:

CREATE TABLE embeddings (
    doc_id BIGINT,
    content STRING,
    embedding_half VECTOR(128, HALF),     -- 16-bit precision
    embedding_float VECTOR(768, FLOAT),   -- 32-bit precision
    embedding_double VECTOR(1024, DOUBLE), -- 64-bit precision
    PRIMARY KEY(doc_id)
)

Vectors are inserted via JDBC PreparedStatements and can be efficiently stored and retrieved using the FoundationDB backend.

Index-Backed Query Execution

FRL's query planner intelligently selects indexes to execute queries efficiently:

-- Queries use indexes automatically
SELECT name FROM customers WHERE email = 'alice@example.com';
-- Uses email_idx if available

-- JOINs using comma-separated FROM clause
SELECT c.name, o.order_id
FROM customers c, orders o
WHERE c.customer_id = o.customer_id;

-- Aggregations backed by indexes
SELECT category, COUNT(*)
FROM products
GROUP BY category;
-- Requires ordered index or primary key on category for streaming aggregate, or a aggregate index for direct retrieval

-- ORDER BY requires index or primary key order
SELECT * FROM customers ORDER BY email;
-- Requires index on email (like email_idx above)

Important: FRL does not perform in-memory sorting or aggregation. Operations like ORDER BY, GROUP BY, and aggregates require underlying indexes to provide the required ordering.

Architecture Notes

FRL is designed for:

  • Horizontal scalability - Thousands of independent database instances
  • Low latency - Millisecond-level initialization and query execution
  • Stateless services - No server-side state; all data in FoundationDB
  • Schema evolution - Templates can evolve over time (template evolution features coming to relational layer; currently available via advanced Record Layer API)

Advanced: Direct Record Layer API

For applications requiring fine-grained control over storage layout, index maintenance, or features not yet available in the SQL Relational layer, the Record Layer provides a low-level Java API using Protocol Buffers.

Note: This API is maintained for advanced use cases but is being positioned as a lower-level alternative to the SQL interface. Features available only through this API will migrate to the SQL layer over time. Long-term support of this lower-level API is not guaranteed once equivalent features are available at the Relational Layer.

Key Record Layer API features:

  • Protobuf-based schema definition - Define records using .proto files
  • Programmatic index management - IndexMaintainer extension points
  • Custom query components - Extend the query planner
  • Schema evolution - MetaDataEvolutionValidator for safe schema changes
  • Low-level control - Direct access to FoundationDB operations

See Record Layer Documentation for details.

Documentation

Getting Help

About

A record-oriented store built on FoundationDB

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages

  • Java 80.3%
  • Prolog 19.3%
  • Other 0.4%
Morty Proxy This is a proxified and sanitized view of the page, visit original site.