README

TOML Parser for Free Pascal

A robust TOML (Tom's Obvious, Minimal Language) parser and serializer for Free Pascal with full TOML v1.1.0 support, including nested tables, arrays of tables, multi-line inline tables, quoted dotted keys, \e and \xHH escapes, optional seconds in times, and strict table-definition rules.

[!NOTE]

Our test suite includes 94 project tests plus a pinned TOML 1.1 conformance gate that passes every official decoder case. See Testing for the measured support baseline.

Table of Contents

• TOML Parser for Free Pascal
  • Table of Contents
  • Overview
  • Features
  • To Do / In Progress
  • Requirements
  • Installation
    • Steps
  • Quick Start
    • Basic Usage
    • Writing TOML Files
  • Documentation
    • Common Use Cases
      • Working with Arrays
      • Nested Tables
      • Hierarchical Paths vs Literal Dotted Keys
      • Serializing Complex Structures
    • Memory Management
      • Creating Tables
      • Adding Nested Tables or Values
      • Freeing Tables:
      • Avoid Explicitly Freeing Nested Objects:
  • API Reference
    • Types
    • Helper Functions for Creating TOML Values
    • Parsing Functions
    • Serialization Functions
  • Testing
    • Test Coverage Overview
    • Sample Test Output
  • Examples
  • Contributing
  • License
  • Acknowledgments

Overview

TOML-FP provides a complete solution for working with TOML configuration files in Free Pascal applications. Whether you need to read configuration files or generate them, TOML-FP offers an intuitive API with comprehensive type support and robust error handling.

Features

• TOML v1.1.0 Support: Decodes all 214 valid cases and rejects all 467 invalid cases of the official toml-test v2.2.0 suite. Valid TOML 1.0 documents remain valid.
• Type-Safe API: Strong typing with compile-time checks
• Memory Management: Automatic cleanup with proper object lifecycle management
• Error Handling: Detailed error messages and exception handling
• Serialization: Convert Pascal objects to TOML and back
• Documentation: Practical examples and API documentation
• Test Suite: 94 project tests plus the official toml-test decoder suite

What TOML 1.1 adds

TOML-FP v1.1.0 accepts the three syntax additions of TOML v1.1.0:

# \e (escape) and \xHH (code points up to U+00FF) in basic strings
prompt = "\e[1mready\e[0m"
latin  = "caf\xe9"

# Seconds are optional in times and date-times
start = 07:32
meeting = 2026-03-14 09:30-07:00

# Inline tables may span lines, carry comments, and end with a comma
server = {
  host = "localhost",   # primary
  ports = [8080, 8081],
}

Compatibility notes:

• There is no TOML 1.0-only strict mode; documents using the syntax above are accepted.
• The serializer always writes seconds (for example 07:32:00), so its output stays readable by TOML 1.0 parsers.
• Input must be valid UTF-8 (a leading byte order mark is skipped), and redefining a table, extending an inline table or extending a static array now raises ETOMLParserException, as the specification requires.

To Do / In Progress

• [ ] More tests for error handling: Robust testing of invalid inputs and ensuring appropriate error messages or handling mechanisms.
• [ ] Add official encoder conformance coverage
• [ ] Performance optimization for large TOML files
• [ ] Additional examples for common use cases

Requirements

• Free Pascal Compiler 3.2.2 or later
• Lazarus IDE 4.8 or later (required for package and test project builds)

Installation

Steps

1. Clone the repository:
   bashCopy

   git clone https://github.com/ikelaiah/toml-fp.git

2. Add to your project:

   • Copy the src directory to your project.

   • Add the src directory to your project's unit search path.

   • Add the main TOML unit to your uses clause:

     pascalCopy

     uses
       TOML;

   • For direct access to specific types like TTOMLValueType or if you prefer explicit qualification, you might also include TOML.Types:

     pascalCopy

     uses
       TOML, TOML.Types; // If needing TTOMLValueType, etc.

     However, for most common operations (parsing, serializing, using helper functions like TOMLTable), uses TOML; is sufficient.

Quick Start

Basic Usage

program BasicParseTOML;

{$mode objfpc}{$H+}{$J-}

uses
  TOML;

var
  Config: TTOMLTable;
  RevisionValue, ProjectValue, ProjectName: TTOMLValue;
  ProjectTable: TTOMLTable;

begin
  // Parse TOML from file
  Config := ParseTOMLFromFile('config.toml');
  try

    // Access a string value
    if (Config.TryGetValue('revision', RevisionValue)) then
      WriteLn('The value of ''revision'' is ', RevisionValue.AsString);

    // Access nested values safely
    if Config.TryGetValue('project', ProjectValue) and
      (ProjectValue is TTOMLTable) then
    begin
      ProjectTable := TTOMLTable(ProjectValue);
      if ProjectTable.TryGetValue('name', ProjectName) then
        WriteLn('Project Name: ', ProjectName.AsString)
      else
        WriteLn('Project name not found');
    end
    else
      WriteLn('Project configuration not found');
  finally
    Config.Free;
  end;
end.

Example config.toml:

# config.toml

revision = "1.2.1af"

[project]
name = "My Amazing Project"
version = "1.0.0"

Writing TOML Files

program BasicSerializeTOML;

{$mode objfpc}{$H+}{$J-}

uses
  TOML;

var
  Config: TTOMLTable;
  Database: TTOMLTable;
begin
  Config := TOMLTable;
  try
    Database := TOMLTable;
    Database.Add('host', TOMLString('localhost'));
    Database.Add('port', TOMLInteger(5432));
    Config.Add('database', Database);

    if SerializeTOMLToFile(Config, 'config.toml') then
      WriteLn('Configuration saved successfully')
    else
      WriteLn('Error saving configuration');
  finally
    Config.Free;
  end.
end.

Documentation

Common Use Cases

Working with Arrays

var
  Config: TTOMLTable;
  Tags: TTOMLArray;
begin
  Config := TOMLTable;
  try
    Tags := TOMLArray;
    Tags.Add(TOMLString('pascal'));
    Tags.Add(TOMLString('toml'));
    Config.Add('tags', Tags);
    
    WriteLn(SerializeTOML(Config));
  finally
    Config.Free;
  end.
end.

Nested Tables

var
  Config: TTOMLTable;
  Database, Server: TTOMLTable;
  SerializedTOML: string;
begin
  Config := TOMLTable;
  try
    Database := TOMLTable;
    Database.Add('host', TOMLString('localhost'));
    Database.Add('port', TOMLInteger(5432));
    
    Server := TOMLTable;
    Server.Add('database', Database); // Nest Database table under Server
    
    Config.Add('server', Server);     // Nest Server table under Config
    
    SerializedTOML := SerializeTOML(Config);
    WriteLn(SerializedTOML);
    // Expected output:
    // [server.database]
    // host = "localhost"
    // port = 5432
  finally
    Config.Free; // Frees Config, Server, and Database tables
  end.
end.

Hierarchical Paths vs Literal Dotted Keys

TOML-FP correctly handles the distinction between hierarchical paths and literal dotted keys as per the TOML specification:

1. Hierarchical Table Paths:

   pascalCopy

   // This creates a nested hierarchy: [server.database]
   ServerTable := TOMLTable;
   DatabaseTable := TOMLTable;
   DatabaseTable.Add('port', TOMLInteger(5432));
   
   ServerTable.Add('database', DatabaseTable);
   Config.Add('server', ServerTable);

   Serializes to:

   tomlCopy

   [server.database]
   port = 5432

2. Literal Dotted Keys:

   pascalCopy

   // This creates a literal dotted key: ["server.database"]
   LiteralTable := TOMLTable;
   LiteralTable.Add('port', TOMLInteger(5432));
   
   Config.Add('server.database', LiteralTable);

   Serializes to:

   tomlCopy

   ["server.database"]
   port = 5432

The serializer automatically determines whether to quote key components based on the TOML specification rules.

[!TIP]
According to the TOML specification: "Quoted keys follow the exact same rules as either basic strings or literal strings and allow you to use a much broader set of key names. Best practice is to use bare keys except when absolutely necessary."

In practice, this means you should design your configuration to use hierarchical nesting with simple key names rather than relying on keys with dots or special characters.

Serializing Complex Structures

program BasicSerializeTOML;

{$mode objfpc}{$H+}{$J-}

uses
  TOML, SysUtils;

var
  Config, ServerConfig: TTOMLTable;
  Ports: TTOMLArray;
  SerializedTOML: string;
begin
  Config := TOMLTable;
  try
    // Create a nested table
    ServerConfig := TOMLTable;
    ServerConfig.Add('host', TOMLString('127.0.0.1'));
    ServerConfig.Add('enabled', TOMLBoolean(True));

    // Create and populate an array
    Ports := TOMLArray;
    Ports.Add(TOMLInteger(80));
    Ports.Add(TOMLInteger(443));
    ServerConfig.Add('ports', Ports);

    // Add the server config to main config
    Config.Add('server', ServerConfig);

    // Add some basic metadata
    Config.Add('version', TOMLFloat(1.0));
    Config.Add('last_updated', TOMLDateTime(Now));

    // Serialize to TOML format
    SerializedTOML := SerializeTOML(Config);
    WriteLn('Generated TOML:');
    WriteLn(SerializedTOML);

    // Save to file
    if SerializeTOMLToFile(Config, 'config.toml') then
      WriteLn('Successfully saved to file')
    else
      WriteLn('Error saving to file');

  finally
    Config.Free;
  end.
end.

This will generate TOML like:

version = 1.0
last_updated = 2024-03-20T15:30:45Z

[server]
host = "127.0.0.1"
enabled = true
ports = [ 80, 443 ]

Note: All values are properly type-checked and memory-managed. The library ensures that:

• Each value has the correct TOML type
• Arrays maintain type consistency
• All objects are properly freed
• Type conversions are validated

Memory Management

[!NOTE]

Only free the top-level owner table to avoid memory management issues.

• Proper memory management is essential to prevent memory leaks in your applications. TOML-FP follows a clear ownership hierarchy where parent tables own their nested tables and values. Here's how it works:

Creating Tables

• When you create a TTOMLTable, it acts as an owner for any nested tables or values you add to it.

Adding Nested Tables or Values

• Use the Add method to insert nested tables or values into a parent table.
• Example:

var
  Config: TTOMLTable;
  Database: TTOMLTable;
begin
  Config := TOMLTable;
  Database := TOMLTable;
  Config.Add('database', Database);
  // ...
  Config.Free; // This frees all nested tables and values
end.

Freeing Tables:

• Only free the top-level owner table.
• Frees all nested tables and values automatically.
• Example:

var
  Config: TTOMLTable;
begin
  Config := TOMLTable;
  // ... add nested tables and values ...
  Config.Free; // This frees all nested tables and values
end.

Avoid Explicitly Freeing Nested Objects:

• Do not manually free nested tables or values to prevent memory management issues.

API Reference

Types

• TTOMLValue - Base type for all TOML values
• TTOMLTable - Represents a TOML table
• TTOMLArray - Represents a TOML array
• TTOMLString - Represents a TOML string value
• TTOMLInteger - Represents a TOML integer value
• TTOMLFloat - Represents a TOML float value
• TTOMLBoolean - Represents a TOML boolean value
• TTOMLDateTime - Represents a TOML datetime value

Helper Functions for Creating TOML Values

• TOMLString

  function TOMLString(const AValue: string): TTOMLString;

Creates a TOML string value.

• TOMLInteger

  function TOMLInteger(const AValue: Int64): TTOMLInteger;

Creates a TOML integer value.

• TOMLFloat

  function TOMLFloat(const AValue: Double): TTOMLFloat;

Creates a TOML float value.

• TOMLBoolean

    function TOMLBoolean(const AValue: Boolean): TTOMLBoolean;

Creates a TOML boolean value.

• TOMLDateTime

    function TOMLDateTime(const AValue: TDateTime): TTOMLDateTime;

Creates a TOML datetime value.

• TOMLArray

  function TOMLArray: TTOMLArray;

Creates a TOML array.

• TOMLTable

  function TOMLTable: TTOMLTable;

Creates a TOML table.

Parsing Functions

• ParseTOML

  function ParseTOML(const ATOML: string): TTOMLTable;

Parses a TOML-formatted string into a TTOMLTable object. Raises ETOMLParserException on error.

• ParseTOMLFromFile

  function ParseTOMLFromFile(const AFileName: string): TTOMLTable;

Parses a TOML file into a TTOMLTable object. Reads the file content and then parses it. Raises ETOMLParserException on parsing errors or file system errors.

Serialization Functions

• SerializeTOML

  function SerializeTOML(const AValue: TTOMLValue): string;

Serializes a TTOMLValue (typically a TTOMLTable for a whole document) into a TOML-formatted string.

• SerializeTOMLToFile

  function SerializeTOMLToFile(const AValue: TTOMLValue; const AFileName: string): Boolean;

Serializes a TTOMLValue and saves it to the specified file. Returns True on success, False on failure (e.g., file I/O error).

Testing

The library includes 94 project tests. The v1.1.0 release was validated locally with Lazarus 4.8 and FPC 3.2.2. GitHub Actions uses the same toolchain to build the Lazarus package and run clean Debug and Release test builds on Linux and Windows.

To run the tests:

From the repository root:

lazbuild --build-all --build-mode=Debug tests/TestRunner.lpi
tests/TestRunner --all --format=plain
lazbuild --build-all --build-mode=Release tests/TestRunner.lpi
tests/TestRunner --all --format=plain

Use tests\\TestRunner.exe on Windows. You can also open tests/TestRunner.lpi in Lazarus and run either build mode.

The pinned toml-test conformance gate runs against the TOML 1.1 profile and currently passes 214/214 valid decoder cases and rejects 467/467 invalid cases, with no known gaps. Encoder conformance is not yet covered.

Test Coverage Overview

See Test-Coverage-Overview.md for details.

• Basic Types:
  • Strings, Integers, Floats, Booleans, DateTime
• Advanced Structures:
  • Multiline and Literal Strings
  • Homogeneous and Mixed-Type Arrays
  • Nested and Inline Tables
• Serialization:
  • Accurate conversion of Pascal objects to TOML format and vice versa
• Error Handling:
  • Robust tests for invalid inputs, duplicate keys, and other edge cases

Sample Test Output

$ ./TestRunner --all --format=plain
 Time:00.032 N:94 E:0 F:0 I:0
  TTOMLTestCase Time:00.032 N:94 E:0 F:0 I:0
    Test01_StringValue
    Test02_IntegerValue
    ...
    Test96_SerializerEscapesDelete

Number of run tests: 94
Number of errors:    0
Number of failures:  0

Examples

Check out the examples directory and the test cases, tests/TestCaseTOML.pas, for more detailed usage examples:

• Basic Configuration Reading
• Writing Complex Data Structures
• Parser error cases in the test suite
• Integration Examples

Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Please ensure:

• Code follows the project's style guidelines
• All tests pass
• New features include appropriate tests
• Documentation is updated

License

This project is licensed under the MIT License - see the LICENSE.md file for details.

Acknowledgments

• TOML specification creators and maintainers
• Free Pascal and Lazarus communities
• SilenceCFF
• All contributors to this project