README

LlamaCppLib library

Object Pascal Notebook Llama.CPP library.

This library allows you to use the power of Llama.CPP within Object Pascal Notebook.

The library provides the following units:

• uLlamaCpp.pas

Prepare the library

To use the library, we must first compile the Delphi LlamaCPPLib.dproj dynamic library project. The project uses Delphi's FMX framework to enable the library to be used on different operating systems. The compilation will create the dynamic library in the dist folder, which also contains the uLlamaCPP.pas lib interface unit.

Get the dependencies

For compiling the project, we need some dependencies. The below script batch allow us to get them:

curl -L https://raw.githubusercontent.com/ElCondor1969/ObjectPascalNotebook/main/src/OPNBHost/uLibInterface.pas -o dependencies/ObjectPascalNotebook/uLibInterface.pas

curl -L https://raw.githubusercontent.com/ElCondor1969/ObjectPascalNotebook/main/src/OPNBHost/uDynLibLoader.pas -o dependencies/ObjectPascalNotebook/uDynLibLoader.pas

curl -L https://raw.githubusercontent.com/ElCondor1969/ObjectPascalNotebook/main/src/OPNBHost/uUtility.pas -o dependencies/ObjectPascalNotebook/uUtility.pas

Get the LlamaCPP libraries

The dist folder must contain the llamacpp subfolder, into which we'll place the Llama.CPP dynamic libraries compatible with our machine's operating system and hardware resources. These libraries can be retrieved from the Release section of the Llama.CPP repository.

Import the library into your Object Pascal Notebook

To import the library, use the following instructions:

Note that the library depends on the LLMBaseTypesLib and that this library must therefore also be present.

uLlamaCpp Unit Documentation

This unit provides a Delphi wrapper for interacting with the Llama.cpp library. It defines configuration types, parameter classes, and the main TLlama class used to load models, tokenize text, and generate completions.

Enumerations

TLlamaAttentionType

Defines the type of attention mechanism used by the model.

TLlamaFlashAttentionType

Controls Flash Attention optimization.

TLlamaSplitMode

Defines how computation is split across GPUs.

TLlamaRopeScalingType

Controls RoPE (Rotary Positional Encoding) scaling strategy.

TLlamaPoolingType

Defines pooling strategies for embeddings or classification.

TGgmlLogLevel

Log severity levels.

TGGMLNumaStrategy

Defines NUMA memory handling strategies.

TGGMLType

Defines tensor data types and quantization formats used in GGML.

Includes:

• Floating point types (F32, F16, BF16, F64)
• Integer types (I8, I16, I32, I64)
• Quantized formats (Q4_0, Q5_1, Q8_0, IQ*, etc.)

Used for memory optimization and inference performance tuning.

Classes

TLlamaModelParams

Represents model-level configuration.

Constructor

constructor Create(AJSON: JSONvariant);

Initializes parameters from a JSON structure.

Properties

TLlamaContextParams

Defines runtime context parameters.

Constructor

constructor Create(AJSON: JSONvariant);

Properties

TCompletationParams

Controls text generation behavior.

Constructor

constructor Create(AJSON: JSONvariant);

Properties

TLlamaSettings

Aggregates all configuration sections.

Constructor

constructor Create(AJSON: JSONvariant);

Methods

function ToString: string;

Returns a string representation of the settings.

Properties

Callback Types

Logging

TLLamaLogCallback = procedure(Sender: TObject; const LogLevel: TGgmlLogLevel; const ALog: string) of object;
TLLamaLogCallback2 = procedure(const LogLevel: TGgmlLogLevel; const ALog: string);

Triggered when the library emits logs.

Token Streaming

TLlamaTokenCallback = procedure(Sender: TObject; const AToken: integer; const ATextToken: string; var AContinue: boolean) of object;
TLlamaTokenCallback2 = procedure(const AToken: integer; const ATextToken: string; var AContinue: boolean);

Called for each generated token. AContinue can be set to False to stop generation.

Notifications

TLlamaNotifyEvent = procedure(Sender: TObject) of object;
TLlamaNotifyEvent2 = procedure();

Used for signaling completion events.

Main Class: TLlama

The primary interface for interacting with Llama.cpp.

Constructor / Destructor

constructor Create;
destructor Destroy; override;

• Initializes and releases internal resources.

Methods

LoadModel

procedure LoadModel(const AModel: string);

Loads a model from the specified file path.

UnloadModel

procedure UnloadModel;

Releases the currently loaded model.

Tokenize

function Tokenize(const Text: string; AddSpecial: Boolean = true; ParseSpecial: Boolean = true): TIntegerArray;

Converts text into tokens.

• AddSpecial: Adds special tokens (e.g., BOS/EOS)
• ParseSpecial: Parses special tokens in input

Detokenize (Single Token)

function Detokenize(Token: integer; IncludeSpecial: Boolean = true): string;

Converts a single token back to text.

Detokenize (Multiple Tokens)

function Detokenize(Tokens: TIntegerArray; IncludeSpecial: Boolean = true): string;

Converts a token array back to text.

StartCompletation

function StartCompletation(const Prompt: string): string;

Generates text completion from a prompt.

Returns the generated text.

StartChatCompletation

procedure StartChatCompletation(var InfoChat: TInfoChat);

Performs chat-style completion using structured conversation data.

Properties

Summary

The uLlamaCpp unit provides:

• Fine-grained control over model and runtime parameters
• Tokenization and detokenization utilities
• Text and chat completion capabilities
• Callback-based streaming and logging

It is designed to offer a flexible and performant Object Pascal Notebook interface to Llama.cpp, suitable for both simple completions and advanced conversational AI integrations.