README

MakerAI ChatTools Framework (Beta) — Extending AI Capabilities in Delphi

🌐 Official Website: https://makerai.cimamaker.com

MakerAI ChatTools is the official capability-extension framework for MakerAI.

Built natively in Delphi, ChatTools allows Artificial Intelligence models to see, hear, read, speak, and act by attaching real-world skills such as PDF processing, OCR, Vision, Speech, Web Search, and Code Interpretation, fully integrated into the MakerAI intelligent orchestration pipeline.

🧠 1. Philosophy: Separated Intelligence

MakerAI adopts a clear architectural principle: separate intelligence from skills.

• Intelligence (LLMs)
  The brain — models such as GPT-5.x, Gemini 3.x, Claude 4.x, or Llama-based local models. They excel at reasoning and language understanding but are often blind or deaf to physical files and external systems.
• Skills (ChatTools)
  The senses and muscles — deterministic tools that interact with the real world. Examples include Ghostscript, DeepSeek-OCR, speech engines, vision models, and web connectors.

The problem we solve:

• No spaghetti code
• No hard dependency on expensive multimodal models
• No vendor lock-in

If your AI model only understands text, MakerAI can lend it eyes and ears through ChatTools — transparently and predictably.

🧠 2. Core Orchestration Concepts

The Universal Envelope (TAiMediaFile)

Raw files are never passed directly to models. Instead, MakerAI wraps them in a unified container:

• Content: binary stream of the file
• Category: automatic detection (PDF, Image, Audio)
• Transcription: extracted or generated text
• Processed State: prevents duplicate processing

This abstraction allows tools and models to cooperate safely and efficiently.

The Decision Tripod

Every orchestration decision is based on three capability vectors:

• NativeInputFiles — what the provider API accepts
• ChatMediaSupports — what the model truly understands
• EnabledFeatures — which ChatTools are active

This makes orchestration deterministic, explainable, and extensible.

🛠 3. Four Usage Scenarios

A. Standalone Tool Usage

Ideal for batch processing and automation pipelines where no conversational AI is required.

Text := TAiOllamaPdfTool.ExtractTextFromFile('C:\\docs\\invoice.pdf');

B. Automatic Orchestration (Bridge Mode)

For blind or lightweight local models. MakerAI intercepts the file, runs the local tool, and injects the extracted text into the conversation.

AiOllamaChat1.PdfTool := AiOllamaPdfTool1;
AiOllamaChat1.ChatMediaSupports := [];
AiOllamaChat1.EnabledFeatures := [Tcm_Pdf];
AiOllamaChat1.Run(AskMsg);

C. Function Calling (AI as Director)

The AI decides when to invoke a tool. Requires models with use_tools support.

procedure TForm1.OnAiFunctionsAction(Sender: TObject; ToolCall: TAiToolsFunction; ...);
begin
  if ToolCall.AskMsg.MediaFiles.Count > 0 then
    ToolCall.Response := AiOllamaPdfTool1.ExtractText(
      ToolCall.AskMsg.MediaFiles[0].Content,
      'temp.pdf'
    );
  Handled := True;
end;

D. Native Multimodal Processing

For cloud models that natively support file input (e.g., GPT-4o, Gemini).

AiOllamaChat1.ChatMediaSupports := [Tcm_Pdf];
AiOllamaChat1.NativeInputFiles := [Tfc_Pdf];

📊 4. Strategy Comparison

🚀 5. Creating a New ChatTool

To implement a new tool (e.g., Text-to-Speech), inherit from the appropriate base class and follow the reporting lifecycle.

type
  TAiMySpeechTool = class(TAiSpeechToolBase)
  protected
    procedure ExecuteSpeechGeneration(const AText: string; ResMsg, AskMsg: TAiChatMessage); override;
  end;

Lifecycle Rules

1. Run asynchronously if required (TTask.Run)
2. Report progress (ReportState)
3. Attach generated media (ResMsg.AddMediaFile)
4. Signal completion (ReportDataEnd)

🔍 6. Deep Dive: PDF + OCR Lessons Learned

Key rules discovered while implementing the Ghostscript + DeepSeek-OCR pipeline:

• Vision models require a newline separator before prompts
• Prompts must be lowercase
• Ghostscript ≥10.x requires -dNOSAFER
• Use <|ocr|> prefixes to force pure OCR output

📋 7. Requirements & Interfaces

Technical Requirements

• MakerAI v3.2+ (Beta)
• Ghostscript installed
• Ollama with vision-capable models

Available Interfaces

• IAiPdfTool
• IAiSpeechTool
• IAiVisionTool
• IAiWebSearchTool
• IAiCodeInterpreter

Developed with ❤️ for the Delphi community.

MakerAI ChatTools — Bringing real-world senses to native AI.