Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2025-10

LLM / MCP

19 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Michael Van Canneyt

Hi,

For those that are busy with LLM and MCP:

I've created 2 repos:

https://gitlab.com/freepascal.org/llm

This contains client-side classes that allow you to use the APIs of the major LLM vendors (OpenAI, Anthropic, Google Gemini and OLLama, JanAI). There is an example for each API.

The intent is to use this to allow using any LLM in the lazarus IDE.

For people that want to go a step further:

The following repo contains an implementation of the MCP (Model Context Protocol):

https://gitlab.com/freepascal.org/mcp

The MCP protocol allows you to write a tool that is usable by most AI agents today. What you can do with the classes is to write an MCP server which is then used by the agent (and allows the LLM to execute commands).

As an example, there is a MCP server that allows an Agent to execute SQL queries on any FPC-supported database. It allows one to ask to retrieve information from the database in natural language. The agent will then transform the natural language to SQL and retrieve the result. I've had surpringly good results using Google Gemini.

The tool by default does not allow to modify the data or structure of the database. Since retrieved data is sent to a LLM, be careful not to send any personal data over the line for privacy reasons.

Articles describing how to use this will follow.

Michael.

Reply by email

Michael Van Canneyt
to Michael Van Canneyt
Michael Van Canneyt wrote 7 lines
On Mon, 6 Oct 2025, Jean SUZINEAU via lazarus wrote: > Very interesting ! It will get even better soon :-)

The lazllms package now installs a TLLMClient on the component palette, allowing you to create a chat with an LLM using just point-and-click in true RAD fashion.

There is a GUI demo to show how this works, so even a beginner should be able to work with this.

Michael.

Reply by email

Jean SUZINEAU
to Michael Van Canneyt

I'm trying to add a minimal support for Perplexity. I got an error in LLM.Client because it references an fpasyncwebclient unit. I couldn't find the source for this unit (in branch main on gitlab), but it compiled with just creating an empty place holder unit fpasyncwebclient.

Reply by email

Michael Van Canneyt
to Jean SUZINEAU
Jean SUZINEAU wrote
I'm trying to add a minimal support for Perplexity.

Looking forward to that.

Tip:
You can ask claude code or Gemini CLI to rework one of the existing ones to work with the perplexity API instead. Both do a very good job when put to such tasks.

I got an error in LLM.Client because it references an fpasyncwebclient unit. I couldn't find the source for this unit (in branch main on gitlab), but it compiled with just creating an empty place holder unit fpasyncwebclient.

Thanks. I corrected that.

I modified the Lazarus AI Assist functionality to use this new functionality. I added a copy of the files in the LLM directory to the lazarus aissist package. I updated that copy as well.

Michael.

Reply by email

Jean SUZINEAU
to Michael Van Canneyt

Le 09/10/2025 ? 21:15, Michael Van Canneyt via lazarus a ?crit?:

Show quoted text 10 lines
On Thu, 9 Oct 2025, Jean SUZINEAU via lazarus wrote: > I'm trying to add a minimal support for Perplexity. Looking forward to that. Tip: You can ask claude code or Gemini CLI to rework one of the existing ones to work with the perplexity API instead. Both do a very good job when put to such tasks.

My fork is at
https://gitlab.com/jsuzineau/freepascal_llm/-/tree/Perplexity?ref_type=heads perplexity_demo.lpr is working, I put a sample of result at

https://gitlab.com/jsuzineau/freepascal_llm/-/blob/Perplexity/Demo/Perplexity/example_of_result_output.txt?ref_type=heads

and the raw json returned at

https://gitlab.com/jsuzineau/freepascal_llm/-/blob/Perplexity/Demo/Perplexity/example_of_result_json.txt?ref_type=heads

Tell? me if you want that I make a pull request.
For now I just have a Perplexity Pro account, I don't know if it's possible to get something with a free Claude / Gemini account ? For now I haven't had time to test your new code with the TLLMClient component.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20251010/75d07d2f/attachment.htm>

Reply by email

Michael Van Canneyt
to Jean SUZINEAU
Jean SUZINEAU wrote 23 lines
Le 09/10/2025 ? 21:15, Michael Van Canneyt via lazarus a ?crit?: > On Thu, 9 Oct 2025, Jean SUZINEAU via lazarus wrote: > >> I'm trying to add a minimal support for Perplexity. > > Looking forward to that. > > Tip: > You can ask claude code or Gemini CLI to rework one of the existing ones to > work with the perplexity API instead. Both do a very good job when put to > such tasks. My fork is at https://gitlab.com/jsuzineau/freepascal_llm/-/tree/Perplexity?ref_type=heads perplexity_demo.lpr is working, I put a sample of result at https://gitlab.com/jsuzineau/freepascal_llm/-/blob/Perplexity/Demo/Perplexity/example_of_result_output.txt?ref_type=heads and the raw json returned at https://gitlab.com/jsuzineau/freepascal_llm/-/blob/Perplexity/Demo/Perplexity/example_of_result_json.txt?ref_type=heads Tell? me if you want that I make a pull request.

Yes, please.

For now I just have a Perplexity Pro account, I don't know if it's possible to get something with a free Claude / Gemini account ?

I don't think so.

For ChatGPT you can set up billing. For Gemini, I also had to use my google account to set up billing.

For now I haven't had time to test your new code with the TLLMClient component.

No hurry...

Michael.

Reply by email

Kostas Michalopoulos
to Michael Van Canneyt: For those that are busy with LLM and MCP: I've created 2 repos:…
Michael Van Canneyt wrote
This contains client-side classes that allow you to use the APIs of the major LLM vendors (OpenAI, Anthropic, Google Gemini and OLLama, JanAI). There is an example for each API.

I did a quick check in the code (via the web interface). Am i wrong or is this only for online/web-based APIs and require using each one separately manually?

Considering that pretty much all of those services provide very similar APIs (mainly inspired by OpenAI's AFAICT), i think it'd be useful to have a components that exposes a common API with other components that can be plugged to it to act as connectors/communicators. Also to not assume any web-based or internet connectivity as there is also the option of using llama.cpp or other inference engines.

Personally i've written some llama.cpp based utilities that i use on my PC, everything is local and uses gguf files directly, not a single socket is touched :-P. I've done this by writing a C++ command-line utility that i call from a Tcl/Tk script, but being able to drag-drop something from Lazarus' component palette into a form would be easier.

Is this under the scope of what you're working on or are you focusing on making online service-specific components?

Kostas

Reply by email

Michael Van Canneyt
to Kostas Michalopoulos
Kostas Michalopoulos wrote 8 lines
On 10/4/25 4:36 PM, Michael Van Canneyt via lazarus wrote: > This contains client-side classes that allow you to use the APIs of the > major LLM vendors (OpenAI, Anthropic, Google Gemini and OLLama, JanAI). > There is an example for each API. I did a quick check in the code (via the web interface). Am i wrong or is this only for online/web-based APIs and require using each one separately manually?

It is currently only for online/web based APIS. But see below.

Show quoted text 7 lines
Considering that pretty much all of those services provide very similar APIs (mainly inspired by OpenAI's AFAICT), i think it'd be useful to have a components that exposes a common API with other components that can be plugged to it to act as connectors/communicators. Also to not assume any web-based or internet connectivity as there is also the option of using llama.cpp or other inference engines.

That is already done:

TLLMClient is an abstraction that under the hood uses the correct low-level API (the various "protocol" classes). Please re-check, I committed it only yesterday.

You can put it on a form, and get started right away. There is a demo for that.

Show quoted text 5 lines
Personally i've written some llama.cpp based utilities that i use on my PC, everything is local and uses gguf files directly, not a single socket is touched :-P. I've done this by writing a C++ command-line utility that i call from a Tcl/Tk script, but being able to drag-drop something from Lazarus' component palette into a form would be easier.

Actually, Benjamin Rosseaux has a TPAsLLM component which does the inference directly in Pascal.

Is this under the scope of what you're working on or are you focusing on making online service-specific components?

If we can make a 'protocol' class for TLLMClient that uses the llama.cpp locally, then I would be very happy too. :).

I intended to make one for the TPasLLM class, but a llama.cpp based one sounds also very good, then we can make use of all improvements in llama.cpp.

If you can help with that, that would be really good :-)

Michael.

Reply by email

Kostas Michalopoulos
to Michael Van Canneyt
Michael Van Canneyt wrote 8 lines
That is already done: TLLMClient is an abstraction that under the hood uses the correct low-level API (the various "protocol" classes). Please re-check, I committed it only yesterday. You can put it on a form, and get started right away. There is a demo for that.

Ah i see, i only looked at the code a couple of days ago when (IIRC) there wasn't such a thing. Overall it looks fine (if a bit high level, there might be a need for tokenization for local stuff - if for no other reason than for progress feedback as it can be slow, but that stuff could be added later).

The only concern i have is that protocols are registered globally and are identified with strings. I think a better and more type-safe approach would be to have TLLMProtocol a TComponent subclass and have the Protocol property of TCustomLLMClient use that. This way packages can install new custom protocol components and protocols can be dropped in the form/module as needed from the Lazarus component palette.

This will also allow for protocol-specific settings, e.g. for llama.cpp there will be need to specify path to gguf files, context size, sampling settings (such as temperature, top-p and top-k), etc that can be edited/saved via Lazarus' object inspector. Same for protocol-specific events, like tokenization and generation progress.

If we can make a 'protocol' class for TLLMClient that uses the llama.cpp locally, then I would be very happy too. :).

It shouldn't be too hard, i might look into it.

Kostas

Reply by email

Michael Van Canneyt
to Kostas Michalopoulos
Kostas Michalopoulos wrote 21 lines
On 10/10/25 12:04 PM, Michael Van Canneyt wrote: > That is already done: > > TLLMClient is an abstraction that under the hood uses the correct low-level > API (the various "protocol" classes). Please re-check, I committed it only > yesterday. > > You can put it on a form, and get started right away. There is a demo for > that. Ah i see, i only looked at the code a couple of days ago when (IIRC) there wasn't such a thing. Overall it looks fine (if a bit high level, there might be a need for tokenization for local stuff - if for no other reason than for progress feedback as it can be slow, but that stuff could be added later). The only concern i have is that protocols are registered globally and are identified with strings. I think a better and more type-safe approach would be to have TLLMProtocol a TComponent subclass and have the Protocol property of TCustomLLMClient use that. This way packages can install new custom protocol components and protocols can be dropped in the form/module as needed from the Lazarus component palette.

Ideally the user needs only 1 component, which is why I chose the current approach.

But having more detailed configuration can be achieved while maintaining the single-component approach. Simply by having a TModelConfiguration sub-component in TLLMClient, which is created automatically using a correct class when you set the protocol.

Then it will visually be part of TLLMClient in the object inspector, and can still contain any number of additional properties (and events) to fit a particular model.

I'll put it on my todo list.

Michael.

Reply by email

Kostas Michalopoulos
to Michael Van Canneyt
Michael Van Canneyt wrote
Ideally the user needs only 1 component, which is why I chose the current approach.

Why is this desirable though?

Lazarus already has components that you use with other components to provide flexibility, such as the database components, so having this separation between the "front end" and "back ends" as different components is already an established practice - and does take advantage of using Lazarus' visual tools.

And if anything, TLLMClient and TLLMProtocol feel a lot like TDataSource and the various TDataSet descendants.

The current approach is also quite fragile and has hidden and unnecessary complications. For example if you install the package in Lazarus, the package's uses clause will include all the individual LLM protocol units which will auto-register themselves and then they will appear in TLLMProtocol's settings drop-down menu. However if you select one without including it manually in the code, starting your application will throw an exception about an unknown protocol.

And considering that you *do* need to manually include the protocol units anyway, this doesn't really save the user anything by only requiring them to drop a single component in the form - you are replacing the need to drop a second component by having them manually go and include the llm protocol unit (something that Lazarus would do automatically if you dropped the protocol as a component in the form).

Show quoted text 10 lines
But having more detailed configuration can be achieved while maintaining the single-component approach. Simply by having a TModelConfiguration sub- component in TLLMClient, which is created automatically using a correct class when you set the protocol. Then it will visually be part of TLLMClient in the object inspector, and can still contain any number of additional properties (and events) to fit a particular model.

Perhaps this particular will work but TBH i think the whole global registration thing is more limiting, fragile and more complicated than it needs to be (there is a lot of code in there that only exists just to handle that global list), considering Lazarus already provides means for doing exactly that sort of thing (with established use).

As a sidenote, i'd suggest placing the LLM and MCP components under the same component category (something like "AI" or "ML" or "Machine Learning" or something like that), as it is annoying to have a single tab in Lazarus take all the space for itself just for 1-2 icons inside.

Also you forgot an absolute path in MCP:
https://gitlab.com/freepascal.org/mcp/-/blob/main/Src/IDE/mcpcontrolreg.pas?ref_type=heads#L268

Kostas

Reply by email

Michael Van Canneyt
to Kostas Michalopoulos
Kostas Michalopoulos wrote 5 lines
On 10/10/25 9:50 PM, Michael Van Canneyt via lazarus wrote: > Ideally the user needs only 1 component, which is why I chose the current > approach. Why is this desirable though?

Because IMO it makes it more easy to use.

Especially in code:
In your approach the developer needs to create 2 classes. In my approach he needs to create only 1 class and set one or more properties.

Show quoted text 8 lines
Lazarus already has components that you use with other components to provide flexibility, such as the database components, so having this separation between the "front end" and "back ends" as different components is already an established practice - and does take advantage of using Lazarus' visual tools. And if anything, TLLMClient and TLLMProtocol feel a lot like TDataSource and the various TDataSet descendants.

I felt it is more like a database connection and drivers for a particular database.

I exactly wanted to get away from the various TSQLConnection descendants we have now and move more to a TSQLConnector kind of approach.

Show quoted text 7 lines
The current approach is also quite fragile and has hidden and unnecessary complications. For example if you install the package in Lazarus, the package's uses clause will include all the individual LLM protocol units which will auto-register themselves and then they will appear in TLLMProtocol's settings drop-down menu. However if you select one without including it manually in the code, starting your application will throw an exception about an unknown protocol.

See below.

Show quoted text 7 lines
And considering that you *do* need to manually include the protocol units anyway, this doesn't really save the user anything by only requiring them to drop a single component in the form - you are replacing the need to drop a second component by having them manually go and include the llm protocol unit (something that Lazarus would do automatically if you dropped the protocol as a component in the form).

No worries there, I am going to have this added automatically when you select the protocol.

Show quoted text 16 lines
> But having more detailed configuration can be achieved while maintaining > the > single-component approach. Simply by having a TModelConfiguration sub- > component in TLLMClient, which is created automatically using a correct > class when you set the protocol. > > Then it will visually be part of TLLMClient in the object inspector, > and can still contain any number of additional properties (and events) to > fit > a particular model. Perhaps this particular will work but TBH i think the whole global registration thing is more limiting, fragile and more complicated than it needs to be (there is a lot of code in there that only exists just to handle that global list), considering Lazarus already provides means for doing exactly that sort of thing (with established use).

This code is under the hood. Only the component developer sees it, and he only needs to add one line to register his 'protocol'. Not exactly the end of the world in terms of code.

The end user (the main target of this all) only sees one component and some properties he can set.

But I'll think about your suggestion.

As a sidenote, i'd suggest placing the LLM and MCP components under the same component category (something like "AI" or "ML" or "Machine Learning" or something like that), as it is annoying to have a single tab in Lazarus take all the space for itself just for 1-2 icons inside.

Good suggestion, done.

Thanks for pointing it out, fixed.

Michael.

Reply by email

Michael Van Canneyt
to Michael Van Canneyt
Show quoted text 9 lines
> And considering that you *do* need to manually include the protocol units > anyway, this doesn't really save the user anything by only requiring them > to drop a single component in the form - you are replacing the need to drop > a second component by having them manually go and include the llm protocol > unit (something that Lazarus would do automatically if you dropped the > protocol as a component in the form). No worries there, I am going to have this added automatically when you select the protocol.

It's implemented. Setting protocol will add the correct unit to the uses clause. Requires at least today's Lazarus if you use the LLM repo. (in laz repo it is of course automatically correct)

Jean Suzineau added support for Perplexity (for which my thanks!), so now we have already 5 supported engines.

Michael.

Reply by email

n7800
to Michael Van Canneyt: For those that are busy with LLM and MCP: I've created 2 repos:…

LLM/MCP outside my area of ??expertise, but I have a question: should they be in the root GitLab?group "freepascal.org" or should they be placed in the subgroup "freepascal.org/lazarus" (next to CCR)? I see there are LPK packages for Lazarus there.

By the way, regarding packages, should they be added to OPM? ? ?

Show quoted text 47 lines
???????, 4 ??????? 2025, 18:37 +05:00 ?? Michael Van Canneyt via lazarus < >: ? Hi, For those that are busy with LLM and MCP: I've created 2 repos: https://gitlab.com/freepascal.org/llm This contains client-side classes that allow you to use the APIs of the major LLM vendors (OpenAI, Anthropic, Google Gemini and OLLama, JanAI). There is an example for each API. The intent is to use this to allow using any LLM in the lazarus IDE. For people that want to go a step further: The following repo contains an implementation of the MCP (Model Context Protocol): https://gitlab.com/freepascal.org/mcp The MCP protocol allows you to write a tool that is usable by most AI agents today. What you can do with the classes is to write an MCP server which is then used by the agent (and allows the LLM to execute commands). As an example, there is a MCP server that allows an Agent to execute SQL queries on any FPC-supported database. It allows one to ask to retrieve information from the database in natural language. The agent will then transform the natural language to SQL and retrieve the result. I've had surpringly good results using Google Gemini. The tool by default does not allow to modify the data or structure of the database. Since retrieved data is sent to a LLM, be careful not to send any personal data over the line for privacy reasons. Articles describing how to use this will follow. Michael. -- _______________________________________________ lazarus mailing list https://lists.lazarus-ide.org/listinfo/lazarus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20251103/e2aa4fae/attachment.htm>

Reply by email

n7800
to Michael Van Canneyt: For those that are busy with LLM and MCP: I've created 2 repos:…

LLM/MCP outside my area of expertise, but I have a question: should they be in the root GitLab group "freepascal.org" or should they be placed in the subgroup "freepascal.org/lazarus" (next to CCR)? I see there are LPK packages for Lazarus there. ?
By the way, regarding packages, should they be added to OPM?

Show quoted text 47 lines
???????, 4 ??????? 2025, 18:37 +05:00 ?? Michael Van Canneyt via lazarus < >: ? Hi, For those that are busy with LLM and MCP: I've created 2 repos: https://gitlab.com/freepascal.org/llm This contains client-side classes that allow you to use the APIs of the major LLM vendors (OpenAI, Anthropic, Google Gemini and OLLama, JanAI). There is an example for each API. The intent is to use this to allow using any LLM in the lazarus IDE. For people that want to go a step further: The following repo contains an implementation of the MCP (Model Context Protocol): https://gitlab.com/freepascal.org/mcp The MCP protocol allows you to write a tool that is usable by most AI agents today. What you can do with the classes is to write an MCP server which is then used by the agent (and allows the LLM to execute commands). As an example, there is a MCP server that allows an Agent to execute SQL queries on any FPC-supported database. It allows one to ask to retrieve information from the database in natural language. The agent will then transform the natural language to SQL and retrieve the result. I've had surpringly good results using Google Gemini. The tool by default does not allow to modify the data or structure of the database. Since retrieved data is sent to a LLM, be careful not to send any personal data over the line for privacy reasons. Articles describing how to use this will follow. Michael. -- _______________________________________________ lazarus mailing list https://lists.lazarus-ide.org/listinfo/lazarus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20251109/72e50000/attachment-0001.htm>

Reply by email

n7800
to Michael Van Canneyt: It will get even better soon :-) Michael.

LLM/MCP outside my area of expertise, but I have a question: should they be in the root GitLab group "freepascal.org" or should they be placed in the subgroup "freepascal.org/lazarus" (next to CCR)? I see there are LPK packages for Lazarus there. ?
By the way, regarding packages, should they be added to OPM? ? ?

Show quoted text 16 lines
???????, 7 ??????? 2025, 12:04 +05:00 ?? Michael Van Canneyt via lazarus < >: ? On Mon, 6 Oct 2025, Jean SUZINEAU via lazarus wrote: > Very interesting ! It will get even better soon :-) Michael. -- _______________________________________________ lazarus mailing list https://lists.lazarus-ide.org/listinfo/lazarus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20251109/3f04800e/attachment.htm>

Reply by email

Michael Van Canneyt
to n7800: LLM/MCP outside my area of expertise, but I have a question: should they be in the root…
Show quoted text 5 lines
LLM/MCP outside my area of expertise, but I have a question: should they be in the root GitLab group "freepascal.org" or should they be placed in the subgroup "freepascal.org/lazarus" (next to CCR)? I see there are LPK packages for Lazarus there.

No, they should not be inside lazarus. The base units can perfectly be used without lazarus. But the lazarus integration of course requires lazarus.

? By the way, regarding packages, should they be added to OPM?

I would wait with that, because the lazarus integration needs lazarus trunk/main. Mattias created a small extension to the IDE interfaces that is used in the integration package.

Michael.

Reply by email