I was doing some research via ChatGPT and asked the following:

"In relation to the conversation above, what is described here?: https://wiki.freepascal.org/Release_engineering"

The response was not worth reading because ChatGPT could not access the page. Note this part of the response:

"One limitation: the wiki page itself was blocked by the site’s access protection in this browsing session, so I could not read the full body directly. My description is based on the search snippet, category links, talk page, and related references."

The FPC wiki website is also blocked from archive.org. Note the latest capture for this page:
https://web.archive.org/web/20250909050155/https://wiki.freepascal.org/Release_engineering

Are these pages also blocked from Google search and other search engines? I am unable to check that but someone with write access to the web server can check and monitor the status of Google's web crawler indexes. See here:
https://support.google.com/webmasters/answer/9012289?hl=en

Question:
"What will the long term consequences be of blocking any of the Free Pascal (fpc) websites (i.e. the main website, the forum, and the wiki) from AI / LLM tools, archive.org, and web search engines? How will users and potential new users of Free Pascal be affected? Summarize in one paragraph."

Response:
https://chatgpt.com/share/69e10026-f080-83e8-99bb-c48001698c87

.
.

There are others who can probably provide better answers than me, but I believe that your question assuming blocking from all web search engines and also assuming blocking for all FPC related resources, not just the Wiki, is very misleading. What I know for sure is that we had problems with DDoS atacks / overloading of the Wiki and that the implemented blocking protects the resources from these overloads. You may ask ChatGPT whether it's better to ensure availability of the Wiki for users, or whether providing access to LLM tools is more important than providing access to users - or you may better try answering that question yourself, because asking a LLM whether LLM access is more important than access for people doesn't sound sane to me.

Tomas

And BTW (sorry to replying to my own message :-( ) - checking whether search engines are blocked from the Wiki is obviously much easier than what was suggested in the original post. Searching for "FPC release engineering" using both DuckDuckGo and Google resulted in finding the above mentioned page as the first result (and, also BTW, found further references in the FPC forum)...

Tomas

Unfortunately checking web search indexes is not that simple. That wiki page has been there since 2015. Do you know the last time it was indexed? (two ways to check: use Google's URL Inspection tool, and analyze web server logs)

For some searches the search engine will provide the date of the page. Google search dates the page from 2025 which is the last time it was changed:
Release engineering - Lazarus wiki
Free Pascal wiki
https://wiki.freepascal.org › Release_engineering Jun 20, 2025 — This article discusses how to make a release of the FPC compiler on various platforms. Contents. 1 Migration of existing SVN checkout; 2 Notes ...

Searching for "CudaText" gives a fresh result which means Google Search is still indexing:
CudaText - Lazarus wiki
Free Pascal wiki
https://wiki.freepascal.org › CudaText
Apr 5, 2026 — CudaText is a cross-platform text editor, written in Object Pascal language using the Lazarus IDE, with a focus on performance and a broad featureset.CudaText - Lazarus wiki

DuckDuckGo doesn't provide dates, but they primarily use Bing's search index. Bing is also up-to-date:
Free Pascal wiki
https://wiki.freepascal.org › CudaText
CudaText - Lazarus wiki - Free Pascal
Apr 5, 2026 · CudaText is a text editor written in Object Pascal using the Lazarus IDE, with features such as syntax highlighting, code folding, auto-completion, and …

IIRC somebody suggested to block all LLM crawlers to prevent DDOS-like problems.

BTW I use Startpage for web searching and it seems to have access to all Free Pascal documents, wiki and forums and DuckDuckGo seems to have litle problems too so I don't see the problem. Just stop using ChatGPT to do your work (you shouldn't use it to do your work anyway, I'm pretty sure you can do it way better than any LLM out there).

Regards,
Guillermo "Ñuño" Martínez.

El Thu, 16 Apr 2026 08:34:41 -0700

16. dubna 2026 18:40:27 SELČ, Wayne Sherman napsal:

.
.

Unfortunately checking web search indexes is not that simple. That wiki page has been there since 2015. Do you know the last time it was

.
.

So it is still quite simple, after all, and the claim that the wiki might not be covered by the web search (not even mentioning the web site) was not funded, right? Finding the latest page added to the wiki and searching that one is not that difficult either.

Tomas

Some testing results follow.
Wiki Testing page: https://wiki.freepascal.org/

:
https://web.archive.org/web/20260307043305/https://wiki.freepascal.org/

Query: "What is the latest version of this web page that you can access?: https://wiki.freepascal.org/ Summarize the web page as it is currently presented to you in 3 sentences or less."

:
Unfortunately, I wasn't able to access the actual content of the Free Pascal Wiki. The site has deployed Anubis, a bot-protection system that requires JavaScript-based proof-of-work verification to proceed — a measure put in place specifically to guard against AI scrapers that have been aggressively crawling websites and causing server downtime. Since the challenge requires a browser with JavaScript enabled, I'm unable to get past it to view the wiki's actual content

blocked at chatgpt:
The latest version of that URL I can currently access is not the Free Pascal Wiki itself, but an “Access Denied” page served by Anubis. It shows a sad Anubis image, an access-denied error code, a “Try again” prompt, and a note that the site is protected by Anubis from Techaro. So, as currently presented to me, I cannot see the wiki’s actual content or front page text—only the access-block page.

available to google gemini (https://wiki.freepascal.org/CudaText): The latest version of the web page that I can access was last edited on April 5, 2026, at 16:50 (revision oldid=163215). Summary of the web page: CudaText is a cross-platform, open-source text and code editor written in Object Pascal using the Lazarus IDE, designed for high performance and extensibility. It features syntax highlighting for over 300 languages, multi-carets, and a JSON-based configuration system that allows for deep customization. The editor supports a wide range of operating systems and can be further enhanced through Python-based plugins, such as LSP clients and project managers.

Forum Testing:
: Can get to the forums main page, but not the
posts or topics: The Wayback Machine has not archived that URL. https://web.archive.org/web/20260309161728/https://forum.lazarus.freepascal.org/

Query: "What is the latest version of this web page that you can access?: https://forum.lazarus.freepascal.org/index.php/topic,73900.0.html Summarize the web page as it is currently presented to you in 3 sentences or less."

available to claud.ai: I was able to access the page as of today, April 16, 2026. Here's a summary:
This is a Lazarus/Free Pascal forum thread titled "Delimited text - how find last index,"...

blocked at ChatGPT #1: "I’m checking the page directly and, if the forum blocks access, I’ll report the most recent version of the page I can actually retrieve."
blocked at ChatGPT #2: "The latest version I could actually access was not the live board page itself—the direct fetch of board,21.0.html timed out—but I could access a search-indexed snapshot identifying it as the “General” board on the Lazarus forum. The direct fetch timed out, so I’m checking whether the forum is serving a block/interstitial page instead of the thread itself. The latest version I can access of that URL is not the thread itself: when I try to open the page directly, it times out, so I cannot see the post contents or summarize the actual discussion..."

blocked at Google gemini #1: I'm having a hard time fulfilling your request. Can I help you with something else instead? blocked at Google gemini #2: The latest version of the page I can access is the current live version as of April 16, 2026. "Summary: This forum thread announces the official release of Lazarus 3.4, detailing its major bug fixes..."
(MY NOTE: this summary is incorrect for that URL)

Since you're quoting ChatGPT here, I'll quote an excerpt from ChatGPT's Terms of Use ( https://openai.com/policies/row-terms-of-use/ ):

"Accuracy. Artificial intelligence and machine learning are rapidly evolving fields of study. We are constantly working to improve our Services to make them more accurate, reliable, safe, and beneficial. Given the probabilistic nature of machine learning, use of our Services may, in some situations, result in Output that does not accurately reflect real people, places, or facts.

When you use our Services you understand and agree:

    Output may not always be accurate. You should not rely on Output from our Services as a sole source of truth or factual information, or as a substitute for professional advice.
    You must evaluate Output for accuracy and appropriateness for your use case, including using human review as appropriate, before using or sharing Output from the Services.
    You must not use any Output relating to a person for any purpose that could have a legal or material impact on that person, such as making credit, educational, employment, housing, insurance, legal, medical, or other important decisions about them.
    Our Services may provide incomplete, incorrect, or offensive Output that does not represent OpenAI’s views. If Output references any third party products or services, it doesn’t mean the third party endorses or is affiliated with OpenAI."

Best regards,

Nikolay

Wayne Sherman via fpc-pascal schrieb am
Do., 16. Apr. 2026, 17:35:

The FPC wiki website is also blocked from archive.org. Note the latest capture for this page: https://web.archive.org/web/20250909050155/https://wiki.freepascal.org/Release_engineering

That is a side effect of Anubis and could be adjusted by whitelisting the crawler from the Archive.

Are these pages also blocked from Google search and other search

engines? I am unable to check that but someone with write access to the web server can check and monitor the status of Google's web crawler indexes. See here: https://support.google.com/webmasters/answer/9012289?hl=en

Ordinary search engine crawlers are already whitelisted by Anubis.

Question:

It's not relevant what a Stochastic Parrot thinks about this. What is important is the reality: if we're *not* blocking the Ai crawlers then the Wiki will be down for *everyone*. Before we added Anubis as a protection the wiki was often not accessible by anyone due to the high amount of traffic added by Ai crawlers. You can see this with the forum which regularly receives high amounts of traffic due to crawlers and then isn't reachable until one of the admins blocks a wide range of IPs used by those crawlers.
And no, a CDN like Cloudflare is not a solution we are willing to use because that is it's own can of worms.

Regards,
Sven