README

Delphi Anomaly Detection Library

Find the odd one out in your data: a fraudulent order, a failing sensor, a traffic spike, a typo that is really a 1000x price. In plain Delphi, with a few lines of code and no dependencies.

Most business applications already hold the data that would reveal a problem early. What is usually missing is the step that looks at every new value and says "this one is not like the others". That is what anomaly detection does, and it is much simpler to add than most people think.

What problems does it solve?

No statistics background needed: you feed the detector normal data, then ask it about new values.

It is small to use

Three lines to learn what is normal, one to ask about a new value:

uses
  AnomalyDetection.Types, AnomalyDetection.Base, AnomalyDetection.Factory;

var
  Detector: IStatisticalAnomalyDetector;
  R: TAnomalyResult;
begin
  Detector := TAnomalyDetectorFactory.CreateThreeSigma;
  Detector.AddValues([120, 135, 128, 142, 118, 131, 125, 139, 133, 127]);  // daily orders
  Detector.Build;

  R := Detector.Detect(980);
  if R.IsAnomaly then
    WriteLn(R.Description);
  // ANOMALY: Value 980,00 above upper limit (153,07), Z-score: 109,60
end;

The result tells you whether a value is anomalous and why: the limits, the distance from normal and a readable description you can show to a user or write to a log.

Live data: monitor a stream

Detector := TAnomalyDetectorFactory.CreateSlidingWindow(100);  // remembers the last 100 values
// ...
if Detector.IsAnomaly(NewReading) then
  RaiseAlert(NewReading);
Detector.AddValue(NewReading);  // let it keep learning

Memory stays constant, however long it runs.

Many fields at once: fraud and complex records

var
  Detector: IDensityAnomalyDetector;
begin
  Detector := TAnomalyDetectorFactory.CreateIsolationForest;
  for Tx in NormalTransactions do
    Detector.AddTrainingData([Tx.Amount, Tx.Hour, Tx.Weekday]);
  Detector.Train;

  if Detector.DetectMultiDimensional([5000, 3, 2]).IsAnomaly then
    FlagForReview;
end;

A 5000 purchase at 3 a.m. may be unusual only as a combination; each field alone looks fine. This is the case where simple thresholds fail and these algorithms shine.

The snippets above are compiled and executed against the current code. Output shown is the real output.

Algorithms at a glance

DBSCAN and LOF compare points with each other, so they are best suited to datasets of moderate size, not to millions of rows.

Also included:

• Confirmation system: require several similar anomalies before raising an alert, so one bad reading does not wake anybody up.
• Ready-made detectors: CreateForWebTrafficMonitoring, CreateForFinancialData, CreateForIoTSensors, CreateForHighDimensionalData...
• Evaluation and tuning: measure precision, recall and F1 on labeled data, with cross-validation and grid/random search to find the best parameters.
• Save and restore: persist a trained detector to a stream and reload it.
• Performance monitor: throughput and timing built in.
• Each detector protects its state with a critical section.

Try it

Installation

1. Clone or download this repository.
2. Add src, src/Core and src/Detectors to your Library Path.
3. Add the units you need to your uses clause.

Requires Delphi 10.3 Rio or later. No external dependencies. The repository includes a DUnitX test suite (90 tests) in Tests/.

The book

This README shows what the library does. How to choose the right algorithm for your data, how to prepare it, how to set thresholds, avoid false alarms and put detection into a real application is the subject of a book on anomaly detection in Delphi, currently in preparation.

Follow Daniele Teti for the announcement.

License

Open source under the Apache License 2.0: free for commercial and personal use.

Courses, consulting, custom development

The library is free. If you would rather get to a working result quickly, Bit Time Professionals offers:

• Courses and workshops on anomaly detection for Delphi teams
• Consulting on which approach fits your data, tuning and evaluation
• Custom development: integration into your application, dashboards, detectors tailored to your domain

Contact: professionals@bittime.it

Community

• Issues: GitHub Issues
• Community: Facebook Group