Block, then match.

Comparing every pair of records is quadratic and infeasible past modest sizes. Blocking narrows two tables to a small set of candidate pairs. Sparkly does this with top-k TF/IDF similarity; Delex combines multiple blocking strategies. What blocking drops, matching can never recover, so blocking is tuned for recall.

A learned matcher needs labeled examples. If you have none, MatchFlow’s active learning selects a small, informative sample from the candidate set (typically about 600 pairs) for you to label using a CLI or web labeler. You can also bring your own labels.

MatchFlow turns labeled pairs into feature vectors, trains a classifier (from scikit-learn or PySpark MLlib), and applies it to the full blocking output to predict match or no-match. It runs on pandas for small data or Spark for large.

Batch resolves a whole table as a job. When records arrive one at a time, MadMatcher-Pro publishes the finished build (index, features, trained matcher) as a serving bundle and matches each incoming record in milliseconds, in-process, with no Spark job per request. The same blocker and matcher answer both ways, so a served match is identical to the batch one. Refreshing the data is a republish and a zero-downtime hot-swap.