Picture two vendor demos in the same week. Both screen a handful of sample names, both return tidy hits filed by crime type, and both dashboards look calm. Neither demo touched one of your customers. A namesake problem in your book, a story that ran only in a regional paper, or an article that has since been taken down will not show up on names the vendor picked.
So the evaluation has to move onto your names. That means a test set assembled before the trial, the same names run through every candidate, counts anyone can recheck, and a file that records why each tool passed or failed. The method below follows the evaluation list the Wolfsberg Group wrote for adverse media screening, which it calls negative news screening.
What should an adverse media screening evaluation prove?
Once the shortlist of adverse media screening software is down to two or three tools, an evaluation answers two questions. Does the tool find the stories that matter on the customers you actually have? And can every hit, kept or dismissed, be explained to an auditor a year later? A tool that passes the first and fails the second still leaves the firm exposed, because the reason recorded on each hit is what a reviewer reads.
The Wolfsberg Negative News Screening FAQs put the recommendation plainly: firms should "conduct appropriate system evaluation to ensure that its screening solution is 'fit-for-purpose'." The same FAQs tie the depth of screening to the customer. A retail customer "will likely have a lower public profile", while wealthy individuals, PEPs and large corporates are more likely to have a high public profile. A private banking book needs a broader test than a mass market one.
How do you test each of the nine Wolfsberg checks?
Table 1. The nine Wolfsberg evaluation checks as trial tests (Negative News Screening FAQs, 2022, question 17)
| Check | Wolfsberg's wording | Test in the trial | Evidence to keep |
|---|---|---|---|
| Coverage | "sufficient media coverage in markets where the FI operates or where the customer base is located" | Screen names whose stories ran only in the local press of your main markets | Hits found and missed, per market |
| Data | "completeness, integrity, and timeliness", including "levels of alert duplication" | Count how often one story arrives as several hits; note how many days pass between a story's publication and its hit | Duplicate count; story date beside hit date |
| Matching | "effectiveness of name matching, and false positive/alerts hit rate" | Run the namesake and spelling variant groups of the test set | Relevant hits, namesakes and misses per tool |
| Archive | "accessibility of media articles which are archived or no longer available in open sources" | Include a story that has since been removed from the web | Whether the tool still shows the article text |
| Translation | "capability to provide news/events translated into the language of the FI's operation" | Include stories that ran only in a language your analysts do not read | Whether a reviewer can judge the hit from the translation, and whether the original is kept |
| Scalability | "ability to define search parameters, search depth and result filtering" | Change one setting, such as the lookback period, and rerun the same names | Hit counts before and after the change |
| Reporting | "availability of relevant dashboards and performance metrics" | Ask for screening volume, alert volume, false positives and true matches for the trial period | The exported report |
| Traceability | "the recording and capability to review all internal screening events, configurations and other logic applied to generate a match" | Pick five closed hits and ask the tool to show who searched, which settings applied and why each matched | Screens or exports for the five |
| Data sharing | "consideration of local data-sharing requirements when utilising external/shared solutions" | Ask where names and results are stored and processed, and who at the vendor can see them | The vendor's written answer, checked by your data protection lead |
The Wolfsberg text adds that "Not all the above aspects can be assessed easily" and gives one example: internet based screening "is limited and may hinder assessment of these criteria." In a trial, reporting, data sharing and part of the archive question depend on what the vendor states rather than on what a screen shows, so ask for those answers in writing and file them with the counts. The other six can be measured on your own names, which is why the test set decides more than the demo does.
How do you build a test set for an adverse media trial?
A test set is a list of names with a known answer. Before the trial, write down for each name whether a relevant story exists and where it ran. Six groups cover most of what a trial has to show:
- Confirmed true matches. Customers with a story your team already judged relevant, including relationships the firm exited. Wolfsberg counts "relationships reviewed and exited" among the figures management should see, so those files already hold the answer.
- Namesakes. Common names in your book that match someone in the news, with the date of birth or incorporation date you hold.
- Spelling and script variants. One person written several ways. Wolfsberg's own example is a single name transcribed as Piotr Czajkowski, Pyotr Tchaikovsky and Piotr Chaykovskiy.
- Local language stories. Names whose only coverage ran in a regional outlet, in the customer's language.
- Removed articles. A story you saved that is no longer online, to test the archive.
- Clean names. Retail customers with no story at all, so the trial measures noise as well as finds.
Put companies and their owners in the set as well as individuals: Wolfsberg's scope question covers "customers, UBOs, related parties or non-customers". To confirm that a story exists before a name goes in, run it through a free adverse media check and save the article links next to the expected answer. Keep the set confidential, and give every vendor the same version.
How do you measure false positives without hiding true matches?
Start with a count a vendor has published. complead's adverse media screening and monitoring page sums up one case in a line, "A payments firm caught a corruption story its old tool never saw". That firm's result held 2 relevant hits of 40 and 0 namesakes. In complead, namesakes are separated using the identifiers you already hold. A count like this only means something once it sits beside the misses on the same names.
Picture a trial on 300 names with 20 confirmed true matches among them. Tool A returns 900 hits and finds 19 of the 20. Tool B returns 250 hits and finds 14. Tool B looks quieter. Then the misses are counted. Fewer hits only help when the misses do not rise with them.
Count five things per tool, for each group of the test set:
- hits per name screened;
- relevant hits, the ones a reviewer would keep;
- misses, the confirmed stories the tool did not return;
- namesakes and immaterial hits, counted apart;
- duplicates, one story counted more than once.
The split between namesakes and immaterial hits follows Wolfsberg's two grounds for discounting. A hit is a false positive when "key secondary identifier information" differs, such as date of birth, date of incorporation or gender. It is immaterial when the customer is the right person but "not the subject", for example a plaintiff, or when the source is not credible. A tool that cuts one kind does nothing for the other, and false positive and false negative rates mean something only when both are counted on the same names.
Misses are the count demos leave out. Wolfsberg's quality assurance list asks firms to assess "the precision of screening and its hit rate" and to test "the false positive alert population to ensure there are no missed true matches". In a trial, that means a reviewer reads a sample of what each tool set aside, not only what it raised. Ask every vendor on the shortlist for relevant hits, namesakes and misses on your test set.
What should a vendor show you about settings and evidence?
Settings decide what reaches the analyst, so a trial has to show who controls them. Wolfsberg asks firms to understand "the evaluation of reliability performed by the vendor" on its sources, and to document every alert management solution with "the rationale for implementation". If a tool discounts hits automatically, the trial file needs the rule and its reason, written down before go live.
Ask each vendor to change the settings live, on your test set: the offense categories in scope, the lowest source tier that counts, how far back a story still matters, and how an allegation weighs against a conviction. Wolfsberg notes that "a multi-year look-back threshold may be appropriate". It can be one period for everything or a different one per offense type. Then ask what happens to a hit the settings exclude. If it disappears, the firm cannot show later why it never reached a reviewer.
Evidence comes next. Each kept or dismissed hit needs the record an analyst keeps after a manual adverse media check: source, date, language, the passage that matched and the reviewer's reason. Ask for that record as an export. A description is not enough.
Those four choices, categories, source tier, lookback and allegation weight, are "settings on your side" in complead. Social platforms and forums are "scored as their own tier". Separate policies can run per segment or jurisdiction, and the effect of a change on open alerts is visible before it applies. Sources come in three layers: formal records such as court filings, enforcement notices, regulatory actions and disqualification registers; media reporting; and the social and open web. What does not meet the policy is not deleted; it is set aside and fully auditable.
In complead, every screening search is recorded with who searched, what was searched, when and the result, including searches that return no match. complead's adverse media data covers 30+ languages across 190+ countries, from more than 300,000 articles a day. Each hit goes "to a risk category, for example, fraud, bribery, financial crime, or organized crime, rather than the article's tone". The original text is kept alongside the translation. One screening can search a name in up to three of those languages at once.
How should a side by side trial run?
Picture a payments firm with two finalists. In week one, both tools screen the full test set with the settings the firm intends to use. In weeks two to four, both screen every new customer and keep monitoring the test set, so a story published during the trial shows whether and when each tool raises it. The same two reviewers work both queues and record their decisions in one template.
Three conditions keep the comparison fair. Use the channel you will use in production, API or batch, because results and speed can differ by channel. Keep settings fixed after week one, or rerun the whole set after any change. And test monitoring, not only onboarding: Wolfsberg allows that after the first review "it may only be necessary to screen against new media events", which is the change ongoing monitoring in AML has to catch.
complead runs beside the current tool with no integration fees, and the API typically integrates in hours; most teams screen within days. Checks run via API, batch files or the web. A firm that already licenses LexisNexis, Dow Jones, LSEG World-Check or Moody's can connect that provider with its own credentials and screen against complead data, the provider's data or both, with matches from all sources deduplicated into a single profile. The provider's hits and those from complead data then end up in the same queue. During monitoring, a new story that meets the policy raises an alert and contributes to the customer risk score.
What changed in 2026?
- 23 June 2026. The Wolfsberg Group published updated Guidance on the Risk-Based Approach, built on the principle that "a focus on everything is a focus on nothing". For an adverse media trial, that favors settings the firm can narrow to its highest risks and explain.
- 30 September 2026. AMLA, the EU anti-money laundering authority, dated its final report on draft standards under Article 28(1) of Regulation (EU) 2024/1624. Its draft Article 8, written for the sources used in customer due diligence, lists "the credibility of the source, including its reputation", official status, how current the information is and accuracy. It also asks firms to understand that information "regardless of the language in which that information is provided". It does not name adverse media, though the same tests fit a vendor's sources. The drafts go to the European Commission for adoption first, so they do not bind yet.
The trial ends in a file. It holds the test set and the expected answers, the counts per tool and per group, the settings each tool ran with, the vendor's written answers on reporting, archive and data sharing, and the name of the person who signed the decision. When an auditor asks why this tool was chosen, or a supervisor asks whether the screening is fit for purpose, the same file answers both.
