---
title: "Siper benchmark: personal data detection on Turkish text"
description: "10 systems on the same 403 human-verified Turkish documents: recall, precision, F1, false alarms and latency, with the method and limits."
url: https://siper.io/benchmark
---

Benchmark · October 2026

# Siper finds 85.7% of the personal data in our Turkish test set. The nearest alternative finds 69.3%.

Measured 2 November 2026, "Turkish and English" option. Every system ran on the same 403 documents.

To read these numbers together, see [how to read a personal data benchmark](https://siper.io/blog/how-to-read-the-benchmark). It explains recall, precision and false alarm.

Special categories

## On health data, Siper finds 81%. The nearest alternative finds 51%.

Siper has the highest recall in 5 of 7 special categories under KVKK Article 6. Siper is the only system above 0.65 in all 7 categories.

Recall of Siper and of the nearest alternative in each special category

| Category | Siper | Nearest alternative |
| --- | --- | --- |
| Health | 81%Higher | 51%GPT-5-mini |
| Religion | 86% | 88%GPT-5-mini, higher |
| Ethnic origin | 91% | 96%GPT-5-mini, higher |
| Political opinion | 88%Higher | 81%GPT-5-mini |
| Union membership | 94%Higher | 89%GPT-5-mini |
| Criminal conviction | 77%Higher | 53%GPT-5-mini |
| Sex life | 90%Higher | 71%GPT-5-mini |

By class

## Siper's recall in each class

Measured 2 November 2026, "Turkish and English" option.

Siper recall for person name, address and seven special-category classes

| Class | Recall |
| --- | --- |
| Person name | 90.4% |
| Address | 77.8% |
| Health | 80.7% |
| Religion | 86.0% |
| Ethnic origin | 91.2% |
| Political opinion | 88.1% |
| Union membership | 94.0% |
| Criminal conviction | 77.4% |
| Sex life | 89.6% |

By category

## Only 4 of 10 systems have a label for all 7 special categories.

Each cell shows recall in one category. Darker cells mean higher recall. A cell marked n/a is out of scope: the system has no label for that category. A cell with 0% is a measured zero, because the system has the label and found nothing correct on this set.

On a phone, scroll the table sideways.

All systemsCan run on site

Showing 10 of 10 systems in both tables. Showing 4 of 10 systems in both tables.

Recall per system in each of the 7 special categories, highest health recall first

| System | Health | Religion | Ethnic origin | Political opinion | Union membership | Criminal conviction | Sex life |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Gurubase Siper | 81% | 86% | 91% | 88% | 94% | 77% | 90% |
| GPT-5-mini | 51% | 88% | 96% | 81% | 89% | 53% | 71% |
| Limina (Private AI) | 42% | 56% | 91% | 37% | n/a | 0% | 4% |
| Qwen2.5-32B | 37% | 30% | 79% | 47% | 85% | 17% | 56% |
| Gemini 2.5 Flash | 24% | 78% | 68% | 39% | 77% | 15% | 52% |
| Google Cloud DLP | 0% | 30% | 60% | 0% | 0% | 6% | n/a |
| Tonic Textual | 0% | n/a | 9% | n/a | n/a | n/a | n/a |
| Azure AI Language | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| OpenAI Privacy Filter | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Microsoft Presidio | n/a | n/a | n/a | n/a | n/a | n/a | n/a |

Systems

## Siper has the highest recall and F1.

Among systems that can run on site, the nearest alternative is Qwen2.5-32B, with recall of 48.3%.

The tables list 9 of the 15 alternatives we evaluated. Ranges are 95% confidence intervals from 1,000 document resamples. Range values and bars use a scale from 0 to 1.

Sorted by recall, highest first. On a phone, scroll the table sideways.

Showing 10 of 10 systems in both tables. Showing 4 of 10 systems in both tables.

Recall, precision, F1, false alarm rate and latency for each system

| System | Can run on site | Recall | Precision | F1 | False alarm | Median latency |
| --- | --- | --- | --- | --- | --- | --- |
| Gurubase Siper | Yes | 85.7%0.839 to 0.877 | 56.1% | 0.6780.652 to 0.700 | 14.3% | 176 msOne 8-vCPU server, no GPU · October 2026 |
| GPT-5-mini | No | 69.3%0.653 to 0.730 | 62.8% | 0.6590.632 to 0.684 | 12.1% | 18,115 msp95 19,621 msCloud API · July 2026 |
| Gemini 2.5 Flash | No | 52.2%0.470 to 0.571 | 61.5% | 0.5650.526 to 0.600 | 1.9% | 1,790 msp95 2,072 msCloud API · July 2026 |
| Limina (Private AI) | No | 51.5%0.477 to 0.552 | 65.2% | 0.5750.543 to 0.605 | 48.6% | 561 msp95 629 msCloud API · July 2026 |
| Qwen2.5-32B | Yes | 48.3%0.446 to 0.521 | 58.5% | 0.5290.496 to 0.561 | Not measured | 19,242 msp95 19,367 msLocal GPU · July 2026 |
| Azure AI Language | No | 26.7%0.220 to 0.311 | 91.8% | 0.4140.355 to 0.466 | 3.4% | 221 msp95 264 msCloud API · July 2026 |
| Google Cloud DLP | No | 24.5%0.202 to 0.288 | 42.6% | 0.3110.265 to 0.354 | 40.2% | 220 msp95 289 msCloud API · July 2026 |
| Tonic Textual | No | 22.2%0.180 to 0.265 | 24.1% | 0.2310.193 to 0.264 | 42.7% | 491 msp95 568 msCloud API · July 2026 |
| Microsoft Presidio | Yes | 19.2%0.156 to 0.229 | 9.0% | 0.1220.098 to 0.146 | 98.1% | 7 msp95 8 msLocal CPU · July 2026 |
| OpenAI Privacy Filter | Yes | 8.9%0.059 to 0.120 | 48.9% | 0.1500.103 to 0.198 | 18.4% | 4,600 msOne 8-vCPU server, no GPU · 2026 |

**Recall**

The share of real personal data that a system finds.

**Precision**

The share of detections that are correct. Siper's precision is 56.1%.

**F1**

One score that combines recall and precision.

**False alarm**

The share of 321 real documents with no personal data that received at least one unnecessary mask. Read it with recall, because a system that detects little also makes few false alarms.

**Median latency**

The middle response time. On-prem systems: median per document on one 8-vCPU server without GPU, unless the row states another setup. Cloud services: API response time including network. Systems ran on different infrastructure, so this is not a direct hardware comparison. Dates also differ, as the table shows. Use it to compare orders of magnitude.

Method

## Every system ran on the same documents and the same scoring software.

- The test set has 403 Turkish documents with human-verified labels. We evaluated the alternatives in October 2026. Siper's own numbers are from 2 November 2026.
- A referee review corrected the test set. We then rescored all systems on the corrected set.
- The scoring software runs independently of the compared products. It measures at entity level.
- Commercial products ran through their official APIs, with real access keys, on the evaluation date.
- We mapped each system's labels to one common taxonomy. A category that a system never claims counts as out of scope, not as zero.
- Each score carries a 95% confidence interval from 1,000 document resamples.
- False alarms come from a separate group of 321 real documents with no personal data.
- We record the date and the system version of each evaluation.

We ran 135 paired comparisons: 15 alternatives across 9 classes (person name, address and seven special-category classes). We used document-level paired resampling, and we claim no lead when the interval of the difference includes zero. 113 comparisons showed a significant difference in Siper's favour, and 20 showed no significant difference. 2 favoured the alternative.

Latency: On-prem systems: median per document on one 8-vCPU server without GPU, unless the row states another setup. Cloud services: API response time including network. Systems ran on different infrastructure, so this is not a direct hardware comparison.

Not in the tables: Global cloud provider C. Its official documentation states that it does not support Turkish personal data detection.

A detailed report with mapping rules, system configurations and rerun notes is available under a non-disclosure agreement. The JSON file carries the status and the source of the data.

[Ask for the report](https://siper.io/contact)[Read these results as JSON](https://siper.io/benchmark.json)

Limits

## Limits of this test

Read the results with these limits in mind.

- The test set is ours, and this is not an independent audit.
- The test set is Turkish only. Siper's accuracy figures use its "Turkish and English" option. Siper offers three language options: [read how they differ](https://siper.io/pii-masking#languages).
- Systems ran on different infrastructure, so use the table to compare orders of magnitude.
- Prompt injection detection is not part of this test.

Next step

## Test Siper on your own text.

[Book a demo](https://siper.io/contact)[Try the playground](https://siper.io/playground)
