Benchmark · October 2026
Siper finds 85.7% of the personal data in our Turkish test set. The nearest alternative finds 69.3%.
Measured 2 November 2026, "Turkish and English" option. Every system ran on the same 403 documents.
To read these numbers together, see how to read a personal data benchmark. It explains recall, precision and false alarm.
Special categories
On health data, Siper finds 81%. The nearest alternative finds 51%.
Siper has the highest recall in 5 of 7 special categories under KVKK Article 6. Siper is the only system above 0.65 in all 7 categories.
| Category | Siper | Nearest alternative |
|---|---|---|
| Health | 81%Higher | 51%GPT-5-mini |
| Religion | 86% | 88%GPT-5-mini, higher |
| Ethnic origin | 91% | 96%GPT-5-mini, higher |
| Political opinion | 88%Higher | 81%GPT-5-mini |
| Union membership | 94%Higher | 89%GPT-5-mini |
| Criminal conviction | 77%Higher | 53%GPT-5-mini |
| Sex life | 90%Higher | 71%GPT-5-mini |
By class
Siper's recall in each class
Measured 2 November 2026, "Turkish and English" option.
| Class | Recall |
|---|---|
| Person name | 90.4% |
| Address | 77.8% |
| Health | 80.7% |
| Religion | 86.0% |
| Ethnic origin | 91.2% |
| Political opinion | 88.1% |
| Union membership | 94.0% |
| Criminal conviction | 77.4% |
| Sex life | 89.6% |
By category
Only 4 of 10 systems have a label for all 7 special categories.
Each cell shows recall in one category. Darker cells mean higher recall. A cell marked n/a is out of scope: the system has no label for that category. A cell with 0% is a measured zero, because the system has the label and found nothing correct on this set.
On a phone, scroll the table sideways.
Showing 10 of 10 systems in both tables. Showing 4 of 10 systems in both tables.
| System | Health | Religion | Ethnic origin | Political opinion | Union membership | Criminal conviction | Sex life |
|---|---|---|---|---|---|---|---|
| Gurubase Siper | 81% | 86% | 91% | 88% | 94% | 77% | 90% |
| GPT-5-mini | 51% | 88% | 96% | 81% | 89% | 53% | 71% |
| Limina (Private AI) | 42% | 56% | 91% | 37% | n/a | 0% | 4% |
| Qwen2.5-32B | 37% | 30% | 79% | 47% | 85% | 17% | 56% |
| Gemini 2.5 Flash | 24% | 78% | 68% | 39% | 77% | 15% | 52% |
| Google Cloud DLP | 0% | 30% | 60% | 0% | 0% | 6% | n/a |
| Tonic Textual | 0% | n/a | 9% | n/a | n/a | n/a | n/a |
| Azure AI Language | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| OpenAI Privacy Filter | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
| Microsoft Presidio | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
Systems
Siper has the highest recall and F1.
Among systems that can run on site, the nearest alternative is Qwen2.5-32B, with recall of 48.3%.
The tables list 9 of the 15 alternatives we evaluated. Ranges are 95% confidence intervals from 1,000 document resamples. Range values and bars use a scale from 0 to 1.
Sorted by recall, highest first. On a phone, scroll the table sideways.
Showing 10 of 10 systems in both tables. Showing 4 of 10 systems in both tables.
| System | Can run on site | Recall | Precision | F1 | False alarm | Median latency |
|---|---|---|---|---|---|---|
| Gurubase Siper | Yes | 85.7%0.839 to 0.877 | 56.1% | 0.6780.652 to 0.700 | 14.3% | 176 msOne 8-vCPU server, no GPU · October 2026 |
| GPT-5-mini | No | 69.3%0.653 to 0.730 | 62.8% | 0.6590.632 to 0.684 | 12.1% | 18,115 msp95 19,621 msCloud API · July 2026 |
| Gemini 2.5 Flash | No | 52.2%0.470 to 0.571 | 61.5% | 0.5650.526 to 0.600 | 1.9% | 1,790 msp95 2,072 msCloud API · July 2026 |
| Limina (Private AI) | No | 51.5%0.477 to 0.552 | 65.2% | 0.5750.543 to 0.605 | 48.6% | 561 msp95 629 msCloud API · July 2026 |
| Qwen2.5-32B | Yes | 48.3%0.446 to 0.521 | 58.5% | 0.5290.496 to 0.561 | Not measured | 19,242 msp95 19,367 msLocal GPU · July 2026 |
| Azure AI Language | No | 26.7%0.220 to 0.311 | 91.8% | 0.4140.355 to 0.466 | 3.4% | 221 msp95 264 msCloud API · July 2026 |
| Google Cloud DLP | No | 24.5%0.202 to 0.288 | 42.6% | 0.3110.265 to 0.354 | 40.2% | 220 msp95 289 msCloud API · July 2026 |
| Tonic Textual | No | 22.2%0.180 to 0.265 | 24.1% | 0.2310.193 to 0.264 | 42.7% | 491 msp95 568 msCloud API · July 2026 |
| Microsoft Presidio | Yes | 19.2%0.156 to 0.229 | 9.0% | 0.1220.098 to 0.146 | 98.1% | 7 msp95 8 msLocal CPU · July 2026 |
| OpenAI Privacy Filter | Yes | 8.9%0.059 to 0.120 | 48.9% | 0.1500.103 to 0.198 | 18.4% | 4,600 msOne 8-vCPU server, no GPU · 2026 |
- Recall
- The share of real personal data that a system finds.
- Precision
- The share of detections that are correct. Siper's precision is 56.1%.
- F1
- One score that combines recall and precision.
- False alarm
- The share of 321 real documents with no personal data that received at least one unnecessary mask. Read it with recall, because a system that detects little also makes few false alarms.
- Median latency
- The middle response time. On-prem systems: median per document on one 8-vCPU server without GPU, unless the row states another setup. Cloud services: API response time including network. Systems ran on different infrastructure, so this is not a direct hardware comparison. Dates also differ, as the table shows. Use it to compare orders of magnitude.
Method
Every system ran on the same documents and the same scoring software.
- The test set has 403 Turkish documents with human-verified labels. We evaluated the alternatives in October 2026. Siper's own numbers are from 2 November 2026.
- A referee review corrected the test set. We then rescored all systems on the corrected set.
- The scoring software runs independently of the compared products. It measures at entity level.
- Commercial products ran through their official APIs, with real access keys, on the evaluation date.
- We mapped each system's labels to one common taxonomy. A category that a system never claims counts as out of scope, not as zero.
- Each score carries a 95% confidence interval from 1,000 document resamples.
- False alarms come from a separate group of 321 real documents with no personal data.
- We record the date and the system version of each evaluation.
We ran 135 paired comparisons: 15 alternatives across 9 classes (person name, address and seven special-category classes). We used document-level paired resampling, and we claim no lead when the interval of the difference includes zero. 113 comparisons showed a significant difference in Siper's favour, and 20 showed no significant difference. 2 favoured the alternative.
Latency: On-prem systems: median per document on one 8-vCPU server without GPU, unless the row states another setup. Cloud services: API response time including network. Systems ran on different infrastructure, so this is not a direct hardware comparison.
Not in the tables: Global cloud provider C. Its official documentation states that it does not support Turkish personal data detection.
A detailed report with mapping rules, system configurations and rerun notes is available under a non-disclosure agreement. The JSON file carries the status and the source of the data.
Limits
Limits of this test
Read the results with these limits in mind.
- The test set is ours, and this is not an independent audit.
- The test set is Turkish only. Siper's accuracy figures use its "Turkish and English" option. Siper offers three language options: read how they differ.
- Systems ran on different infrastructure, so use the table to compare orders of magnitude.
- Prompt injection detection is not part of this test.
Next step