A blocking framework for entity resolution in highly heterogeneous information spaces

George Papadakis, Ekaterini Ioannou, Themis Palpanas, Claudia Niederee, Wolfgang Nejdl

Research output: Contribution to journalArticleScientificpeer-review

Abstract

In the context of entity resolution (ER) in highly heterogeneous, noisy, user-generated entity collections, practically all block building methods employ redundancy to achieve high effectiveness. This practice, however, results in a high number of pairwise comparisons, with a negative impact on efficiency. Existing block processing strategies aim at discarding unnecessary comparisons at no cost in effectiveness. In this paper, we systemize blocking methods for clean-clean ER (an inherently quadratic task) over highly heterogeneous information spaces (HHIS) through a novel framework that consists of two orthogonal layers: the effectiveness layer encompasses methods for building overlapping blocks with small likelihood of missed matches; the efficiency layer comprises a rich variety of techniques that significantly restrict the required number of pairwise comparisons, having a controllable impact on the number of detected duplicates. We map to our framework all relevant existing methods for creating and processing blocks in the context of HHIS, and additionally propose two novel techniques: attribute clustering blocking and comparison scheduling. We evaluate the performance of each layer and method on two large-scale, real-world data sets and validate the excellent balance between efficiency and effectiveness that they achieve.
Original languageEnglish
Pages (from-to)2665-2682
JournalIEEE Transactions on Knowledge and Data Engineering
Volume25
Issue number12
DOIs
Publication statusPublished - 1 Dec 2013
Externally publishedYes

Fingerprint

Processing
Redundancy
Scheduling
Costs

Cite this

Papadakis, George ; Ioannou, Ekaterini ; Palpanas, Themis ; Niederee, Claudia ; Nejdl, Wolfgang. / A blocking framework for entity resolution in highly heterogeneous information spaces. In: IEEE Transactions on Knowledge and Data Engineering. 2013 ; Vol. 25, No. 12. pp. 2665-2682.
@article{8dbcc9b5f4294f9fb5be921ea2d3cf75,
title = "A blocking framework for entity resolution in highly heterogeneous information spaces",
abstract = "In the context of entity resolution (ER) in highly heterogeneous, noisy, user-generated entity collections, practically all block building methods employ redundancy to achieve high effectiveness. This practice, however, results in a high number of pairwise comparisons, with a negative impact on efficiency. Existing block processing strategies aim at discarding unnecessary comparisons at no cost in effectiveness. In this paper, we systemize blocking methods for clean-clean ER (an inherently quadratic task) over highly heterogeneous information spaces (HHIS) through a novel framework that consists of two orthogonal layers: the effectiveness layer encompasses methods for building overlapping blocks with small likelihood of missed matches; the efficiency layer comprises a rich variety of techniques that significantly restrict the required number of pairwise comparisons, having a controllable impact on the number of detected duplicates. We map to our framework all relevant existing methods for creating and processing blocks in the context of HHIS, and additionally propose two novel techniques: attribute clustering blocking and comparison scheduling. We evaluate the performance of each layer and method on two large-scale, real-world data sets and validate the excellent balance between efficiency and effectiveness that they achieve.",
author = "George Papadakis and Ekaterini Ioannou and Themis Palpanas and Claudia Niederee and Wolfgang Nejdl",
year = "2013",
month = "12",
day = "1",
doi = "10.1109/TKDE.2012.150",
language = "English",
volume = "25",
pages = "2665--2682",
journal = "IEEE Transactions on Knowledge and Data Engineering",
issn = "1041-4347",
publisher = "IEEE COMPUTER SOC",
number = "12",

}

A blocking framework for entity resolution in highly heterogeneous information spaces. / Papadakis, George; Ioannou, Ekaterini; Palpanas, Themis; Niederee, Claudia; Nejdl, Wolfgang.

In: IEEE Transactions on Knowledge and Data Engineering, Vol. 25, No. 12, 01.12.2013, p. 2665-2682.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - A blocking framework for entity resolution in highly heterogeneous information spaces

AU - Papadakis, George

AU - Ioannou, Ekaterini

AU - Palpanas, Themis

AU - Niederee, Claudia

AU - Nejdl, Wolfgang

PY - 2013/12/1

Y1 - 2013/12/1

N2 - In the context of entity resolution (ER) in highly heterogeneous, noisy, user-generated entity collections, practically all block building methods employ redundancy to achieve high effectiveness. This practice, however, results in a high number of pairwise comparisons, with a negative impact on efficiency. Existing block processing strategies aim at discarding unnecessary comparisons at no cost in effectiveness. In this paper, we systemize blocking methods for clean-clean ER (an inherently quadratic task) over highly heterogeneous information spaces (HHIS) through a novel framework that consists of two orthogonal layers: the effectiveness layer encompasses methods for building overlapping blocks with small likelihood of missed matches; the efficiency layer comprises a rich variety of techniques that significantly restrict the required number of pairwise comparisons, having a controllable impact on the number of detected duplicates. We map to our framework all relevant existing methods for creating and processing blocks in the context of HHIS, and additionally propose two novel techniques: attribute clustering blocking and comparison scheduling. We evaluate the performance of each layer and method on two large-scale, real-world data sets and validate the excellent balance between efficiency and effectiveness that they achieve.

AB - In the context of entity resolution (ER) in highly heterogeneous, noisy, user-generated entity collections, practically all block building methods employ redundancy to achieve high effectiveness. This practice, however, results in a high number of pairwise comparisons, with a negative impact on efficiency. Existing block processing strategies aim at discarding unnecessary comparisons at no cost in effectiveness. In this paper, we systemize blocking methods for clean-clean ER (an inherently quadratic task) over highly heterogeneous information spaces (HHIS) through a novel framework that consists of two orthogonal layers: the effectiveness layer encompasses methods for building overlapping blocks with small likelihood of missed matches; the efficiency layer comprises a rich variety of techniques that significantly restrict the required number of pairwise comparisons, having a controllable impact on the number of detected duplicates. We map to our framework all relevant existing methods for creating and processing blocks in the context of HHIS, and additionally propose two novel techniques: attribute clustering blocking and comparison scheduling. We evaluate the performance of each layer and method on two large-scale, real-world data sets and validate the excellent balance between efficiency and effectiveness that they achieve.

U2 - 10.1109/TKDE.2012.150

DO - 10.1109/TKDE.2012.150

M3 - Article

VL - 25

SP - 2665

EP - 2682

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

SN - 1041-4347

IS - 12

ER -