Data Cleaning publication categorizer

We propose a new data-driven transformation that facilitates many data mining, interpretation, and analysis tasks. Our approach, called MembershipMap, strives to extract the underlying sub-concepts of each raw attribute, and uses the orthogonal union of these sub-concepts to define a new space. The sub-concept soft labels of each point in the original space determine the position of that point in the new space. Since sub-concept labels are prone to uncertainty inherent in the original data and in the initial extraction process, a combination of labeling schemes that are based on different measures of uncertainty will be presented. In particular, we introduce the CrispMap, SoftMap, and PossibilisticMap. We show that the MembershipMap can be used as a flexible pre-processing tool to support such tasks as: sampling, data cleaning, and outlier detection.