BIROn - Birkbeck Institutional Research Online

    Intelligent choice of the number of clusters in K-means clustering: an experimental study with different cluster Spreads

    Chiang, M.M.T. and Mirkin, Boris (2010) Intelligent choice of the number of clusters in K-means clustering: an experimental study with different cluster Spreads. Journal of Classification 27 (1), pp. 3-40. ISSN 0176-4268.

    Full text not available from this repository.

    Abstract

    The issue of determining “the right number of clusters” in K-Means has attracted considerable interest, especially in the recent years. Cluster intermix appears to be a factor most affecting the clustering results. This paper proposes an experimental setting for comparison of different approaches at data generated from Gaussian clusters with the controlled parameters of between- and within-cluster spread to model cluster intermix. The setting allows for evaluating the centroid recovery on par with conventional evaluation of the cluster recovery. The subjects of our interest are two versions of the “intelligent” K-Means method, ik-Means, that find the “right” number of clusters by extracting “anomalous patterns” from the data one-by-one. We compare them with seven other methods, including Hartigan’s rule, averaged Silhouette width and Gap statistic, under different between- and within-cluster spread-shape conditions. There are several consistent patterns in the results of our experiments, such as that the right K is reproduced best by Hartigan’s rule – but not clusters or their centroids. This leads us to propose an adjusted version of iK-Means, which performs well in the current experiment setting.

    Metadata

    Item Type: Article
    Keyword(s) / Subject(s): K-Means clustering, number of clusters, anomalous pattern, Hartigan’s rule, gap statistic
    School: Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences
    Research Centres and Institutes: Structural Molecular Biology, Institute of (ISMB)
    Depositing User: Administrator
    Date Deposited: 01 Feb 2011 15:10
    Last Modified: 09 Aug 2023 12:30
    URI: https://eprints.bbk.ac.uk/id/eprint/1898

    Statistics

    Activity Overview
    6 month trend
    0Downloads
    6 month trend
    392Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item