An investigation into the stability of contextual document clustering.

Niall Rooney, David Patterson, Mykola Galushka, Vladimir Dobrynin, Elena Smirnova

    Research output: Contribution to journalArticle

    2 Citations (Scopus)

    Abstract

    In this article, we assess the effectiveness of Contextual Document Clustering (CDC) as a means of indexing within a dynamic and rapidly changing environment. We simulate a dynamic environment, by splitting two chronologically ordered datasets into time-ordered segments and assessing how the technique performs under two different scenarios. The first is when new documents are added incrementally without reclustering [incremental CDC (iCDC)], and the second is when reclustering is performed [nonincremental CDC (nCDC)]. The datasets are very large, are independent of each other, and belong to two very different domains. We show that CDC itself is effective at clustering very large document corpora, and that, significantly, it lends itself to a very simple, efficient incremental document addition process that is seen to be very stable over time despite the size of the corpus growing considerably. It was seen to be effective at incrementally clustering new documents even when the corpus grew to six times its original size. This is in contrast to what other researchers have found when applying similar simple incremental approaches to document clustering. The stability of iCDC is accounted for by the unique manner in which CDC discovers cluster themes.
    LanguageEnglish
    Pages256-266
    JournalJASIST
    Volume59
    Issue number2
    DOIs
    Publication statusPublished - Jan 2008

    Cite this

    Rooney, N., Patterson, D., Galushka, M., Dobrynin, V., & Smirnova, E. (2008). An investigation into the stability of contextual document clustering. 59(2), 256-266. https://doi.org/10.1002/asi.20740
    Rooney, Niall ; Patterson, David ; Galushka, Mykola ; Dobrynin, Vladimir ; Smirnova, Elena. / An investigation into the stability of contextual document clustering. 2008 ; Vol. 59, No. 2. pp. 256-266.
    @article{520c92e4059c4a7a96792b26b9430376,
    title = "An investigation into the stability of contextual document clustering.",
    abstract = "In this article, we assess the effectiveness of Contextual Document Clustering (CDC) as a means of indexing within a dynamic and rapidly changing environment. We simulate a dynamic environment, by splitting two chronologically ordered datasets into time-ordered segments and assessing how the technique performs under two different scenarios. The first is when new documents are added incrementally without reclustering [incremental CDC (iCDC)], and the second is when reclustering is performed [nonincremental CDC (nCDC)]. The datasets are very large, are independent of each other, and belong to two very different domains. We show that CDC itself is effective at clustering very large document corpora, and that, significantly, it lends itself to a very simple, efficient incremental document addition process that is seen to be very stable over time despite the size of the corpus growing considerably. It was seen to be effective at incrementally clustering new documents even when the corpus grew to six times its original size. This is in contrast to what other researchers have found when applying similar simple incremental approaches to document clustering. The stability of iCDC is accounted for by the unique manner in which CDC discovers cluster themes.",
    author = "Niall Rooney and David Patterson and Mykola Galushka and Vladimir Dobrynin and Elena Smirnova",
    year = "2008",
    month = "1",
    doi = "10.1002/asi.20740",
    language = "English",
    volume = "59",
    pages = "256--266",
    number = "2",

    }

    Rooney, N, Patterson, D, Galushka, M, Dobrynin, V & Smirnova, E 2008, 'An investigation into the stability of contextual document clustering.', vol. 59, no. 2, pp. 256-266. https://doi.org/10.1002/asi.20740

    An investigation into the stability of contextual document clustering. / Rooney, Niall; Patterson, David; Galushka, Mykola; Dobrynin, Vladimir; Smirnova, Elena.

    Vol. 59, No. 2, 01.2008, p. 256-266.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - An investigation into the stability of contextual document clustering.

    AU - Rooney, Niall

    AU - Patterson, David

    AU - Galushka, Mykola

    AU - Dobrynin, Vladimir

    AU - Smirnova, Elena

    PY - 2008/1

    Y1 - 2008/1

    N2 - In this article, we assess the effectiveness of Contextual Document Clustering (CDC) as a means of indexing within a dynamic and rapidly changing environment. We simulate a dynamic environment, by splitting two chronologically ordered datasets into time-ordered segments and assessing how the technique performs under two different scenarios. The first is when new documents are added incrementally without reclustering [incremental CDC (iCDC)], and the second is when reclustering is performed [nonincremental CDC (nCDC)]. The datasets are very large, are independent of each other, and belong to two very different domains. We show that CDC itself is effective at clustering very large document corpora, and that, significantly, it lends itself to a very simple, efficient incremental document addition process that is seen to be very stable over time despite the size of the corpus growing considerably. It was seen to be effective at incrementally clustering new documents even when the corpus grew to six times its original size. This is in contrast to what other researchers have found when applying similar simple incremental approaches to document clustering. The stability of iCDC is accounted for by the unique manner in which CDC discovers cluster themes.

    AB - In this article, we assess the effectiveness of Contextual Document Clustering (CDC) as a means of indexing within a dynamic and rapidly changing environment. We simulate a dynamic environment, by splitting two chronologically ordered datasets into time-ordered segments and assessing how the technique performs under two different scenarios. The first is when new documents are added incrementally without reclustering [incremental CDC (iCDC)], and the second is when reclustering is performed [nonincremental CDC (nCDC)]. The datasets are very large, are independent of each other, and belong to two very different domains. We show that CDC itself is effective at clustering very large document corpora, and that, significantly, it lends itself to a very simple, efficient incremental document addition process that is seen to be very stable over time despite the size of the corpus growing considerably. It was seen to be effective at incrementally clustering new documents even when the corpus grew to six times its original size. This is in contrast to what other researchers have found when applying similar simple incremental approaches to document clustering. The stability of iCDC is accounted for by the unique manner in which CDC discovers cluster themes.

    U2 - 10.1002/asi.20740

    DO - 10.1002/asi.20740

    M3 - Article

    VL - 59

    SP - 256

    EP - 266

    IS - 2

    ER -

    Rooney N, Patterson D, Galushka M, Dobrynin V, Smirnova E. An investigation into the stability of contextual document clustering. 2008 Jan;59(2):256-266. https://doi.org/10.1002/asi.20740