BIROn - Birkbeck Institutional Research Online

    Extending symbol interdependency: perceptual scene vectors

    Ghose, Ajitesh and Cooper, Richard P. (2018) Extending symbol interdependency: perceptual scene vectors. In: ESLP 2018, 10-12 Aug 2018, Lancaster, UK.

    [img]
    Preview
    Text
    ESLP2018_Ghose_Cooper.pdf

    Download (22kB) | Preview

    Abstract

    Louwerse (2011) advances the symbol interdependency hypothesis by demonstrating empirically the importance of statistical regularities in linguistic surface structures. Symbol interdependency posits that meaning extraction attributed to embodied representations or algorithms should instead be attributed to language. In a series of seven computational experiments, we find that language surface structure best encodes meaning when the structural cues are sufficiently constrained by modeller- determined feature sets, with performance deteriorating for randomly selected language surface cues. We further find that Latent Semantic Analysis’ meaning encoding improves as weaker dimensions are removed. These findings collectively indicate that although language is important, increasing the relevance of linguistic statistical regularities is also critical. We introduce Perceptual Scene Vectors (PSVs), a novel approach that uses object co-occurrences from images to automatically extract strong associative and taxonomic relationships. This approach extracts these associations more successfully than the language-based approaches, measured both qualitatively and quantitatively, with an original application of a cluster-correspondence metric. PSVs encode meaning without modellers hand-coding relevant features. This provides an ecologically valid approach to extending symbol interdependency beyond language and partially solving the relevance problem in semantics by grounding meaning extraction in real-world visual scenes.

    Metadata

    Item Type: Conference or Workshop Item (Lecture)
    School: Birkbeck Faculties and Schools > Faculty of Science > School of Psychological Sciences
    Research Centres and Institutes: Cognition, Computation and Modelling, Centre for
    Depositing User: Ajitesh Ghose
    Date Deposited: 16 Nov 2018 13:59
    Last Modified: 02 Aug 2023 17:43
    URI: https://eprints.bbk.ac.uk/id/eprint/23697

    Statistics

    Activity Overview
    6 month trend
    87Downloads
    6 month trend
    246Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item