Extending symbol interdependency: perceptual scene vectors

Ghose, Ajitesh and Cooper, Richard P. (2018) Extending symbol interdependency: perceptual scene vectors. In: ESLP 2018, 10-12 Aug 2018, Lancaster, UK.

Abstract

Louwerse (2011) advances the symbol interdependency hypothesis by demonstrating empirically the importance of statistical regularities in linguistic surface structures. Symbol interdependency posits that meaning extraction attributed to embodied representations or algorithms should instead be attributed to language. In a series of seven computational experiments, we find that language surface structure best encodes meaning when the structural cues are sufficiently constrained by modeller- determined feature sets, with performance deteriorating for randomly selected language surface cues. We further find that Latent Semantic Analysis’ meaning encoding improves as weaker dimensions are removed. These findings collectively indicate that although language is important, increasing the relevance of linguistic statistical regularities is also critical. We introduce Perceptual Scene Vectors (PSVs), a novel approach that uses object co-occurrences from images to automatically extract strong associative and taxonomic relationships. This approach extracts these associations more successfully than the language-based approaches, measured both qualitatively and quantitatively, with an original application of a cluster-correspondence metric. PSVs encode meaning without modellers hand-coding relevant features. This provides an ecologically valid approach to extending symbol interdependency beyond language and partially solving the relevance problem in semantics by grounding meaning extraction in real-world visual scenes.

Metadata

Statistics

DownloadsShow export options

Export as XMLJSONCSV

Activity Overview

6 month trend

114Downloads

6 month trend

298Hits

Additional statistics are available via IRStats2.

Archive Staff Only (login required)

Edit/View Item