Inferring and Comparing the Concept Spaces of Humans and Neural Networks

One fundamental goal of cognitive science is to understand how humans mentally represent concepts. One approach to this is to survey humans how they perceive objects and their similarity. The first part of this talk will cover VICE, a Bayesian method for inferring geometric representations of concepts, i.e. vector embeddings, whose geometry is consistent with similarity task responses. Representations from VICE are very effective at predicting human responses to object similarity tasks. Representation dimensions from VICE are highly interpretable, thus revealing what properties are important when humans consider object similarity.

The second part of this talk will investigate how these human representations compare to neural network representations on the THINGS dataset, a collection of 1,854 object categories carefully curated for cognitive science research. Here we see what kinds of networks are best aligned to human understanding and what object properties that humans find important are also understood by neural networks.