What chatGPT cannot do for you: Explanation of human symbol processing based on brain-constrained networks

Jul 20, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Guest talk by Prof. Andreas Knoblauch: Learning mechanisms in neural networks and in artificial intelligence

28.07.2023, 16 - 18 c.t. FU Berlin JK 31/102 (Habelschwerdter Allee 45)

Jul 14, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Guest talk by Phuc Nguyen: A brain-constrained neural network model of proper names and category terms

31.05.2023, 16 -18 c.t. FU Berlin, JK 31/102 (Habelschwerdter Allee 45)

May 19, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

New PhD student hired

A new PhD student, Agata Feledyn, has just joined the team. Welcome!

Apr 03, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

New paper: Brain-constrained neural modeling explains fast mapping of words to meaning

Marika Constant, Friedemann Pulvermüller and Rosario Tomasello have used brain-constrained neural networks to simulate key features of semantic associative learning with only a few simultaneous exposures to objects, actions and corresponding word forms. This corresponds to so-called "fast-mapping": the rapid association of symbols and meaning. Most neural network learning algorithms fail to achieve rapid information storape quickly, raising the question of whether there can be a mechanistic explanation of fast-mapping. The researchers have compared two different networks: one with prior encounters with phonological and conceptual knowledge, as claimed by fast-mapping theory and one without. Simulations showed that word-specific representations emerged in the network modelled after fast-mapping theory after 1-10 learning events, but only after 40-100 learning events in the direct word learning model. These findings provide a better understanding of the critical mechanisms underlying the human brain's unique ability to acquire new words quickly. You can read the full paper published in  Cerebral Cortex   here .

Mar 01, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Poster at CNS: Verbal Working Memory depends on Network Architecture: Evidence from a Brain-constrained Network Model

Maxime Carrière will present a poster on his current research in the MatCo project at the CNS conference on Saturday, March 25th. Humans are able to learn and use a broad range of words and other symbols, whereas Monkeys are limited to acquiring small vocabularies of signs, including sounds and gestures. Earlier works showed this difference may depend on network's architecture, and especially the specifically connectivity features of areas in the left-perisylvian cortex known to be relevant for language. We are here asking whether these observations generalize across different model types and can be confirmed (1) in brain-constrained models mimicking a larger range of cortical areas, including perisylvian as well as additional areas relevant for sensorimotor, conceptual and semantic processing (2) in both meanfield models and networks of spiking integrate-and-fire neurons, and (3) in models implementing important features of the connectivity of human and monkey brains. While all models built distributed cell assemblies (CAs), CA sizes were larger in the meanfield than in the spiking models. Furthermore, larger cell assemblies emerged in the 'Human model' (HM) than it the 'monkey model' (MM). The duration of the phase where activity was held in the CAs differed significantly and substantially between models, with much longer reverberation times for humans than monkey's architectures and for mean field than spiking models. These results confirm that the difference in brain architecture between monkeys and humans, especially the connectivity provided by the arcuate fascicle, may be crucial for maintenance of reverberatory activity in word related cell assemblies and thus for the emergence of working memory specifically for spoken words.

Feb 15, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

New PhD student hired

A new PhD student,  Fynn Dobler , has just joined the team. Welcome!

Feb 01, 2023 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

New paper: Influence of language on perception and concept formation in a brain-constrained deep neural network model

Malte Henningsen-Schomers, Max Garagnani and Friedemann Pulvermüller have used a brain-constrained neural network model to investigate the influence of language on concept formation. The model acquired several distinct concrete and abstract concepts with or without an associated linguistic label. The resulting neural representations were evaluated on two properties: whether instances of a concept were similar to each other, and whether instances of distinctly different concepts were dissimilar. They have found that supplying a label referring to the different concepts improved both within-concept similarity and between-concept dissimilarity. This effect was particularly striking for abstract concepts: without a label, abstract concepts scored low on both properties. These results offer a neurobiological explanation for causal effects of language strcuture both on concept formation and perceptuo-motor processing of acquired concepts. Furthermore, they offer a novel prediction: such "Whorfian" effects of language on perception should be modulated by the concreteness and abstractness of the acquired concept, with a stronger effect on abstract concepts. You can read the full paper published in  Philosophical Transactions of the Royal Society B   here , and explore the data with an interactive data visualization here .

Dec 30, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Master thesis defense: "Language effects on concrete and abstract concepts: The temporal dynamics of semantic processing in a brain-constrained neural network model"

Fynn Dobler, a student assistant in the MatCo project, has successfully defended his master thesis on the temporal dynamics of semantic processing in the Potsdam Embodied Cognition Group (PECoG) . The first supervisor was Prof Dr Dr Friedemann Pulvermüller, the second supervisor was Prof Dr Martin Fischer. The talk was recorded and is available online here .

Nov 23, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Talk, science slam and poster session at CoNSoLER conference

Three team members will present novel research results at the CoNSoLER conference in Poznan. On Friday, 07.10., Maxime Carrière will present his research on the impact of different network architectures - either resembling the cortex of an ape or a human - on object and action word learning in a Science Slam session. He will present the results in more detail during his poster session on Sunday, 09.10. On Sunday, 09.10., Friedemann Pulvermüller will give a keynote on brain-constrained neural network modelling and its usage in neurolinguistic and cognitive research. Fynn Dobler will hold a talk about the impact of learning words on semantic processing of concrete and abstract concepts in the model.

Oct 01, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

SturopX seminar on neural network modelling

This winter semester, Malte Henningsen-Schomers and Fynn Dobler will organize a SturopX seminar on the applicability of brain-constrained neural network to various linguistic research questions. 15 students will have the opportunity to acquire and apply skills in neural network modelling, data science, data visualization and linguistics to design and conduct a research project.

Oct 01, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Talk at workshop "From associations to cognition": Associative mechanisms for building concepts and semantics

On July 4th, Prof. Dr. Dr. Friedemann Pulvermüller will give a talk an associative mechanisms for building concepts and semantics during the workshop " From association to cognition ", hosted by the Laboratoire de Psychologie Cognitive of Aix-Marseille university. He will present novel insights from neurocomputational modelling research undertaken in the MatCo project.

Jul 01, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Poster at ICON 2022: Influence of linguistic labels on concept formation and perception in a deep unsupervised neural network model

Malte Henningsen-Schomers will present MatCo research in a poster session at the ICON conference in Helsinki on 20.05.2022. OBJECTIVES/RESEARCH QUESTION Whether language influences perception and thought remains a subject of intense debate. We address this question in a brain-constrained neurocomputational model of fronto-occipital (extrasylvian) and frontotemporal (perisylvian) cortex including spiking neurons. The unsupervised neural network was simultaneously presented with word forms (phonological patterns, “labels”) in perisylvian areas and semantic grounding information (sensory-motor patterns, “percepts”) in extrasylvian areas representing either concrete or abstract concepts. Following the approach used in a previous simulation, each to-belearned concept was modeled as a triplet of partly overlapping percepts; the models were trained under two conditions: each instance of a perceptual triplet (patterns in extrasylvian areas) was repeatedly paired with patterns in perisylvian areas consisting of either (a) a corresponding word form (label condition), or (b) noise (no-label condition). We quantified the emergence of neuronal representations for the conceptually-related percepts using dissimilarity (Euclidean distance) of neuronal activation vectors during perceptual stimulation. Category learning was measured as the difference between within- and between-concept dissimilarity values (DissimDiff) of perceptual activation patterns. RESULTS A repeated-measures ANOVA with factors SemanticType (concrete/abstract) and Labelling showed main effects of both SemanticType and Labelling, and a significant interaction. We also quantified the “label effect” in percentage change from NoLabel to Label conditions, separately for between- and withincategory dissimilarities. This showed that the label effect was mainly driven by changes in between-category dissimilarity, was significantly larger for abstract than concrete concepts, and became even larger in the “deeper” layers of the model. CONCLUSION A brain-constrained neurocomputational simulation was performed to explore putative brain mechanisms of associating conceptual categories with linguistic labels. We found a clear Whorfian effect of category labels on the processing of conceptual instances: the model’s activity in response to perceptuo-motor grounding patterns was modulated depending on whether or not labels had been provided during the training phase. Labels were highly beneficial for semantic category learning performance, and this benefit was more strongly pronounced for abstract compared to concrete concepts and even more so in the deeper-lying semantic ‘hub’ areas of the model than in the primary areas, where stimulation was given.

May 01, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Teaching at Berlin School of Mind and Brain: Meaning in mind and brain

This semester, Prof. Dr. Dr. Friedemann Pulvermüller will give the lecture "Meaning in mind and brain" at the Berlin School of Mind and Brain. Language has been investigated from a range of perspectives. Linguists have described it as a formal system focusing on levels that range from phonology to syntax, semantics and pragmatics. Both linguists and psychologists worked on models focusing on the time course of linguistic processing, so that these psycholinguistic models could be tested in behavioral experiments. Most recently, neuroand cognitive scientists have attempted to spell out the brain mechanisms of language in terms of neuronal structure and function. These efforts are founded in neuroscience data about the brain loci that activate when specific linguistic operations occur, the time course of their activation and the effects of specific lesions. The lecture series will provide a broad introduction into these linguistic, psycholinguistic and neurolinguistics research streams and highlight a range of cutting-edge behavioral and neuroscience findings addressing a broad range of linguistic issues, including, for example, the recognition of words, the parsing of sentences, the computation of the meaning and of the communicative function of utterances. Language development and language disorders caused by disease of the brain will also be in the focus. To accommodate language processing, psycho- and neurolinguists make use of theoretical and computational models. The modeling approaches discussed range from theoretical models of the language system to language processing to (neuro-)computationally implemented models. The 3 experimental approaches under discussion will range from behavioral (reaction time studies, eye tracking) to neuroimaging methods (EEG, MEG, fMRI, NIRS) and neuropsychological ones (patient studies, TMS, tDCS). Preparatory readings: Knoeferle, P., & Guerra, E. (2016). Visually situated language comprehension. Linguistics and Language Compass, 10(2), 66–82. https://doi.org/10.1111/lnc3.12177 Munster, K., & Knoeferle, P. (2017). Situated Language Processing Across the Lifespan: A Review. International Journal of English Linguistics, 7(1), 1–13. https://doi.org/10.5539/ijel.v7n1p1 Pulvermüller, F., & Fadiga, L. (2016). Brain language mechanisms built on action and perception. In G. Hickok & S. L. Small (Eds.), Neurobiology of language (pp. 311-324). Amsterdam: Elsevier. Pulvermuller, F. (2018). Neural reuse of action perception circuits for language, concepts and communication. Progress in Neurobiology, 160, 1-44. doi: 10.1016/j.pneurobio.2017.07.001

Apr 01, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Paper available as Open Access

The paper "Biological constraints on neural network models of cognitive function" is now available as an open access manuscript on Europe PMC .

Mar 05, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Talk: Neural mechanisms of form meaning assemblage in construction learning at 9th International Conference of the German Cognitive Linguistics Association

Prof. Dr. Dr. Friedemann Pulvermüller is giving a talk at the 9 th International Conference of the German Cognitive Linguistics Association, detailing the neural basis for research in MatCo. You can read the abstract here.

Mar 03, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

New administrator hired

A new administrator, Margarita Wiese, has just joined the team. Welcome!

Feb 01, 2022 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Interactive data visualization for modelling concrete and abstract concepts

We published a new interactive visualization of the data in Henningsen-Schomers & Pulvermüller (2021) on our website .

Nov 15, 2021 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

New Publication: Modelling concrete and abstract concepts using brain-constrained deep neural networks

Malte Henningsen-Schomers and Friedemann Pulvermüllers new paper  Modelling concrete and abstract concepts using brain-constrained deep neural networks has just been published in Psychological Research.

Nov 11, 2021 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition

Guest Talk by Prof. Dr. Günther Palm: How to compute with cell assemblies and phase sequences

One of our external collaborators, Prof. Dr. Günther Palm, held a guest talk on the topic "How to compute with cell assemblies and phase sequences". Please find more details here .

Nov 03, 2021 | ERC Advanced Grant 883811, MatCo: Material Constraints Enabling Human Cognition