Main Content

Research

The TNM-Lab combines multidisciplinary fields from speech science, social, cognitive and clinical psychology with experimental neurosciences. Research of the TNM-Lab is focused on:

• action & perception
• multisensory integration
• functional (dys-)connectivity
• psychotherapy, intervention and neural plasticity
• social neuroscience
• gesture & language

All research areas are highly intertwined and founded projects are usually connected to more than one topic (see figure below). The main connecting goal is the translation of basic research from experimental psychology and neuroimaging to clinical investigations in patients with mental disorders, in particular schizophrenia and anxiety disorders.

• Inhalt ausklappen Inhalt einklappen Action & Perception

  Regarding action and perception we are interested in the perception of one’s own action consequences (CRC/TRR 135 A3), the neural processing of actions in a language context (G3, FI_3) as well as the production-perception mechanisms regarding social cognitive functioning (GUI; STR 1146/4-1).

  The goal of CRC/TRR subproject A3 (PACT) is to investigate the effects of predictive mechanisms on multisensory consequences of one’s own actions. The investigations focus on three different aspects: 1) supramodal vs. unimodal effects of action-sensory feedback matching processing, 2) the neural correlates of supramodal predictive mechanisms and their consequences and 3) the investigation of predictive mechanisms for tool-use actions and the perception of related multisensory consequences. Based on these findings related dysfunctions in patients with schizophrenia and hallucinations as well as ego-disturbances are investigated in a DFG project (PACT-SZ; STR 1146/9-1/2) and the new period of the CRC/TRR135.

  Regarding the perception and comprehension of others people’s actions, we focus mainly on multisensory integration processes relevant for the integration of speech and gesture.

  Representative publications:

  Kavroulakis, E., Kemenade, B. M. Van, Kircher, T., & Straube, B. (2022). The effect of self-generated versus externally generated actions on timing , duration , and amplitude of blood oxygen level dependent response for visual feedback processing. Human Brain Mapping, (July), 1–16. https://doi.org/10.1002/hbm.26053

  Uhlmann, L., Pazen, M., Van Kemenade, B. M., Steinsträter, O., Harris, L. R., Kircher, T., & Straube, B. (2020). Seeing your own or someone else ’ s hand moving in accordance with your action : The neural interaction of agency and hand identity. Human Brain Mapping (HBM), 41, 2474–2489. https://doi.org/10.1002/hbm.24958

  van Kemenade, B. M., Arikan, B. E., Podranski, K., Steinsträter, O., Kircher, T., & Straube, B. (2019). Distinct Roles for the Cerebellum, Angular Gyrus, and Middle Temporal Gyrus in Action–Feedback Monitoring. Cerebral Cortex, 29(4), 1520–1531. https://doi.org/10.1093/cercor/bhy048

• Inhalt ausklappen Inhalt einklappen Multisensory Integration

  Regarding multisensory integration we investigate different levels of integration for example during the perception on one’own action consequences (action & perception) or the neural processing of actions in a language context (gesture & language).

  Representative publications:

  Straube, B., Wroblewski, A., Jansen, A., & He, Y. (2018). The connectivity signature of co-speech gesture integration: The superior temporal sulcus modulates connectivity between areas related to visual gesture and auditory speech processing. NeuroImage, 181(2018), 539–549. https://doi.org/10.1016/J.NEUROIMAGE.2018.07.037

  Krala, M., van Kemenade, B. M., Straube, B., Kircher, T., & Bremmer, F. (2019). Predictive coding in a multisensory path integration task: an fMRI study. Journal of Vision, 19(11)(13), 1–15. https://doi.org/10.1167/19.11.13

  Straube, B., Kemenade, B. M. van, Arikan, B. E., Fiehler, K., Leube, D. T., Harris, L. R., & Kircher, T. (2017). Predicting the Multisensory Consequences of One’s Own Action: BOLD Suppression in Auditory and Visual Cortices. PLOS ONE, 12(1), e0169131. https://doi.org/10.1371/JOURNAL.PONE.0169131

• Inhalt ausklappen Inhalt einklappen Functional (Dys-)Connectivity

  In addition to clinically relevant differences in BOLD amplitude, differences in connectivity can be observed in patients with mental disorders, which could be explained by dysfunctional processing. In the TNM-lab functional connectivity, as indication of successful information flow between brain regions, is considered a relevant marker of dysfunctional processing. For example, a dysfunctional fronto-temporal connectivity could explain dysfunctions in patients with schizophrenia for the processing of gestures in an abstract sentence context (Straube et al., 2014). Furthermore, Dynamic Causal Modelling (DCM) suggests impaired effective connectivity in verbal pathways of patients with schizophrenia spectrum disorders during gesture-speech integration (Wroblewski et al., 2020).

  Another clinically relevant example suggests that a dysfunctional coupling between anterior cingulate cortex and the amygdala is related to non-response in a cognitive behavioral therapy (Lueken et al., 2013).

  Representative publications:

  Lueken, U., Straube, B., Konrad, C., Wittchen, H.-U., Ströhle, A., Wittmann, A., … Kircher, T. (2013). Neural Substrates of Treatment Response to Cognitive-Behavioral Therapy in Panic Disorder With Agoraphobia. American Journal of Psychiatry, 170(11), 1345–55. http://doi.org/10.1176/appi.ajp.2013.12111484

  Straube, B., Green, A., Sass, K., & Kircher, T. (2014). Superior Temporal Sulcus Disconnectivity During Processing of Metaphoric Gestures in Schizophrenia. Schizophrenia Bulletin, 40(4), 936–944. http://doi.org/10.1093/schbul/sbt110

  Straube, B., Wroblewski, A., Jansen, A., & He, Y. (2018). The connectivity signature of co-speech gesture integration: The superior temporal sulcus modulates connectivity between areas related to visual gesture and auditory speech processing. NeuroImage, 181(2018), 539–549. https://doi.org/10.1016/J.NEUROIMAGE.2018.07.037

  Wroblewski, A., He, Y., & Straube, B. (2020). Dynamic Causal Modelling suggests impaired effective connectivity in patients with schizophrenia spectrum disorders during gesture-speech integration. Schizophrenia Research IF: 4.569

  Bitsch, F., Berger, P., Nagels, A., Falkenberg, I., & Straube, B. (2021). Characterizing the theory of mind network in schizophrenia reveals a sparser network structure. Schizophrenia Research, 228, 581–589. https://doi.org/10.1016/j.schres.2020.11.026

• Inhalt ausklappen Inhalt einklappen Psychotherapy, Intervention & Neural Plasticity

  The goal of translational neuroimaging is not only to investigate dysfunctions in patients with mental disorders, but also to explore changes in neural processing and behavior which can be induced by cognitive behavioral therapy (CBT) or specific interventions such as humor training (see EKFS), gesture training (see G+) or the application of transcranial direct current stimulation (tDCS).

  For the effect of CBT, see for example:

  Yang, Y., Lueken, U., Richter, J., Hamm, A., Wittmann, A., Konrad, C., … Straube*, B., & Kircher*, T. (2020). Effect of CBT on Biased Semantic Network in Panic Disorder: A Multicenter fMRI Study using Semantic Priming. The American Journal of Psychiatry, 177:197–199; doi: 10.1176/appi.ajp.2019.20010008. *contributed equally

  Short summary of the results (Yang et al., 2020) as cartoon (Link)

  For the gesture training, see for example:

  Riedl, L., Nagels, A., Sammer, G., & Straube, B. (2020). A multimodal speech-gesture training intervention for patients with schizophrenia and its neural underpinnings – the study protocol of a randomized controlled pilot trial. Frontiers in Psychiatry, 11, 110. https://doi.org/10.3389/FPSYT.2020.00110

  For tDCS effects, see for example:

  Schülke, R., & Straube, B. (2019). Transcranial Direct Current Stimulation Improves Semantic Speech–Gesture Matching in Patients With Schizophrenia Spectrum Disorder. Schizophrenia Bulletin, 45(3), 522–530. https://doi.org/10.1093/schbul/sby144

• Inhalt ausklappen Inhalt einklappen Social Neuroscience

  Mental disorders and specifically schizophrenia is characterized by social cognitive dysfunctions. Social functioning is further related to perception and action in many ways. The goal of the TNM-Lab is to investigate how the perception of actions (PACT/PACT-SZ) and the neural correlates of gesture processing (EEG-CON, Gestik3) are related to social cognitive functioning in patients with schizophrenia. Furthermore, neural correlates of the perception and production of negative facial emotions is investigated in healthy subjects with genetic or environmental risk for schizophrenia (GUI), to demonstrate how neural production-perception matching processes could be modulated my genetic risk and might contribute to the development of schizophrenia. In this context, social intergroup relations are of specific interest, since they might contribute to the increased risk for schizophrenia observed in migrants. Finally, we investigate the effect of an humor training on social cognitive functioning in a longitudinal fMRI design (EKFS).

  Representative publications:

  Suffel, A., Nagels, A., Steines, M., Kircher, T., & Straube, B. (2020). Feeling addressed ! The neural processing of social communicative cues in patients with major depression. Human Brain Mapping, 41, 3541–3554. https://doi.org/10.1002/hbm.25027

  Steines, M., Krautheim, J. T., Neziroğlu, G., Kircher, T., & Straube, B. (2020). Conflicting group memberships modulate neural activation in an emotional production-perception network. Cortex, 126, 153–172. https://www.sciencedirect.com/science/article/pii/S0010945220300022

  Krautheim, J. T., Dannlowski, U., Steines, M., Neziroğlu, G., Acosta, H., Sommer, J., … Kircher, T. (2019). Intergroup empathy: Enhanced neural resonance for ingroup facial emotion in a shared neural production-perception network. NeuroImage, 194(March), 182–190. https://doi.org/10.1016/j.neuroimage.2019.03.048

• Inhalt ausklappen Inhalt einklappen Gesture & Language

  Comprehension of natural language is a complex capacity, depending on several cognitive and neural systems. Over the last years, knowledge of the brain processes underlying single word and sentence processing has grown by examining phonological, semantic and syntactic/sentence processing networks. But not only speech is a communicative source, features such as tone of voice, facial expression, body posture, and gestures also transmit meaning that is decoded and supports comprehension.                                            

  In healthy subjects the brain seems to not be distracted by the simultaneously incoming auditory (verbal) and visual (gestural) communicative information. Instead, it is even able to integrate speech and gesture information and takes advantage of both information channels.

  The research of the gesture-language-projects of the TNM-Lab (EEG-CON, Gestik3) is mainly concerned with the question of

  - how speech and gesture meaning interact during comprehension
  - how this interaction is dependent on the abstractness of the communicated information and
  - whether patients with schizophrenia demonstrate aberrant integration of speech and gesture

  We use behavioral rating studies, EEG and fMRI to shed some light on these issues.

  With regard to speech-gesture integration, we could already show that activity in the left posterior temporal lobe is related to integration of concrete speech and gesture information (iconic co-verbal gestures) (Green et al., 2009; Straube, Green, Bromberger, & Kircher, 2011a). In a different set of experiments we found that the processing of abstract speech and gesture information (metaphoric co-verbal gestures) leads to additional activity in the left inferior frontal gyrus (Kircher et al., 2009; Straube, Green, et al., 2011a).

  In patients with schizophrenia we found that the area in the left posterior superior/medial temporal gyrus was activated to the same extent as in healthy control subjects when iconic co-verbal gestures were contrasted to unimodal control conditions (speech-alone, gesture-alone) (Straube, Green, Sass, Kirner-Veselinovic, & Kircher, 2013). However, for the integration of metaphoric gestures we found aberrant processing in the patient group. For this type of co-verbal gestures it is important to build an abstract relation between concrete visual and abstract verbal information. For building this relation, additional online integration or unification processes in distantly located regions of the brain seem to be relevant (Kircher et al., 2009; Straube, Green, Bromberger, & Kircher, 2011b; Straube et al., 2013) and are likely to be disturbed in patients.

  Thus, finally we could show that patients demonstrate a specific functional disconnectivity of the left superior temporal sulcus and the inferior frontal gyrus bilaterally during the processing of metaphoric gestures (Straube, Green, Sass, & Kircher, 2013). This disconnectivity might be the basis of dysfunctional integration of gesture information into an abstract sentence context and thus an underlying reason for interpersonal communication problems in patients.

  Further research is concerned with contextual factors such as body orientation. Here it is of interest, how contextual factors influence the neural processing of speech and gesture information (Straube, Green, Chatterjee, & Kircher, 2011; Straube, Green, Jansen, Chatterjee, & Kircher, 2010; Nagels, Kircher, Steines, & Straube, 2015) and further interact with for example social in contrast to object related communication content (Straube et al., 2010).

  Finally, we are interested in episodic memory processes which contribute to speech and gesture integration and comprehension (Straube, Green, Chatterjee, et al., 2011; Straube, Green, Weis, Chatterjee, & Kircher, 2009; Straube, Meyer, Green, & Kircher, 2014).

  Actual research is focused on

  - temporal aspects of neural speech and gesture integration using EEG and fMRI (He et al., 2015)
  - functional connectivity during integration of auditory and visual information (Straube et al., 2018)
  - predictive mechanisms during comprehension
  - social aspects of speech and gesture processing and related dysfunctions in schizophrenia (He et al.,  
    2021) and Depression (Suffel et al., 2021)
  - gesture processing in a natural language context (Cuevas et al., 2019)

  Representative publications:

  Riedl, L., Nagels, A., Sammer, G., Choudhury, M., Nonnenmann, A., Sütterlin, A., … Straube, B. (2022). Multimodal speech-gesture training in patients with schizophrenia spectrum disorder : Effects on quality of life and neural processing. Schizophrenia Research, 246(April), 112–125. https://doi.org/10.1016/j.schres.2022.06.009

  Choudhury, M., Steines, M., Nagels, A., Riedl, L., Kircher, T., & Straube, B. (2021). Neural Basis of Speech-Gesture Mismatch Detection in Schizophrenia Spectrum Disorders. Schizophrenia Bulletin, 47(6), 1761–1771. https://doi.org/10.1093/schbul/sbab059

  Cuevas, P., He, Y., Billino, J., Kozasa, E., & Straube, B. (2021). Age-related effects on the neural processing of semantic complexity in a continuous narrative: Modulation by gestures already present in young to middle-aged adults. Neuropsychologia, 151(107725). https://doi.org/10.1016/J.NEUROPSYCHOLOGIA.2020.107725

  Cuevas, P.*, He, Y.*, Steines, M. & Straube, B.* (2022). The processing of semantic complexity and co-speech gestures in schizophrenia: a naturalistic, multimodal fMRI study. Schizophrenia Bulletin Open, Volume 3, Issue 1, January 2022 *contributed equally