Publications

The buttons with small symbols for open access , open data , preregistration , and open material provide direct links to the sources. Not all of these badges have been officially awarded by the publishing journal (many journals have not implemented the badges yet, and some discarded them again).

The cards to the right provide a Haiku Summary for each publication (an idea I took from John Helveston and Andrew Heiss).

Meta-data and citation counts are from OpenAlex.

Preprints

Evaluating Research Practices in Clinical Psychology with RESQUE: Development of a Clinical Expansion Pack
Fink‐Lamotte J, Heller C, Hilbert K, Jacobs N, Gärtner A & Schönbrodt F (2026). https://doi.org/10.31234/osf.io/6gwr8_v1

PDF Open Material

Researchers assess scores -
Numbers dance with careful hands -
Quality in the mix

×

Abstract

Background: The Research Quality Evaluation Scheme for Psychological Research (RESQUE) provides a structured scheme for assessing research quality. Objective: This study aims to assess the reliability and feasibility of RESQUE for evaluating transparency?and reproducibility-related practices within clinical psychology.Method: Fifty-five research articles, contributed by 20 early-career researchers, were evaluated for the presence and quality of Open Data, Open Materials, Preregistration, and Reproducible Code, and Methodological Rigor.Results: RESQUE showed good reliability when scores were aggregated, with excellent agreement for the overall score in Phase 2 (ICC(1,1)=.80). The Open Materials category required refinement. Compared with “top selected papers” from Personality and Social Psychology (PSP), clinical psychology showed higher preregistration availability (but lower preregistration quality), whereas PSP showed higher data and code availability.Conclusion: The findings support RESQUE as useful for research assessment in clinical psychology but indicate the need for field-sensitive refinements—particularly to better capture FAIR/restricted data sharing and to improve currently under-specified categories.

Indicators for the Assessment of Theoretical Rigor
Musfeld P, Freyer N, Lange J, Schönbrodt F, Dongen N, Peikert A, Schubert A, Geppert R, Borgstede M & Leising D (2026). https://doi.org/10.31234/osf.io/e7tdc_v3

PDF Open Material

Steeped standards rise -
Theory's gentle rigor -
Science's careful dance

×

Abstract

We provide a set of indicators for assessing theoretical rigor in psychological research. The indicators include criteria that we consider to be hygiene standards for the reporting and testing of theories in psychological research. In general, they are applicable to research that develops and/or evaluates a (part of a) theory. The indicators are part of the RESQUE framework of responsible research assessment (https://resque-framework.github.io/website/), and aim to incentivize theory-guided work in academic hiring and promotion. Similarly, they can also be used as step-by-step guidance to facilitate more precise theory building and reporting (https://doi.org/10.31219/osf.io/7qvfz).

On the responsible use of metrics and indicators in research assessment
Crepaldi D, Heyard R, Nawrot KA, Schönbrodt FD, Kldiashvili E, Leist A, Lozano D, Maniadis Z, Sipola T & Vindišar J (2026). MetaArXiv. https://doi.org/10.31222/osf.io/9cdns_v1

PDF

Trust in numbers falters -
Metrics guide with gentle care -
Wisdom in the mix

×

Abstract

There is growing awareness that research assessment in many countries and disciplines creates an incentive system that contributes to making science less reliable and academia more unfair. This jeopardizes the bond of trust between the community at large and the world of research and tertiary education, with dangerous and widespread societal implications. Much of the blame is attributed to the inappropriate use of quantitative indicators, which spurred several initiatives to limit their impact and/or encourage their more appropriate use. We propose a set of guidelines for using indicators responsibly. We argue that indicators should reliably measure a well-defined quality dimension (e.g., impact, innovation, methodological rigour); be transparent, reproducible and non-proprietary; be readily available and economically accessible; and be adapted appropriately according to the context of the assessment exercise (e.g., seniority, field of research, evaluation goals). We also argue that the use of indicators should always be integrated with a qualitative, in-depth evaluation, and that a multi-dimensional approach – where a multiplicity of indicators inform on a multiplicity of constructs – is fundamental in most assessment exercises. These principles are presented with a focus on practical implementation in realistic evaluation settings, and with a focus on their psychometric foundations.

Inter-Rater Reliability in Assessing the Methodological Quality of Research Papers in Psychology
Etzel F, Seyffert-Müller A, Schönbrodt F, Kreuzer L, Gärtner A, Knischewski P & Leising D (2024). https://doi.org/10.31234/osf.io/4w7rb

PDF Open Data Open Material Preregistration

Rating scores align deep -
Science's rigor weighed by many -
Quality's steady mark

×

Abstract

Given the apparent validity deficiencies of many well-established metrics of research productivity (such as h-indices and journal impact factors), the demand for viable alternatives is growing. This paper presents two empirical studies in which groups of raters (n1 = 3, n2 = 9) assessed the methodological rigor of research papers (k1 = 52, k2 = 110) using detailed catalogs of relatively well-defined quality criteria. The main endpoint in both studies was inter-rater reliability, which is a necessary prerequisite for any subsequent use of such assessments (e.g., as part of hiring or promotion procedures). Both studies showed that the application of several open science practices (e.g., open data, preregistration) may in fact be assessed with good reliability (Kappa > .60, ICCs > .75), even by raters who received little to no training, and within reasonable amounts of time (M1 = 21.7, M2 = 40.2 minutes on average). When aggregated across indicators, inter-rater reliability for this type of assessment was good to excellent (Study 1: ICC(1, 1) = .91, Study 2: ICC(1,1) = .74). A subsample of papers in Study 2 was drawn randomly from the recent literature (2020-2022) and showed that typical papers in contemporary psychology still espouse very low methodological rigor. Study 2 also showed that criteria related to consensus-building do not yet espouse sufficient reliability. Standard criterion sets for assessing the methodological rigor of empirical research should be used more widely in evaluating submissions to scientific journals, as well as published research (e.g., in evaluating the research productivity of individuals, groups or institutions). Such evaluations will be also facilitated by establishing clearer and more widely-adopted reporting standards.

The OpenLAV video database for affect induction: Analyzing the uniformity of video stimuli effects
Israel L, Paukner P, Schiestel L, Diepold K & Schönbrodt F (2021). https://doi.org/10.31234/osf.io/vhmbq

PDF

Varied emotions flow -
Video's subtle, shifting dance -
Feelings, a complex

×

Abstract

The Open Library for Affective Videos (OpenLAV) is a new video database for experimental emotion induction. The 188 videos (mean duration: 40 s; range: 12–71 s) have a CC-BY license. Ratings for valence, arousal, several appraisals, and emotion labels were assessed from 434 US-American participants in an online study (on average 70 ratings per video), along with personality traits from the raters (Big 5 personality dimensions and several motive dispositions). The OpenLAV is able to induce a large variety of different emotions, but the videos vary in uniformity of emotion induction. Based on different variability metrics, we recommend videos for the most uniform induction of different emotions. Moreover, the predictive power of personality traits on emotion ratings was analyzed using a machine-learning approach. In contrast to previous research, no effects of personality on the emotional experience were found.

Genug ist genug: Unethisches Verhalten in der Wissenschaft muss endlich systematisch angegangen werden!
Leising D, Dshemuchadse M, Schönbrodt FD & Scherbaum S (2021). Zenodo. https://doi.org/10.5281/zenodo.5126637

PDF

Ethics ignored long -
Whispers of misconduct spread -
Trust in science worn

×

Abstract

Es ist unübersehbar geworden: Unsere Universitäten und Forschungseinrichtungen haben Integritätsprobleme. Unsere Glaubwürdigkeit als Forschende leidet unter immer neuen Fällen von wissenschaftlichem Fehlverhalten, Machtmissbrauch, Mobbing, und Veruntreuung. Im Folgenden skizzieren wir, einer Bitte des DGPs-Vorstands folgend, drei aus unserer Sicht zentrale Ursachen für die Entstehung und Aufrechterhaltung solcher Probleme, und bieten Lösungsvorschläge an. 

Testing fit patterns with polynomial regression models
Schönbrodt F (2016). https://doi.org/10.31219/osf.io/ndggf

PDF

Pattern in the dust -
Polynomial curves reveal -
Congruent design

×

Abstract

Fit hypotheses, also labeled ‘congruence’, ‘discrepancy’, or ‘congruity’ hypotheses, contain the notion that an outcome is optimal when two predictor variables match well, while incongruent/discrepant combinations of the predictors lead to a suboptimal outcome. Previous statistical frameworks for analyzing fit hypotheses emphasized the necessity of commensurable scales, which means that both predictors must be measured on the same content dimension and on the same numerical scale. In some research areas, however, it is impossible to achieve scale equivalence, because the predictors have to be measured with different methods, such as explicit attitudes (e.g., questionnaires) and implicit attitudes (e.g., reaction time task). In this paper, I differentiate numerical congruence from fit patterns, a concept that does not depend on the notion of commensurability, and hence can be applied to fit hypotheses with incommensurable scales. Polynomial regression can be used to test for the presence of a fit pattern in empirical data. I propose several new regression models for testing fit patterns which are statistically simpler and conceptually more meaningful than a full polynomial model. An R package is introduced which provides user-friendly functions for the computation, visualization, and model comparison of several fit patterns. An empirical example on implicit/explicit motive fit demonstrates the usage of the new methods.

Awareness And Use Of Open Research Practices: An International Survey Of Researchers Across Disciplines
No authors indicated

PDF Open Data Open Material Preregistration

Data flows like a stream -
Researchers seek to reveal -
Truth in open light

×

Abstract

NA

Why using age as a proxy for testosterone is a bad deal
Schönbrodt FD (2012). OSF Preprints. https://osf.io/9tkwk/

PDF

Burning youthful fuel -
Age cannot measure the flame -
Hormones mislead

×

Abstract

In their article "Deal or No Deal: Hormones and the Mergers and Acquisitions Game," Levi, Li, and Zhang (2010) claimed that they investigated the effect of testosterone on CEOs’ decisions in mergers and acquisitions. However, they did not measure testosterone levels directly. Rather, they tried to use CEO age as a proxy, based on a previously documented negative correlation between age and testosterone level. In this comment, I argue that it is not reasonable to use age as a proxy for testosterone, and that Levi et al.’s study does not tell us anything about testosterone. General remarks on using proxy variables are given.

2026

To Be FAIR: Theory Specification Needs an Update
Lissa C, Peikert A, Ernst M, Dongen N, Schönbrodt F & Brandmaier A (2026). Perspectives on Psychological Science. https://doi.org/10.1177/17456916251401850

PDF Open Material

Citation count: 0

Structures of the mind -
Findable, accessible, shared -
Wisdom's gentle flow

×

Abstract

Open science innovations have focused on rigorous theory testing, yet methods for specifying, sharing, and iteratively improving theories remain underdeveloped. To address this limitation, we introduce FAIR theory, a standard for specifying theories as findable, accessible, interoperable, and reusable digital objects. FAIR theories are findable in well-established archives; accessible in terms of their availability and ability to be understood; interoperable for specific purposes, such as selecting control variables; and reusable in that they can be iteratively and collaboratively improved on. This article adapts the FAIR principles for theory; reflects on current FAIR practices in relation to psychological theory; and discusses FAIR theories' potential impact in terms of reducing research waste, enabling metaresearch on theories' structure and development, and incorporating theory into reproducible research workflows-from hypothesis generation to simulation studies. We present a conceptual workflow for FAIRifying theory that builds on existing open science principles and infrastructures. More detailed tutorials, worked examples, and convenience functions to automate this workflow are available in the theorytools R package. FAIR theory constitutes a structured protocol for archiving, communicating about, and iteratively improving theory, addressing a critical gap in open scholarly practices and potentially increasing the efficiency of cumulative knowledge acquisition in psychology and beyond.

A Conceptual Framework For Computational Reproductions: Formal Definitions And Epistemic Functions
Kohrt F, Melinscak F, McElreath R & Schönbrodt FD (2026). Journal of Trial & Error. https://osf.io/preprints/metaarxiv/7rnfq_v1

PDF Open Material

Citation count: NA

Echoes of results sought -
Cracks in computational walls -
Truth in consistent line

×

Abstract

Reproductions, which we define as “redoing activities” that work with the same collected data (or simulation source code) and aim to keep the computation as similar as possible, enable the verification of a scientific work. We seek to advance the understanding and conduct of reproductions by providing a conceptual framework and guidance to reproducers. In the first part, the epistemic function of reproductions and replications regarding mistakes – and faults as a consequence – is discussed. In the second part, the logic underlying reproductions is formulated, accompanied by formal definitions of central terms. The third and final part will provide practical guidance on conducting and reporting reproductions. In short, a reproduction involves performing another computation and comparing the results for consistency, that is, exploring the impact of computational choices on results before suspecting faults as a last resort. Computational choices by reproducers are necessitated by obstacles such as underspecification and grounds for conflicting choices. Regarding faults, four cases are emphasized: (1) incoherent descriptions, (2) different collected data (or simulation source code), (3) different data analysis source code, and (4) incorrect reporting of results. Separate from evaluating the consistency of results, reproductions can investigate their support for a claim. Finally, improper reporting practices can reduce the epistemic value of scientific works and their reproductions. Therefore, reproducers should also report the scope of their reproduction by detailing, among other things, which results were targeted, which computational stages were redone, and which descriptions were consulted.

2025

Responsible Research Assessment (Parts I and II): Responses to the Commentaries
Leising D, Gärtner A & Schönbrodt F (2025). Meta-Psychology. https://doi.org/10.15626/mp.2024.4603

PDF

Citation count: 0

Quality measures unfold -
Scores and weights, a balance -
Science's gentle dance

×

Abstract

We give a brief overview of our deliberations in responding to the commentaries on our two target papers presenting the RESQUE (Research Quality Evaluation) framework. While we were able to incorporate many suggestions for improvement directly, we acknowledge that other areas (e.g., quality in theorizing) will require further elaboration. In this paper, we specifically touch on the following issues: (a) eligible types of publication, (b) measurability, (c) quality criteria for software and datasets, (d) theoretical rigor, (e) quantity, (f) authorship, (g) potential bias (against certain methodologies, types of research contributions, or subdisciplines), (h) overall rigor score, (i) weighting of individual indicators, (j) types of data, (k) impact, (l) interdisciplinary value, (m) teaching, (n) expertise, (o) gaming the new metrics, and (p) representativeness. The RESQUE framework has met with largely positive reception so far, but continues to evolve and will thrive best when community involvement stays strong.

Responsible Research Assessment I: Implementing DORA and CoARA for hiring and promotion in psychology
Schönbrodt F, Gärtner A, Frank M, Gollwitzer M, Ihle M, Mischkowski D, Phan L, Schmitt M, Scheel A, Schubert A, Steinberg U & Leising D (2025). Meta-Psychology. https://doi.org/10.15626/mp.2024.4601

PDF

Citation count: 2

Metrics misplaced -
Broader measures sought quality -
Research sets free

×

Abstract

The use of journal impact factors and other metric indicators of research productivity, such as the h-index, has been heavily criticized for being invalid for the assessment of individual researchers and for fueling a detrimental "publish or perish" culture. Multiple initiatives call for developing alternatives to existing metrics that better reflect quality (instead of quantity) in research assessment. This report, written by a task force established by the German Psychological Society, proposes how responsible research assessment could be done in the field of psychology. We present four principles of responsible research assessment in hiring and promotion and suggest a two-phase assessment procedure that combines the objectivity and efficiency of indicators with a qualitative, discursive assessment of shortlisted candidates. The main aspects of our proposal are (a) to broaden the range of relevant research contributions to include published data sets and research software, along with research papers, and (b) to place greater emphasis on quality and methodological rigor in research evaluation.

Responsible Research Assessment II: A specific proposal for hiring and promotion in psychology
Gärtner A, Leising D, Freyer N, Musfeld P, Jens L & Schönbrodt F (2025). Meta-Psychology. https://doi.org/10.15626/mp.2024.4604

PDF Open Material

Citation count: 0

Metrics falter slow -
Research quality shines forth bright -
Science's gentle pace

×

Abstract

Traditional metric indicators of scientific productivity (e.g., journal impact factor; h-index) have been heavily criticized for being invalid and fueling a culture that focuses on the quantity, rather than the quality, of a person's scientific output. There is now a wide-spread demand for viable alternatives to current academic evaluation practices. In a previous report, we laid out four basic principles of a more responsible research assessment in academic hiring and promotion processes (Schönbrodt et al., 2025). The present paper offers a specific proposal for how these principles may be implemented in practice: We argue in favor of broadening the range of relevant research contributions and propose a set of concrete quality criteria (including a ready-to-use online tool) for research articles. These criteria are supposed to be used primarily in the first phase of the assessment process. Their function is to help establish a minimum threshold of methodological (i.e., theoretical and empirical) rigor that candidates need to pass in order to be further considered for hiring or promotion. In contrast, the second phase of the assessment process focuses more on the actual content of candidates' research and necessarily uses more narrative means of assessment. The debate over ways of replacing current invalid evaluation criteria with ones that relate more closely to scientific quality continues. Its course and outcome will depend on the willingness of researchers to get involved and help shape it.

A Checklist for Incentivizing and Facilitating Good Theory Building
Lange J, Freyer N, Musfeld P, Schönbrodt F & Leising D (2025). Zeitschrift für Psychologie. https://doi.org/10.1027/2151-2604/a000604

PDF Open Material

Citation count: 2

Clarity's gentle rule -
Theory's rigorous steps taken -
Mind's sharp, precise path

×

Abstract

Abstract: Psychology is facing a theory crisis characterized by ambiguous theories that lead to inconclusive theory tests. To address this problem, we propose a checklist designed to establish hygiene standards for assessing theoretical rigor in psychological research. This checklist contains operationalized criteria applicable to the development of both verbal and formal theories as well as tests of these. If the checklist is used as part of evaluation schemes (e.g., in hiring), it can further incentivize theory building, and if it is used as step-by-step guidance for how to proceed during theory building, it may also facilitate better theorizing. Developing and testing precise theories in psychology is more challenging than is typically acknowledged. Given this, we advocate for the (re-)creation of a dedicated field of theoretical psychology and specialized journals that support the publication of well-developed theories even without tests.

Interpersonal versus intrapersonal emotion regulation: Intensity of negative emotion predicts usage probability.
Claus N, Flechsenhar A, Motka F, Sckopke P, Schönbrodt F, Stefan A, Werner G, Cludius B & Zygar–Hoffmann C (2025). Emotion. https://doi.org/10.1037/emo0001508

PDF Open Material Preregistration

Citation count: 1

Turbulent heart seeks calm -
In another's gentle voice -
Shared sorrow's solace

×

Abstract

Emotional intensity seems to be related to the selection of emotion regulation (ER) strategies. Previous studies have focused primarily on intrapersonal ER. In contrast, everyday use of interpersonal ER remains understudied, with the association between perceived emotional intensity and selection of intra- or intrinsic interpersonal ER still unclear. In a preregistered study, we used experience sampling method data from a community sample of individuals in a heterosexual relationship (n = 183) who provided reports on their daily lives over a period of 4 weeks during the first COVID-19 lockdown in 2020. Surveys included questions about various emotions, their perceived intensity, and selected ER strategies. Results revealed more frequent use of ER overall with increasing emotional intensity of all negative emotions (except for boredom). In line with our hypothesis, results showed that greater emotional intensity increased the probability of interpersonal ER more strongly than intrapersonal ER for most negative emotions, in both women and men. Additionally, we found that men used significantly more intrapersonal than interpersonal strategies at average emotional intensity of all negative emotions. A preference for interpersonal ER in the face of highly intense emotions may be related to a higher demand for resources, which is compensated through social connection. In the association between perceived emotional intensity and ER selection, our research highlights the importance of considering both intra- and interpersonal ER strategies. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Improving the probability of reaching correct conclusions about congruence hypotheses: Integrating statistical equivalence testing into response surface analysis.
Humberg S, Schönbrodt F & Nestler S (2025). Psychological Methods. https://doi.org/10.1037/met0000743

PDF Open Material

Citation count: 1

Meaning meets the line -
Equivalence tested with care -
Truth's probability rise

×

Abstract

Many psychological theories imply that the degree of congruence between two variables (e.g., self-rated and objectively measured intelligence) is related to some psychological outcome (e.g., life satisfaction). Such congruence hypotheses can be tested with response surface analysis (RSA), in which a second-order polynomial regression model is estimated and suitably interpreted. Whereas several strategies exist for this interpretation, they all contain rationales that diminish the probability of drawing correct conclusions. For example, a frequently applied strategy involves calculating six auxiliary parameters from the estimated regression weights and accepting the congruence hypothesis if they satisfy certain conditions. In testing the conditions, a nonsignificant null-hypothesis test of some parameters is taken as evidence that the parameter is zero. This interpretation is formally inadmissible and adversely affects the probability of making correct decisions about the congruence hypothesis. We address this limitation of the six-parameter strategy and other RSA strategies by proposing that statistical equivalence testing (SET) be integrated into RSA. We compare the existing and new RSA strategies with a simulation study and find that the SET strategies are sensible alternatives to the existing strategies. We provide code templates for implementing the SET strategies. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Practicing Theory Building in a Many Modelers Hackathon
Dongen N, Finnemann A, Ron J, Tiokhin L, Wang S, Algermissen J, Altmann E, Bahník Š, Chuang L, Dumbravă A, Fuenderich J, Geiger S, Gerasimova D, Golan A, Herbers J, Jekel M, Kunnari A, Lin Y, Moreau D, Oberholzer Y, Peetz H, Rohrer J, Rothers A, Schönbrodt F, Seetahul Y, Szabelska A, Tonge N, Walasek N, Werner M & Borsboom D (2025). Meta-Psychology. https://doi.org/10.15626/mp.2023.3688

PDF Open Material

Citation count: 0

Math and minds combine -
Theories take shape, anew born -
Science's gentle dance

×

Abstract

Scientific theories reflect some of humanity's greatest epistemic achievements. The best theories motivate us to search for discoveries, guide us towards successful interventions, and help us to explain and organize knowledge. Such theories require a high degree of specificity, which in turn requires formal modeling. Yet, in psychological science, many theories are not precise and psychological scientists often lack the technical skills to formally specify existing theories. This problem raises the question: How can we promote formal theory development in psychology, where there are many content experts but few modelers? In this paper, we discuss one strategy for addressing this issue: a Many Modelers approach. Many Modelers consists of mixed teams of modelers and non-modelers that collaborate to create a formal theory of a phenomenon. Here, we report a proof of concept of this approach, which we piloted as a three-hour hackathon at the Society for the Improvement of Psychological Science conference in 2021. After surveying the participants, results suggest that (a) psychologists who have never developed a formal model can become (more) excited about formal modeling + and theorizing; (b) a division of labor in formal theorizing is possible where only one or a few team members possess the prerequisite modeling expertise; and (c) first working prototypes of a theoretical model can be created in a short period of time. These results show some promise for the many modelers approach as a team science tool for theory development.

Automatically Coding Implicit Motives in Picture Story Exercises: The Automated Motive Coder
Brede M, Schönbrodt FD, Hagemeyer B & Lerche V (2025). https://doi.org/10.36190/2025.31

PDF Open Material

Citation count: NA

Codes in silent scenes -
Motive whispers revealed now -
Story's hidden truth

×

Abstract

The Picture Story Exercise (PSE) is a projective measure in personality psychology where individuals create narratives based on ambiguous images. Traditionally, the coding of these narratives has been labor-intensive. We introduce the Automated Motive Coder (AMC), which employs recent advances in natural language processing and machine learning to automate the coding of PSE narratives. Trained on an extensive dataset, the AMC demonstrates accuracy comparable to expert coders for both original and translated texts. The model offers support for multiple languages that were absent in prior methods while improving in accuracy and speed. To illustrate its effectiveness, we tested and successfully replicated the established psychological effect of gender difference in the affiliation motive. The AMC can be utilized through established machine learning tools, offering a pragmatic and reliable method for coding across several languages. This tool provides an option to reduce the workload involved in PSE coding, promoting efficiency and consistency in motive assessment.

2024

The reflection of motive dispositions in everyday agentic motivational states.
Zygar–Hoffmann C, Hagemeyer B & Schönbrodt F (2024). Motivation Science. https://doi.org/10.1037/mot0000344

PDF Open Data Open Material Preregistration

Citation count: 1

Hidden desires shine -
Daily drives and desires -
Freedom's subtle spark

×

Abstract

Individuals vary in their dispositional motives, which are expected to manifest in a different intensity of motivational states experienced on a momentary level.In two preregistered experience sampling studies (N = 130, N = 508) involving partners of intimate relationships, we investigated whether implicit and explicit agency motives were associated with average levels of experienced motivational states in everyday life.Whereas Study 1 did not show the predicted associations, it prompted a closer examination of the power and independence components of agency.Study 2 confirmed absolute and incremental effects of implicit and explicit motives in predicting motivational states, when outcomes were assessed on the broad level of agency, and on the specific level of independence.For the specific level of power, explicit but not implicit motives predicted average levels of motivational states.We conclude that self-reported agentic motivational states may reflect both implicit and explicit motives and that a differentiation of agency may be warranted for certain research questions.

Towards responsible research assessment: How to reward research quality
Gärtner A, Leising D & Schönbrodt F (2024). PLoS Biology. https://doi.org/10.1371/journal.pbio.3002553

PDF

Citation count: 28

Merit not the numbers -
Quality's gentle whisper -
Incentivize truth

×

Abstract

Researchers would be more willing to prioritize research quality over quantity if the incentive structure of the academic system aligned with this goal. The winner of a 2023 Einstein Foundation Award for Promoting Quality in Research explains how they rose to this challenge.

2023

Replacing academic journals
Brembs B, Huneman P, Schönbrodt F, Nilsonne G, Susi T, Siems R, Perakakis P, Trachana V, Ma L & Rodríguez‐Cuadrado S (2023). Royal Society Open Science. https://doi.org/10.1098/rsos.230206

PDF

Citation count: 37

Scholarly nets unfurl -
Monopolies broken free -
Knowledge flows wild free

×

Abstract

Replacing traditional journals with a more modern solution is not a new idea. Here, we propose ways to overcome the social dilemma underlying the decades of inaction. Any solution needs to not only resolve the current problems but also be capable of preventing takeover by corporations: it needs to replace traditional journals with a decentralized, resilient, evolvable network that is interconnected by open standards and open-source norms under the governance of the scholarly community. It needs to replace the monopolies connected to journals with a genuine, functioning and well-regulated market. In this new market, substitutable service providers compete and innovate according to the conditions of the scholarly community, avoiding sustained vendor lock-in. Therefore, a standards body needs to form under the governance of the scholarly community to allow the development of open scholarly infrastructures servicing the entire research workflow. We propose a redirection of money from legacy publishers to the new network by funding bodies broadening their minimal infrastructure requirements at recipient institutions to include modern infrastructure components replacing and complementing journal functionalities. Such updated eligibility criteria by funding agencies would help realign the financial incentives for recipient institutions with public and scholarly interest.

Empfehlungen zur Bewertung wissenschaftlicher Leistungen bei Berufungsverfahren in der Psychologie
Gärtner A, Leising D & Schönbrodt F (2023). Psychologische Rundschau. https://doi.org/10.1026/0033-3042/a000630

PDF

Citation count: 10

Metrics meet the heart -
Quality over quantity's call -
Science's gentle art

×

Abstract

Zusammenfassung: Traditionelle metrische Indikatoren für wissenschaftliche Produktivität (z. B. Impact-Faktoren von Zeitschriften, h-Index) werden kritisiert, weil (a) empirische Studien ihre Validität in Frage stellen und sie (b) eine Kultur zu fördern scheinen, die der reinen Quantität Vorrang gegenüber der inhaltlichen Qualität der Forschung gibt. Der Ruf nach spezifischen, besseren Alternativen zu den derzeit gängigen Leistungsindikatoren wird immer lauter. Die Deutsche Gesellschaft für Psychologie hat nach der Unterzeichnung von DORA und dem Beitritt zu CoARA eine Arbeitsgruppe mit der Frage befasst, wie eine verantwortungsvollere Form der Forschungsbewertung für die Psychologie aussehen könnte. Über den aktuellen Stand dieser Überlegungen berichten wir hier: Konkret schlagen wir ein zweistufiges Bewertungsverfahren vor, das die Objektivität und Effizienz metrischer Indikatoren (Phase 1) mit einer vertieften, diskursiven Bewertung spezifischer Forschungsinhalte verbindet (Phase 2) verbindet. Wir plädieren für eine Ausweitung des Spektrums relevanter Forschungsbeiträge und schlagen Qualitätskriterien für Forschungsartikel, Datensätze und Forschungssoftware vor, welche vor allem deren methodische Stringenz erfassen. Anhand dieser Kriterien wird in der ersten Phase des Bewertungsprozesses ermittelt, ob eine Mindestschwelle für methodische Stringenz erreicht wird, die Kandidat:innen erfüllen müssen, um für eine Einstellung oder Beförderung in Frage zu kommen. Im Gegensatz dazu sollen in der zweiten Phase des Beurteilungsprozesses Metriken keine Rolle mehr spielen, sondern die spezifischen Inhalte der Forschungsarbeiten der Kandidat:innen im Fokus einer vertieften Diskussion stehen. Wir stellen konkrete Erhebungsinstrumente bereit, die zum Teil bereits in der Praxis erprobt wurden. Unser Ziel ist es, eine breite Diskussion über das Thema anzuregen und letztlich dazu beizutragen, dass die Bedeutung der wissenschaftlichen Qualität im Rahmen von Berufungsverfahren spürbar gestärkt wird.

Comparing the vibration of effects due to model, data pre-processing and sampling uncertainty on a large data set in personality psychology
Klau S, Felix F, Patel C, Ioannidis J, Boulesteix A & Hoffmann S (2023). Meta-Psychology. https://doi.org/10.15626/mp.2020.2556

PDF Open Data Open Material

Citation count: 12

Flexibility's dark face -
Uncertainty's subtle sway -
Truth's delicate dance

×

Abstract

Researchers have great flexibility in the analysis of observational data. If combined with selective reporting and pressure to publish, this flexibility can have devastating consequences on the validity of research findings. We extend the recently proposed vibration of effects approach to provide a framework comparing three main sources of uncertainty which lead to instability in empirical findings, namely data pre-processing, model, and sampling uncertainty. We analyze the behavior of these sources for varying sample sizes for two associations in personality psychology. Through the joint investigation of model and data pre-processing vibration, we can compare the relative impact of these two types of uncertainty and identify the most influential analytical choices. While all types of vibration show a decrease for increasing sample sizes, data pre-processing and model vibration remain non-negligible, even for a sample of over 80000 participants. The increasing availability of large data sets that are not initially recorded for research purposes can make data pre-processing and model choices very influential. We therefore recommend the framework as a tool for transparent reporting of the stability of research findings.

Machine Learning and Risk Assessment: Random Forest Does Not Outperform Logistic Regression in the Prediction of Sexual Recidivism
Etzler S, Schönbrodt F, Pargent F, Eher R & Rettenberger M (2023). Assessment. https://doi.org/10.1177/10731911231164624

PDF Open Material

Citation count: 10

Patterns missed by chance -
Predictions made with certainty -
Regression's gentle

×

Abstract

Although many studies supported the use of actuarial risk assessment instruments (ARAIs) because they outperformed unstructured judgments, it remains an ongoing challenge to seek potentials for improvement of their predictive performance. Machine learning (ML) algorithms, like random forests, are able to detect patterns in data useful for prediction purposes without explicitly programming them (e.g., by considering nonlinear effects between risk factors and the criterion). Therefore, the current study aims to compare conventional logistic regression analyses with the random forest algorithm on a sample of N = 511 adult male individuals convicted of sexual offenses. Data were collected at the Federal Evaluation Center for Violent and Sexual Offenders in Austria within a prospective-longitudinal research design and participants were followed-up for an average of M = 8.2 years. The Static-99, containing static risk factors, and the Stable-2007, containing stable dynamic risk factors, were included as predictors. The results demonstrated no superior predictive performance of the random forest compared with logistic regression; furthermore, methods of interpretable ML did not point to any robust nonlinear effects. Altogether, results supported the statistical use of logistic regression for the development and clinical application of ARAIs.

Quality research needs good working conditions
Rahal R, Fiedler S, Adetula A, Berntsson R, Dirnagl U, Feld G, Fiebach C, Himi S, Horner A, Lonsdorf T, Schönbrodt F, Silan M, Wenzler M & Azevedo F (2023). Nature Human Behaviour. https://doi.org/10.1038/s41562-022-01508-2

PDF

Citation count: 41

Clean labs, happy mind -
Science blooms in calm spaces -
Knowledge's gentle birth

×

Abstract

NA

Big little lies: a compendium and simulation of p -hacking strategies
Stefan A & Schönbrodt F (2023). Royal Society Open Science. https://doi.org/10.1098/rsos.220346

PDF Open Material

Citation count: 101

P-hacked results spin -
Deception in data's veil -
Truth in numbers lies

×

Abstract

In many research fields, the widespread use of questionable research practices has jeopardized the credibility of scientific results. One of the most prominent questionable research practices is p-hacking. Typically, p-hacking is defined as a compound of strategies targeted at rendering non-significant hypothesis testing results significant. However, a comprehensive overview of these p-hacking strategies is missing, and current meta-scientific research often ignores the heterogeneity of strategies. Here, we compile a list of 12 p-hacking strategies based on an extensive literature review, identify factors that control their level of severity, and demonstrate their impact on false-positive rates using simulation studies. We also use our simulation results to evaluate several approaches that have been proposed to mitigate the influence of questionable research practices. Our results show that investigating p-hacking at the level of strategies can provide a better understanding of the process of p-hacking, as well as a broader basis for developing effective countermeasures. By making our analyses available through a Shiny app and R package, we facilitate future meta-scientific research aimed at investigating the ramifications of p-hacking across multiple strategies, and we hope to start a broader discussion about different manifestations of p-hacking in practice.

Replication of the natural selection of bad science
Kohrt F, Smaldino P, McElreath R & Schönbrodt F (2023). Royal Society Open Science. https://doi.org/10.1098/rsos.221306

PDF Open Material

Citation count: 8

Selective flaws arise -
Brief exuberance of zeal -
Science's quiet test

×

Abstract

This study reports an independent replication of the findings presented by Smaldino and McElreath (Smaldino, McElreath 2016 R. Soc. Open Sci. 3, 160384 (doi:10.1098/rsos.160384)). The replication was successful with one exception. We find that selection acting on scientist's propensity for replication frequency caused a brief period of exuberant replication not observed in the original paper due to a coding error. This difference does not, however, change the authors' original conclusions. We call for more replication studies for simulations as unique contributions to scientific quality assurance.

2022

Proximal and distal honor fit and subjective well‐being in the Mediterranean region
Kirchner‐Häusler A, Schönbrodt F, Üskül A, Vignoles V, Rodríguez‐Bailón R, Castillo V, Cross S, Yalçın M, Harb C, Husnu S, Ishii K, Karamaouna P, Kafetsios K, Kateri E, Matamoros‐Lima J, Miniesy R, Na J, Özkan Z, Pagliaro S, Psaltis C, Rabie D, Teresi M & Uchida Y (2022). Journal of Personality. https://doi.org/10.1111/jopy.12803

PDF Open Data Open Material

Citation count: 11

Village pride unites -
Shadows of dignity fade -
Golden light remains

×

Abstract

Our findings highlight the interplay between individual and environmental factors in honor as well as the important role honor plays in well-being in the Mediterranean region.

Demonstration of a ‘leapfrog’ randomized controlled trial as a method to accelerate the development and optimization of psychological interventions
Blackwell S, Schönbrodt F, Woud M, Wannemüller A, Bektas B, Rodrigues M, Hirdes J, Stumpp M & Margraf J (2022). Psychological Medicine. https://doi.org/10.1017/s0033291722003294

PDF Open Data Open Material Preregistration

Citation count: 16

Trials leap and bound -
Healing steps in sync, swift pace -
Minds in faster line

×

Abstract

The study demonstrates feasibility of the leapfrog design and provides a foundation for its adoption as a method to accelerate treatment development in mental health. Registration: clinicaltrials.gov, NCT04791137.

The true role that suppressor effects play in condition-based regression analysis: None. A reply to Fiedler (2021).
Humberg S, Dufner M, Schönbrodt F, Geukes K, Hutteman R, Zalk M, Denissen J, Nestler S & Back M (2022). Journal of Personality and Social Psychology. https://doi.org/10.1037/pspp0000428

PDF Open Material

Citation count: 1

Debunking Fiedler -
Suppressor effects fall silent -
Truth in numbers shines

×

Abstract

Condition-based regression analysis (CRA) is a statistical method for testing self-enhancement effects. That is, CRA indicates whether, in a set of empirical data, people with higher values on the directed discrepancy self-view S minus reality criterion R (i.e., S-R) tend to have higher values on some outcome variable (e.g., happiness). In a critical comment, Fiedler (2021) claims that CRA yields inaccurate conclusions in data with a suppressor effect. Here, we show that Fiedler's critique is unwarranted. All data that are simulated in his comment show a positive association between S-R and H, which is accurately detected by CRA. By construction, CRA indicates an association between S-R and H only when it is present in the data. In contrast to Fiedler's claim, it also yields valid conclusions when the outcome variable is related only to the self-view or when there is a suppressor effect. Our clarifications provide guidance for evaluating Fiedler's comment, clear up with the common heuristic that suppressor effects are always problematic, and assist readers in fully understanding CRA. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Open Data Statement GRN
Toelch U & Schönbrodt FD (2022). Zenodo. https://zenodo.org/records/6840403

PDF

Citation count: 0

Reproducibility -
German openness abounds -
Data freely shared

×

Abstract

Statement of the German Reproducibility on Open Data (in German)

The predictive power of insomnia symptoms on other aspects of mental health during the COVID‐19 pandemic: a longitudinal study
Werner G, Cludius B, Sckopke P, Stefan A, Schönbrodt F & Zygar–Hoffmann C (2022). Journal of Sleep Research. https://doi.org/10.1111/jsr.13641

PDF Open Material Preregistration

Citation count: 6

Stirring sleeplessness -
Predicts a troubled mind's fate -
Yet calm in the storm

×

Abstract

Symptoms of insomnia are an important risk factor for the development of mental disorders, especially during stressful life periods such as the coronavirus disease 2019 (COVID-19) pandemic. However, up to now, most studies have used cross-sectional data, and the prolonged impact of insomnia symptoms during the pandemic on later mental health remains unclear. Therefore, we investigated insomnia symptoms as a predictor of other aspects of mental health across 6 months, with altogether seven assessments (every 30 days, t0-t6), in a community sample (N = 166-267). Results showed no mean-level increase of insomnia symptoms and/or deterioration of mental health between baseline assessment (t0) and the 6- month follow-up (t6). As preregistered, higher insomnia symptoms (between persons) across all time points predicted reduced mental health at the 6-month follow-up. Interestingly, contrary to our hypothesis, higher insomnia symptoms at 1 month, within each person (i.e., compared to that person's symptoms at other time points), predicted improved rather than reduced aspects of mental health 1 month later. Hence, we replicated the predictive effect of averagely increased insomnia symptoms on impaired later mental health during the COVID-19 pandemic. However, we were surprised that increased insomnia symptoms at 1 month predicted aspects of improved mental health 1 month later. This unexpected effect might be specific for our study population and a consequence of our study design. Overall, increased insomnia symptoms may have served as a signal to engage in, and successfully implement, targeted countermeasures, which led to better short-term mental health in this healthy sample.

Motivational Interdependence in Couple Relationships
Pusch S, Schönbrodt F, Zygar–Hoffmann C & Hagemeyer B (2022). Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2022.827746

PDF Open Data Open Material

Citation count: 6

Motive dance unfolds -
Compassion's subtle ripple -
Love's gentle resonance

×

Abstract

This article presents an integrative conceptual model of motivational interdependence in couples, the MIC model. Based on theoretical tenets in motivation psychology, personality psychology, and research on interpersonal perception, the MIC model postulates that two partners' motive dispositions fundamentally interact in shaping their individual motivation and behavior. On a functional level, a partner's motivated behavior is conceptualized as an environmental cue that can contribute to an actor's motive expression and satisfaction. However, the partner's motivated behavior is considered to gain this motivational relevance only via the actor's subjective perception. Multilevel analyses of an extensive experience sampling study on partner-related communal motivation (N = up to 60,803 surveys from 508 individuals nested in 258 couples) supported the MIC model. Participants, particularly those with strong communal motive dispositions, behaved more communally at moments when they perceived their partners to behave more communally. In addition, participants experienced momentary boosts in satisfaction when they behaved more communally and, at the same time, perceived their partners' behavior as similarly communal. Broader implications of the MIC model for research on romantic relationships are discussed.

Ten steps toward a better personality science – how quality may be rewarded more in research evaluation
Leising D, Thielmann I, Glöckner A, Gärtner A & Schönbrodt F (2022). Personality Science. https://doi.org/10.5964/ps.6029

PDF

Citation count: 50

Measure worth not yields -
Quality's quiet rewards shine -
Science's gentle art

×

Abstract

This target article is part of a theme bundle including open peer commentaries (https://doi.org/10.5964/ps.9227) and a rejoinder by the authors (https://doi.org/10.5964/ps.7961). We point out ten steps that we think will go a long way in improving personality science. The first five steps focus on fostering consensus regarding (1) research goals, (2) terminology, (3) measurement practices, (4) data handling, and (5) the current state of theory and evidence. The other five steps focus on improving the credibility of empirical research, through (6) formal modelling, (7) mandatory pre-registration for confirmatory claims, (8) replication as a routine practice, (9) planning for informative studies (e.g., in terms of statistical power), and (10) making data, analysis scripts, and materials openly available. The current, quantity-based incentive structure in academia clearly stands in the way of implementing many of these practices, resulting in a research literature with sometimes questionable utility and/or integrity. As a solution, we propose a more quality-based reward scheme that explicitly weights published research by its Good Science merits. Scientists need to be increasingly rewarded for doing good work, not just lots of work.

Ten steps toward a better personality science – a rejoinder to the comments
Leising D, Thielmann I, Glöckner A, Gärtner A & Schönbrodt F (2022). Personality Science. https://doi.org/10.5964/ps.7961

PDF

Citation count: 15

Consortium calls forth -
Diverse voices seek consensus -
Science's gentle art

×

Abstract

We respond to the comments (https://doi.org/10.5964/ps.9227) on our “Ten Steps” paper (https://doi.org/10.5964/ps.6029), focusing on the most prominent themes: (1) What motivates scientists?, (2) Consensus-building (Is our field ready? May there be adverse side-effects? How shall we do it?), (3) How may institutional change be facilitated?, (4) Diversity (of participants, stimuli, methodology, measures, and among researchers), (5) The reliability of our proposed scoring system, and (6) The real-world relevance of personality research. We stand by our call for more concerted consensus-building and offer a few clarifications in this regard. We also issue four specific calls to action to our colleagues in the field: (a) specify legitimate paths to greater consensus, (b) explicate what constitutes good “qualitative” research, (c) help establish a widely used, public domain item database, and (d) determine what the most important contemporary goals of personality research are.

Gute wissenschaftliche Praxis „hands-on“
Schönbrodt F, Zygar–Hoffmann C, Frank M & Gollwitzer M (2022). Psychologische Rundschau. https://doi.org/10.1026/0033-3042/a000590

PDF Open Material

Citation count: 3

Good practice hands on -
Science in the maker space -
Learning in the act

×

Abstract

NA

A review of applications of the Bayes factor in psychological research.
Heck D, Boehm U, Böing-Messing F, Bürkner P, Derks K, Dienes Z, Fu Q, Gu X, Karimova D, Kiers H, Klugkist I, Kuiper R, Lee M, Leenders R, Leplaa H, Linde M, Ly A, Meijerink-Bosman M, Moerbeek M, Mulder J, Pálfi B, Schönbrodt F, Tendeiro J, Bergh D, Lissa C, Ravenzwaaij D, Vanpaemel W, Wagenmakers E, Williams D, Zondervan‐Zwijnenburg M & Hoijtink H (2022). Psychological Methods. https://doi.org/10.1037/met0000454

PDF Open Data Open Material

Citation count: 80

Calculating the weight -
Hypotheses rise and fall -
Probability's gentle

×

Abstract

The last 25 years have shown a steady increase in attention for the Bayes factor as a tool for hypothesis evaluation and model selection. The present review highlights the potential of the Bayes factor in psychological research. We discuss six types of applications: Bayesian evaluation of point null, interval, and informative hypotheses, Bayesian evidence synthesis, Bayesian variable selection and model averaging, and Bayesian evaluation of cognitive models. We elaborate what each application entails, give illustrative examples, and provide an overview of key references and software with links to other applications. The article is concluded with a discussion of the opportunities and pitfalls of Bayes factor applications and a sketch of corresponding future research lines. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Efficiency in sequential testing: Comparing the sequential probability ratio test and the sequential Bayes factor test
Stefan A, Schönbrodt F, Evans N & Wagenmakers E (2022). Behavior Research Methods. https://doi.org/10.3758/s13428-021-01754-8

PDF Open Material

Citation count: 12

Models unfold slowly -
Data's subtle whispers rise -
Efficiency's dance

×

Abstract

In a sequential hypothesis test, the analyst checks at multiple steps during data collection whether sufficient evidence has accrued to make a decision about the tested hypotheses. As soon as sufficient information has been obtained, data collection is terminated. Here, we compare two sequential hypothesis testing procedures that have recently been proposed for use in psychological research: Sequential Probability Ratio Test (SPRT; Psychological Methods, 25(2), 206-226, 2020) and the Sequential Bayes Factor Test (SBFT; Psychological Methods, 22(2), 322-339, 2017). We show that although the two methods have different philosophical roots, they share many similarities and can even be mathematically regarded as two instances of an overarching hypothesis testing framework. We demonstrate that the two methods use the same mechanisms for evidence monitoring and error control, and that differences in efficiency between the methods depend on the exact specification of the statistical models involved, as well as on the population truth. Our simulations indicate that when deciding on a sequential design within a unified sequential testing framework, researchers need to balance the needs of test efficiency, robustness against model misspecification, and appropriate uncertainty quantification. We provide guidance for navigating these design decisions based on individual preferences and simulation-based design analyses.

Eliciting Short-Term Closeness in Couple Relationships With Ecological Momentary Interventions
Zygar–Hoffmann C, Cristoforo L, Wolf L & Schönbrodt F (2022). Collabra Psychology. https://doi.org/10.1525/collabra.38599

PDF Open Data Open Material Preregistration

Citation count: 1

Eyes locked, love shared deep -
Closeness blooms with gentle touch -
Love's fleeting moments

×

Abstract

Relationship closeness is considered an important psychological variable for studying couple relationships, and is often postulated as cause for important relationship outcomes. The current study evaluates four micro-interventions for their suitability to experimentally elicit feelings of closeness towards one’s partner. Using participants’ smartphones, and a combination of experience sampling, event sampling, and ecological momentary interventions, individuals reported for a week on their experiences of closeness before and after completing daily either a neutral task or a task meant to enhance relationship closeness. The closeness tasks included showing physical affection, sharing a childhood memory, looking each other in the eyes for five minutes, and discussing shared life achievements. Results of intention-to-treat analyses on a within-person level showed that closeness increased from pre- to post-measurement on average more strongly on days of any of the four examined closeness conditions than on days of the neutral control conditions. Interindividual variability of this effect was observed, emphasizing the relevance of using within-person designs to evaluate such interventions. Exploratory analyses showed that effect sizes declined across time within the day. This study provides instruments for research on causal effects of closeness in everyday relationship life, and an evidence basis for smartphone-delivered interventions in practitioner settings.

2021

Measuring motivational relationship processes in experience sampling: A reliability model for moments, days, and persons nested in couples
Schönbrodt F, Zygar–Hoffmann C, Nestler S, Pusch S & Hagemeyer B (2021). Behavior Research Methods. https://doi.org/10.3758/s13428-021-01701-7

PDF Open Material

Citation count: 19

Moral moments fall -
Couple's daily harmony -
Reliability dance

×

Abstract

NA

Replicability, Robustness, and Reproducibility in Psychological Science
Nosek B, Hardwicke T, Moshontz H, Allard A, Corker K, Dreber A, Fidler F, Hilgard J, Struhl M, Nuijten M, Rohrer J, Romero F, Scheel A, Scherer L, Schönbrodt F & Vazire S (2021). Annual Review of Psychology. https://doi.org/10.1146/annurev-psych-020821-114157

PDF Open Data Open Material Preregistration

Citation count: 694

Shadows cast on truths -
Replication's gentle grasp -
Knowledge's gentle spark

×

Abstract

Replication-an important, uncommon, and misunderstood practice-is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.

Open Library for Affective Videos (OpenLAV)
Israel L, Paukner P, Schiestel L, Diepold K & Schönbrodt F (2021). Psychology Archives. https://doi.org/10.23668/psycharchives.5042

PDF Open Data Open Material

Citation count: 3

Viral emotions born -
Ratings cascade like a tide -
Affect's subtle dance

×

Abstract

The Open Library for Affective Videos (OpenLAV) is a video database created for experimental emotion induction. The 188 videos of the database have a CC-BY license and were tested in a crowdsourcing study on Amazon MTurk. Valence/arousal ratings, several appraisal ratings, and emotion labels for the videos were assessed from 422 US-American participants, with an average of 71 ratings per video. Furthermore, multiple personality traits from the raters were assessed.

The multiplicity of analysis strategies jeopardizes replicability: lessons learned across disciplines
Hoffmann S, Schönbrodt F, Elsas R, Wilson R, Strasser U & Boulesteix A (2021). Royal Society Open Science. https://doi.org/10.1098/rsos.201925

PDF

Citation count: 56

Multiplicity reigns -
Uncertainty in the code -
Truth in variance lies

×

Abstract

For a given research question, there are usually a large variety of possible analysis strategies acceptable according to the scientific standards of the field, and there are concerns that this multiplicity of analysis strategies plays an important role in the non-replicability of research findings. Here, we define a general framework on common sources of uncertainty arising in computational analyses that lead to this multiplicity, and apply this framework within an overview of approaches proposed across disciplines to address the issue. Armed with this framework, and a set of recommendations derived therefrom, researchers will be able to recognize strategies applicable to their field and use them to generate findings more likely to be replicated in future studies, ultimately improving the credibility of the scientific process.

2020

Perception of communal motives in couples: Accuracy, bias, and their associations with relationship length
Pusch S, Schönbrodt F, Zygar–Hoffmann C & Hagemeyer B (2020). Journal of Research in Personality. https://doi.org/10.1016/j.jrp.2020.104060

PDF Open Material

Citation count: 13

Partners see each whole -
Bias in love's gentle deceit -
Length in tender grasp

×

Abstract

NA

Validity of content‐based techniques for credibility assessment—How telling is an extended meta‐analysis taking research bias into account?
Oberlader V, Quinten L, Banse R, Volbert R, Schmidt A & Schönbrodt F (2020). Applied Cognitive Psychology. https://doi.org/10.1002/acp.3776

PDF

Citation count: 37

Shadows of deceit dance -
Bias creeps in, truths lost in haze -
Truth's fragile estimate

×

Abstract

Summary Content‐based techniques for credibility assessment (Criteria‐Based Content Analysis [CBCA], Reality Monitoring [RM]) have been shown to distinguish between experience‐based and fabricated statements in previous meta‐analyses. New simulations raised the question whether these results are reliable revealing that using meta‐analytic methods on biased datasets lead to false‐positive rates of up to 100%. By assessing the performance of and applying different bias‐correcting meta‐analytic methods on a set of 71 studies we aimed for more precise effect size estimates. According to the sole bias‐correcting meta‐analytic method that performed well under a priori specified boundary conditions, CBCA and RM distinguished between experience‐based and fabricated statements. However, great heterogeneity limited precise point estimation (i.e., moderate to large effects). In contrast, Scientific Content Analysis (SCAN)—another content‐based technique tested—failed to discriminate between truth and lies. It is discussed how the gap between research on and forensic application of content‐based credibility assessment may be narrowed.

Cubic response surface analysis: Investigating asymmetric and level-dependent congruence effects with third-order polynomial models.
Humberg S, Schönbrodt F, Back M & Nestler S (2020). Psychological Methods. https://doi.org/10.1037/met0000352

PDF Open Material

Citation count: 42

Levels blur the edge -
Congruence's subtle dance -
Outcome's delicate

×

Abstract

Congruence hypotheses play a major role in many areas of psychology. They refer to, for example, the consequences of person-environment fit, similarity, or self-other agreement. For example, are people psychologically better adjusted when their self-view is in line with their reputation? A valid statistical approach that can be applied to investigate congruence hypotheses of this kind is quadratic Response Surface Analysis (RSA) in which a second-order polynomial model is fit to the data and appropriately interpreted. However, quadratic RSA does not allow researchers to investigate more precise expectations about a congruence effect. Do the data support an asymmetric congruence effect, in the sense that congruence leads to the highest (or lowest) outcome, but incongruence in one direction (e.g., self-view exceeds reputation) affects the outcome differently than incongruence in the other direction (e.g., self-view falls behind reputation)? Is there a level-dependent congruence effect, such that the amount of congruence is more strongly related to the outcome variable for some levels of the predictors (e.g., high self-view and reputation) than for others (e.g., low self-view and reputation)? Such complex congruence hypotheses have frequently been suggested in the literature, but they could not be investigated because an appropriate statistical approach has yet to be developed. Here, we present analytical strategies, based on third-order polynomial models, that enable users to investigate asymmetric and level-dependent congruence effects, respectively. To facilitate the correct application of the suggested approaches, we provide respective step-by-step guidelines, corresponding R syntax, and illustrative analyses using simulated and real data. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Management und Bereitstellung von Forschungsdaten in der Psychologie: Überarbeitung der DGPs-Empfehlungen
Gollwitzer M, Abele A, Fiebach C, Ramthun R, Scheel A, Schönbrodt F & Steinberg U (2020). NA. https://doi.org/10.31234/osf.io/hcxtm

PDF

Citation count: 4

Transparency of mind -
Structured data sets unfold -
Science's gentle breeze

×

Abstract

Die Bereitstellung von Forschungsdaten im Rahmen von wissenschaftlichen Publikationen und öffentlich geförderten Forschungsprojekten wird heutzutage als ein zentraler Aspekt offener und transparenter Wissenschaftspraxis angesehen und von immer mehr Förderinstitutionen und wissenschaftlichen Fachzeitschriften eingefordert. Forschende sollten sich daher bemühen, die sogenannten FAIR-Prinzipien zu erfüllen, d.h. Forschungsdaten sollten auffindbar, zugänglich, interoperabel und wiederverwendbar sein. Ein systematisches Forschungsdatenmanagement unterstützt diese Ziele und soll es gleichzeitig ermöglichen, diese effizient zu verwirklichen. Mit den vorliegenden überarbeiteten Empfehlungen zum Management und zur Bereitstellung von Forschungsdaten spezifiziert die Deutsche Gesellschaft für Psychologie (DGPs) wichtige grundlegende Prinzipien des Datenmanagements in der Psychologie und greift hierbei auch Rückmeldungen der DGPs-Mitglieder auf, die im Rahmen einer Umfrage aus dem Jahr 2018 gewonnen wurden. In einem ersten Schritt werden auf Basis disziplinspezifischer Definitionen von Roh-, Primär-, Sekundär- und Metadaten Empfehlungen hinsichtlich des Verarbeitungsgrades der im Rahmen einer Datenveröffentlichung bereitzustellenden Daten gegeben. In der Folge werden datenschutz- sowie urheber- und nutzungsrechtliche Aspekte der Datenbereitstellung diskutiert, bevor die qualitativen Anforderungen an vertrauenswürdige Repositorien zur Bereitstellung von Forschungsdaten definiert werden. Ausführlich wird in der Folge auf pragmatische Aspekte der Datenbereitstellung eingegangen, etwa auf die Unterschiede zwischen Datenveröffentlichungen vom Typ 1 und Typ 2, auf Nutzungsembargos, auf die Definition des "scientific use" durch Nachnutzerinnen und -nutzer von veröffentlichten Daten sowie Empfehlungen zum Umgang mit etwaigen Konfliktfällen. Besonders hervorzuheben ist die neue Empfehlung unterschiedlicher Zugriffsklassen für die Bereitstellung von Daten mit unterschiedlichen datenschutzrechtlichen oder forschungsethischen Anforderungen. Diese reichen von komplett offenen Daten ohne Nutzungsbeschränkungen ("Zugriffsklasse 0") über an standardisierte Bedingungen (z.B. die Nachnutzung für rein wissenschaftliche Zwecke) geknüpfte Datenbereitstellung ("Zugriffsklasse 1") über individualisierte Nutzungsverträge ("Zugriffsklasse 2") bis hin zu einem gesicherten Datenzugriff, der nur unter streng kontrollierten Bedingungen (z.B. in einem Forschungsdatenzentrum) erfolgen kann ("Zugriffsklasse 3"). Die Umsetzung dieser wichtigen Neuerung ist jedoch geknüpft an technische Implementationen des Zugriffsklassenkonzeptes durch Datenrepositorien, die die entsprechenden Funktionalitäten bereitstellen. Zusammenfassend zielen die überarbeiteten Empfehlungen darauf ab, Forscherinnen und Forschern pragmatische Wege für den offenen und transparenten Umgang mit psychologischen Forschungsdaten aufzuzeigen und dabei auch strukturelle Herausforderungen eines für alle Beteiligten gewinnbringenden "data sharings" anzusprechen und entsprechende Maßnahmen vorzuschlagen.

Predicting affective appraisals from facial expressions and physiology using machine learning
Israel L & Schönbrodt F (2020). Behavior Research Methods. https://doi.org/10.3758/s13428-020-01435-y

PDF Open Data Open Material

Citation count: 10

Electro pulses dance -
Secrets of the heart revealed -
Pleasure's subtle signs

×

Abstract

The present study explored the interrelations between a broad set of appraisal ratings and five physiological signals, including facial EMG, electrodermal activity, and heart rate variability, that were assessed in 157 participants watching 10 emotionally charged videos. A total of 134 features were extracted from the physiological data, and a benchmark comparing different kinds of machine learning algorithms was conducted to test how well the appraisal dimensions can be predicted from these features. For 13 out of 21 appraisals, a robust positive R² was attained, indicating that the dimensions are actually related to the considered physiological channels. The highest R² (.407) was reached for the appraisal dimension intrinsic pleasantness. Moreover, the comparison of linear and nonlinear algorithms and the inspection of the links between the appraisals and single physiological features using accumulated local effects plots indicates that the relationship between physiology and appraisals is nonlinear. By constructing different importance measures for the assessed physiological channels, we showed that for the 13 predictable appraisals, the five channels explained different amounts of variance and that only a few blocks incrementally explained variance beyond the other physiological channels.

Motivated behavior in intimate relationships: Comparing the predictive value of motivational variables
Zygar–Hoffmann C, Pusch S, Hagemeyer B & Schönbrodt F (2020). Social Psychological Bulletin. https://doi.org/10.32872/spb.2873

PDF Open Data Open Material Preregistration

Citation count: 14

Relationship's gentle tide -
Motives flow, hearts connect strong -
Love's subtle, yet clear

×

Abstract

Motivational variables are considered fundamental factors influencing the occurrence of behavior. The current study compared different types of motivational variables (implicit and explicit motive dispositions, motivation as states and as aggregated person-level variables) in their ability to predict communal and agentic behavior reports in intimate relationships. 510 individuals completed measures of dispositional communion and agency motives and participated in a dyadic experience sampling study with five assessments per day across four weeks. They reported on their momentary communal and agentic motivation, as well as on their own and their partner’s behaviors. All examined types of motivational variables predicted certain behavior reports on the between-person or within-person level and had incremental effects beyond the other motivational variables in at least one motive domain. Directly replicating and conceptually extending prior research, the effects of motivational states and their aggregates were consistently found across behavioral outcomes, across self- and partner-reports and across the motive domains of communion and agency. Using the example of motivational states, the general value of assessing within-person variables for psychological phenomena in ESM-designs is discussed.

Database of Expert-Coded German PSE Stories
Schönbrodt F, Hagemeyer B, Brandstätter V, Czikmantori T, Gröpel P, Hennecke M, Israel L, Janson K, Kemper N, Köllner M, Kopp P, Mojzisch A, Müller-Hotop R, Prüfer J, Quirin M, Scheidemann B, Schiestel L, Schulz‐Hardt S, Sust L, Zygar C & Schultheiss O (2020). Psychology Archives. https://doi.org/10.23668/psycharchives.2738

PDF Open Data Open Material

Citation count: 2

Imagined desires -
nested in coded stories -
motives in silence

×

Abstract

We present two openly accessible databases related to the assessment of implicit motives using Picture Story Exercises (PSEs): (a) A database of 183,415 German sentences, nested in 26,389 stories provided by 4,570 participants, which have been coded by experts using Winter’s coding system for the implicit affiliation/intimacy, achievement, and power motives, and (b) a database of 54 classic and new pictures which have been used as PSE stimuli. Updated picture norms are provided which can be used to select appropriate pictures for PSE applications. Based on an analysis of the relations between raw motive scores, word count, and sentence count, we give recommendations on how to control motive scores for story length, and validate the recommendation with a meta-analysis on gender differences in the implicit affiliation motive that replicates existing findings. We discuss to what extent the guiding principles of the story length correction can be generalized to other content coding systems for narrative material. Several potential applications of the databases are discussed, including (un)supervised machine learning of text content, psychometrics, and better reproducibility of PSE research.

Recalling Experiences: Looking at Momentary, Retrospective and Global Assessments of Relationship Satisfaction
Zygar–Hoffmann C & Schönbrodt F (2020). Collabra Psychology. https://doi.org/10.1525/collabra.278

PDF Open Material

Citation count: 29

Memory's fleeting gleam -
Shadows of love's present past -
Love's true, brief, pure spark

×

Abstract

Relationship satisfaction can be assessed in retrospection, as a global evaluation, or as a momentary state. In two experience sampling studies (N = 130, N = 510) the specificities of these assessment modalities are examined. We show that 1) compared to other summary statistics like the median, the mean of relationship satisfaction states describes retrospective and global evaluations best (but the difference to some other summary statistics was negligible); 2) retrospection introduces an overestimation of the average annoyance in the relationship reported on a momentary basis, which results in an overall negative mean-level bias for retrospective relationship satisfaction; 3) this bias is most strongly moderated by global relationship satisfaction at the time of retrospection; 4) snapshots of momentary relationship satisfaction get representative of global evaluations after approximately two weeks of sampling. The findings extend the recall bias reported in the literature for retrospection of negative affect to the domain of relationship evaluations and assist researchers in designing efficient experience sampling studies.

2019

Truth and Wishful Thinking: How Interindividual Differences in Communal Motives Manifest in Momentary Partner Perceptions
Pusch S, Schönbrodt F, Zygar–Hoffmann C & Hagemeyer B (2019). European Journal of Personality. https://doi.org/10.1002/per.2227

PDF Open Material Preregistration

Citation count: 21

Perceiver's desire -
Truth and wishful thinking blur -
Communal hearts hide

×

Abstract

Although rooted in reality, partner perceptions often reflect wishful thinking due to perceivers’ needs. Dispositional needs, or motives, can differ between persons; however, little is known about their differential associations with everyday partner perception. The present study used data from a 4–week experience sampling study ( N = up to 60942 surveys from 510 individuals nested in 259 couples) to examine the effects of perceivers’ partner–related implicit and explicit communal motives on the perception of (i) global communal partner behaviour and (ii) specific communal and uncommunal partner behaviours. The results of truth and bias models of judgement and quasi–signal detection analyses indicate that strong implicit communal approach motives and strong explicit communal motives are associated with the tendency to overestimate the partner's communal behaviour. Additionally, strong implicit communal approach motives were associated with the tendency to avoid perceptions of uncommunal partner behaviour. Neither implicit nor explicit communal motives had an effect on accuracy in the perception of particularly communal partner behaviour. The results highlight the relevance of both implicit and explicit communal motives for momentary partner perceptions and emphasise the benefits of dyadic microlongitudinal designs for a better understanding of the mechanisms through which individual differences manifest in couples’ everyday lives. © 2019 The Authors. European Journal of Personality published by John Wiley & Sons Ltd on behalf of European Association of Personality Psychology

A Series of Meta-Analytic Tests of the Efficacy of Long-Term Psychoanalytic Psychotherapy
Woll C & Schönbrodt F (2019). European Psychologist. https://doi.org/10.1027/1016-9040/a000385

PDF Open Data Open Material Preregistration

Citation count: 29

Treatment's subtle gain -
Slight improvement in the depths -
Complex minds slowly

×

Abstract

Abstract. Recent meta-analyses come to conflicting conclusions about the efficacy of long-term psychoanalytic psychotherapy (LTPP). Our first goal was to reproduce the most recent meta-analysis by Leichsenring, Abbass, Luyten, Hilsenroth, and Rabung (2013) who found evidence for the efficacy of LTPP in the treatment of complex mental disorders. Our replicated effect sizes were in general slightly smaller. Second, we conducted an updated meta-analysis of randomized controlled trials comparing LTPP (lasting for at least 1 year and 40 sessions) to other forms of psychotherapy in the treatment of complex mental disorders. We focused on a transparent research process according to open science standards and applied a series of elaborated meta-analytic procedures to test and control for publication bias. Our updated meta-analysis comprising 191 effect sizes from 14 eligible studies revealed small, statistically significant effect sizes at post-treatment for the outcome domains psychiatric symptoms, target problems, social functioning, and overall effectiveness (Hedges’ g ranging between 0.24 and 0.35). The effect size for the domain personality functioning (0.24) was not significant ( p = .08). No signs for publication bias could be detected. In light of a heterogeneous study set and some methodological shortcomings in the primary studies, these results should be interpreted cautiously. In conclusion, LTPP might be superior to other forms of psychotherapy in the treatment of complex mental disorders. Notably, our effect sizes represent the additional gain of LTPP versus other forms of primarily long-term psychotherapy. In this case, large differences in effect sizes are not to be expected.

Training students for the Open Science future
Schönbrodt F (2019). Nature Human Behaviour. https://doi.org/10.1038/s41562-019-0726-z

PDF

Citation count: 24

Truth in science pursuit -
Fear of market's narrow view -
Openness sets free

×

Abstract

Many PhD students are enthusiastic about robust scientific practices, but afraid that ‘doing good science’ will jeopardize their chances on the job market, argues Felix Schönbrodt, Managing Director of the LMU Open Science Center. Aligning incentives and preparing students for a job market that values contributions to Open Science will be key.

Introducing the Leapfrog Design: A Simple Bayesian Adaptive Rolling Trial Design for Accelerated Treatment Development and Optimization
Blackwell S, Woud M, Margraf J & Schönbrodt F (2019). Clinical Psychological Science. https://doi.org/10.1177/2167702619858071

PDF

Citation count: 33

Bayes' silent guide -
Leapfrogging to best doses -
Science's swift leap

×

Abstract

The application of basic science research to the development and optimization of psychological treatments holds great potential. However, this process of clinical translation is challenging and time-consuming, and the standard route by which it proceeds is inefficient. Adaptive rolling designs, which originated within cancer treatment research, provide an alternative methodology with potential to accelerate development and optimization of psychological treatments. In such designs, multiple treatment options are tested simultaneously, with sequential Bayesian analyses used to remove poorly performing arms. Further, new treatment arms informed by the latest research findings can be introduced into the existing infrastructure as the trial progresses. These features dramatically reduce the sample sizes needed and offer a means for more rapid and efficient clinical translation. This article outlines the utility of such designs to clinical psychological science, focusing on a new variant termed the leapfrog design, and discusses their potential uses to accelerate clinical translation.

Emotion Prediction with Weighted Appraisal Models – Towards Validating a Psychological Theory of Affect
Israel L & Schönbrodt F (2019). IEEE Transactions on Affective Computing. https://doi.org/10.1109/taffc.2019.2940937

PDF Open Material

Citation count: 9

Feeling weighted deep -
Evaluations shape the soul -
Emotion's fragile dance

×

Abstract

Appraisal theories are a prominent approach for the explanation and prediction of emotions. According to these theories, the subjective perception of an emotion results from a series of specific event evaluations. To validate and extend one of the most known representatives of appraisal theory, the Component Process Model by Klaus Scherer, we implemented four computational appraisal models that predicted emotion labels based on prototype similarity calculations. Different weighting algorithms, mapping the models’ input to a distinct emotion label, were integrated in the models. We evaluated the plausibility of the models’ structure by assessing their predictive power and comparing their performance to a baseline model and a highly predictive machine learning algorithm. Model parameters were estimated from empirical data and validated out-of-sample. All models were notably better than the baseline model and able to explain part of the variance in the emotion labels. The preferred model, yielding a relatively high performance and stable parameter estimations, was able to predict a correct emotion label with an accuracy of 40.2 percent and a correct emotion family with an accuracy of 76.9 percent. The weighting algorithm of this favored model corresponds to the weighting complexity implied by the Component Process Model, but uses differing weighting parameters.

Attitudes Toward Open Science and Public Data Sharing
Abele A, Gollwitzer M, Steinberg U & Schönbrodt F (2019). Social Psychology. https://doi.org/10.1027/1864-9335/a000384

PDF Open Data Open Material

Citation count: 77

Transparent truth sought high -
Fears and hopes in open minds -
Data's gentle gift

×

Abstract

Abstract. Central values of science are, among others, transparency, verifiability, replicability, and openness. The currently very prominent Open Science (OS) movement supports these values. Among its most important principles are open methodology (comprehensive and useful documentation of methods and materials used), open access to published research output, and open data (making collected data available for re-analyses). We here present a survey conducted among members of the German Psychological Society ( N = 337), in which we applied a mixed-methods approach (quantitative and qualitative data) to assess attitudes toward OS in general and toward data sharing more specifically. Attitudes toward OS were distinguished into positive expectations (“hopes”) and negative expectations (“fears”). These were un-correlated. There were generally more hopes associated with OS and data sharing than fears. Both hopes and fears were highest among early career researchers and lowest among professors. The analysis of the open answers revealed that generally positive attitudes toward data sharing (especially sharing of data related to a published article) are somewhat diminished by cost/benefit considerations. The results are discussed with respect to individual researchers’ behavior and with respect to structural changes in the research system.

Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter E, Schönbrodt F, Gervais W & Hilgard J (2019). Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245919847196

PDF Open Data Open Material

Citation count: 559

Statistical corrections -
Fickle, flawed, and fleeting too -
Truth in shifting sands

×

Abstract

Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/ .

Response surface analysis with multilevel data: Illustration for the case of congruence hypotheses.
Nestler S, Humberg S & Schönbrodt F (2019). Psychological Methods. https://doi.org/10.1037/met0000199

Citation count: 136

Fitted curves abound -
Congruence in shifting sands -
Value in the depths

×

Abstract

Response surface analysis (RSA) is a statistical approach that enables researchers to test congruence hypotheses; the proposition that the degree of congruence between people's values in 2 psychological constructs should be positively or negatively related to their value in an outcome variable. This is done by estimating a polynomial regression model and using the graph of the model and several parameters as a guide to interpret the resulting regression coefficients in terms of the congruence hypothesis. One problem with using RSA in applied research is that the model and the interpretation of the model's parameters in terms of congruence effects have only been thoroughly developed for single-level data. Here, we present an extension of RSA to multilevel data. Among other things we show how the standard errors can be computed and how researchers can decide whether the occurrence of a congruence effect depends on a Level 2 covariate. We illustrate the suggested extension with 2 examples that guide readers through the test of congruence effects in the case of multilevel data. We also provide R scripts that researchers can adopt to conduct multilevel RSA. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

A tutorial on Bayes Factor Design Analysis using an informed prior
Stefan A, Gronau Q, Schönbrodt F & Wagenmakers E (2019). Behavior Research Methods. https://doi.org/10.3758/s13428-018-01189-8

PDF Open Material

Citation count: 278

Stats guide the curve -
Prior knowledge shapes the road -
Certainty's slope

×

Abstract

NA

2018

The Dominance, Prestige, and Leadership Account of Social Power Motives
Suessenbach F, Loughnan S, Schönbrodt F & Moore A (2018). European Journal of Personality. https://doi.org/10.1002/per.2184

PDF Open Data Open Material Preregistration

Citation count: 75

Will and admiration -
Different paths to power's throne -
Leader, or to rule

×

Abstract

The power motive predicts influential social behaviour; however, its heterogeneous conceptualisations have produced inconsistent results. To overcome this problem, we developed and validated a unitary taxonomy of social power motives based on established delineations of social hierarchies: the dominance, prestige, and leadership account. While we could measure these motives both reliably and distinctively (study 1), we also showed they strongly related to a common power desire (study 2). Assessing their nomological networks (studies 3 and 4), we demonstrated distinct associations between the dominance motive (D: wanting to coerce others into adhering to one's will) and anger and verbal aggression; the prestige motive (P: wanting to obtain admiration and respect) and the fear of losing reputation and claiming to have higher moral concerns; the leadership motive (L: wanting to take responsibility in and for one's group) and emotional stability and helping behaviour. Furthermore, while D uniquely predicted agonistic/retaliatory behaviour in dictator games (study 5), L uniquely predicted the attainment of higher employment ranks in various professions (study 7). Finally, at least to some degree, P and L related positively, and D negatively to prosocial donating behaviour (study 6). This taxonomy represents a novel and powerful approach to predicting influential social behaviour. © 2018 European Association of Personality Psychology

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings
Klein R, Vianello M, Hasselman F, Adams B, Adams R, Alper S, Aveyard M, Axt J, Babalola M, Bahník Š, Batra R, Berkics M, Bernstein M, Berry D, Białobrzeska O, Binan E, Bocian K, Brandt M, Busching R, Rédei A, Cai H, Cambier F, Cantarero K, Carmichael C, Cerić F, Chandler J, Chang J, Chatard A, Chen E, Cheong W, Cicero D, Coen S, Coleman J, Collisson B, Conway M, Corker K, Curran P, Cushman F, Dagona Z, Dalgar I, Rosa A, Davis W, Bruijn M, Schutter L, Devos T, Vries M, Doğulu C, Dozo N, Dukes K, Dunham Y, Durrheim K, Ebersole C, Edlund J, Eller A, English A, Finck C, Frankowska N, Freyre M, Friedman M, Galliani E, Gandi J, Ghoshal T, Giessner S, Gill T, Gnambs T, Gómez Á, González R, Graham J, Grahe J, Grahek I, Green E, Hai K, Haigh M, Haines E, Hall M, Heffernan M, Hicks J, Houdek P, Huntsinger J, Huynh H, IJzerman H, Inbar Y, Innes-Ker Å, Jiménez‐Leal W, John M, Joy-Gaba J, Kamiloglu R, Kappes H, Karabatı S, Karick H, Keller V, Kende A, Kervyn N, Knežević G, Kovacs C, Krueger L, Kurapov G, Kurtz J, Lakens D & Lazarević L (2018). Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245918810225

PDF Open Data Open Material Preregistration

Citation count: 1001

Studies dance with varied pace -
Significance and effect fade -
Truth in many forms

×

Abstract

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance ( p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion ( p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.

Testing Similarity Effects with Dyadic Response Surface Analysis
Schönbrodt F, Humberg S & Nestler S (2018). European Journal of Personality. https://doi.org/10.1002/per.2169

PDF Open Material

Citation count: 167

Pairful echoes blend -
Similarities unfold deep -
Relationships mapped

×

Abstract

Dyadic similarity effect hypotheses state that the (dis)similarity between dyad members (e.g. the similarity on a personality dimension) is related to a dyadic outcome variable (e.g. the relationship satisfaction of both partners). Typically, these hypotheses have been investigated by using difference scores or other profile similarity indices as predictors of the outcome variables. These approaches, however, have been vigorously criticized for their conceptual and statistical shortcomings. Here, we introduce a statistical method that is based on polynomial regression and addresses most of these shortcomings: dyadic response surface analysis. This model is tailored for similarity effect hypotheses and fully accounts for the dyadic nature of relationship data. Furthermore, we provide a tutorial with an illustrative example and reproducible R and Mplus scripts that should assist substantive researchers in precisely formulating, testing, and interpreting their dyadic similarity effect hypotheses. © 2018 European Association of Personality Psychology

Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results
Silberzahn R, Uhlmann E, Martin D, Anselmi P, Aust F, Awtrey E, Bahník Š, Bai F, Bannard C, Bonnier E, Carlsson R, Cheung F, Christensen G, Clay R, Craig M, Rosa A, Dam L, Evans M, Cervantes I, Fong N, Gamez-Djokic M, Glenz A, Gordon-McKeon S, Heaton T, Hederos K, Heene M, Mohr A, Högden F, Hui K, Johannesson M, Kalodimos J, Kaszubowski E, Kennedy D, Lei R, Lindsay T, Liverani S, Madan C, Molden D, Molleman E, Morey R, Mulder L, Nijstad B, Pope N, Pope B, Prenoveau J, Rink F, Robusto E, Roderique H, Sandberg A, Schlüter E, Schönbrodt F, Sherman M, Sommer S, Sotak K, Spain S, Spörlein C, Stafford T, Stefanutti L, Täuber S, Ullrich J, Vianello M, Wagenmakers E, Witkowiak M, Yoon S & Nosek B (2018). Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245917747646

PDF Open Material

Citation count: 832

Choices shroud the truth -
Varying hands, one same fact -
Subjectivity reigns

×

Abstract

Twenty-nine teams involving 61 analysts used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 ( Mdn = 1.31) in odds-ratio units. Twenty teams (69%) found a statistically significant positive effect, and 9 teams (31%) did not observe a significant relationship. Overall, the 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy in which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective, analytic choices influence research results.

Is accurate, positive, or inflated self-perception most advantageous for psychological adjustment? A competitive test of key hypotheses.
Humberg S, Dufner M, Schönbrodt F, Geukes K, Hutteman R, Küfner A, Zalk M, Denissen J, Nestler S & Back M (2018). Journal of Personality and Social Psychology. https://doi.org/10.1037/pspp0000204

PDF Open Data Open Material

Citation count: 75

Balanced positivity -
Blurs the line of inflated pride -
Self-perception's dance

×

Abstract

Empirical research on the (mal-)adaptiveness of favorable self-perceptions, self-enhancement, and self-knowledge has typically applied a classical null-hypothesis testing approach and provided mixed and even contradictory findings. Using data from 5 studies (laboratory and field, total N = 2,823), we used an information-theoretic approach combined with Response Surface Analysis to provide the first competitive test of 6 popular hypotheses: that more favorable self-perceptions are adaptive versus maladaptive (Hypotheses 1 and 2: Positivity of self-view hypotheses), that higher levels of self-enhancement (i.e., a higher discrepancy of self-viewed and objectively assessed ability) are adaptive versus maladaptive (Hypotheses 3 and 4: Self-enhancement hypotheses), that accurate self-perceptions are adaptive (Hypothesis 5: Self-knowledge hypothesis), and that a slight degree of self-enhancement is adaptive (Hypothesis 6: Optimal margin hypothesis). We considered self-perceptions and objective ability measures in two content domains (reasoning ability, vocabulary knowledge) and investigated 6 indicators of intra- and interpersonal psychological adjustment. Results showed that most adjustment indicators were best predicted by the positivity of self-perceptions. There were some specific self-enhancement effects, and evidence generally spoke against the self-knowledge and optimal margin hypotheses. Our results highlight the need for comprehensive and simultaneous tests of competing hypotheses. Implications for the understanding of underlying processes are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

From Motive Dispositions to States to Outcomes: An Intensive Experience Sampling Study on Communal Motivational Dynamics in Couples
Zygar–Hoffmann C, Hagemeyer B, Pusch S & Schönbrodt F (2018). European Journal of Personality. https://doi.org/10.1002/per.2145

PDF Open Data Open Material Preregistration

Citation count: 33

Couples in harmony -
Motivation's gentle dance -
Love's tender, daily beat

×

Abstract

Embedded in a theoretically founded process model (termed Dynamics of Motive Satisfaction, ‘DynaMoS’), the present study examined the links between the implicit dispositional communion motive, everyday motivational dynamics, and relationship outcomes in couples. Within–subject processes are proposed to explain between–subject associations of dispositional motives and relationship satisfaction. For an empirical test of the model, data on the dispositional partner–related need for communion and global relationship satisfaction were obtained from 152 individuals in heterosexual relationships. In an extensive experience sampling spanning 2 weeks, a subsample of 130 individuals answered questions about their current motivational states, mood, state relationship satisfaction, and experiences with their partner five times a day. The results were largely consistent with the DynaMoS model: (1) individuals with a strong dispositional implicit communion motive reported more often to be in a communal motivational state; (2) communally motivated individuals were more likely to engage in subsequent instrumental behaviour; and (3) relationship experiences that potentially satisfy communion motivation led to more positive relationship outcomes when individuals were motivated before compared with when they were not. It is discussed how these results and the experience sampling method can foster our understanding of how dispositional characteristics translate into everyday processes and shape relationship outcomes. Copyright © 2018 European Association of Personality Psychology

Forschungstransparenz als hohes wissenschaftliches Gut stärken
Schönbrodt F, Maier M, Heene M & Bühner M (2018). Psychologische Rundschau. https://doi.org/10.1026/0033-3042/a000386

PDF

Citation count: 11

Trusted science dreams -
Open hearts and transparent -
Evidence we seek

×

Abstract

Zusammenfassung. Groß angelegte Replikationsprojekte der letzten Jahre legen ein aus unserer Sicht beunruhigendes Ausmaß an nicht-replizierbaren Befunden in der wissenschaftlichen Literatur nahe, sowohl in der Psychologie als auch in anderen Disziplinen. Basierend auf einer Analyse einiger Ursachen dieser Situation argumentieren wir, dass der Wandel hin zu einer offenen Wissenschaft („Open Science“) eine Konsequenz aus der Glaubwürdigkeitskrise sein muss. Wir plädieren für konkrete und machbare Änderungen in den Arbeitseinheiten und Instituten vor Ort, und zeigen exemplarisch, welche Schritte am Department Psychologie der Ludwig-Maximilians-Universität München umgesetzt wurden. Diese Schritte betreffen Anreizstrukturen, die Forschungskultur, die Lehre und die Verzahnung mit der Ethikkommission. Sie haben das Ziel, eine reproduzierbarere und glaubwürdigere Forschung zu unterstützen, ohne unnötige bürokratische Belastungen zu erzeugen.

2017

FAQ zu Open Data und Open Science in der Sportpsychologie
Schönbrodt F & Scheel A (2017). Zeitschrift für Sportpsychologie. https://doi.org/10.1026/1612-5010/a000217

PDF

Citation count: 4

Good science pursued -
Free riders hide in the shadows -
Truth's subtle, hard won

×

Abstract

Präambel: Viele Diskussionspunkte um Open Data lassen sich grob in zwei Perspektiven einteilen. (A) die Qualitätsperspektive, bzw. die Gemeinwohlperspektive: Wie können wir gute Wissenschaft machen und echtes Wissen generieren? Was können wir aus der aktuellen Replikationskrise lernen und in Zukunft besser machen? Die andere Perspektive könnte man als (B) individuell-strategische Perspektive bezeichnen: Was bedeuten die neuen Forschungspraktiken und -anforderungen für die individuellen Karrieren von Forscherinnen und Forschern, insbesondere aus dem Nachwuchs? Wo liegen strategische Vor- oder Nachteile? Gibt es „free rider“ oder „research parasites“ ( Longo & Drazen, 2016 ; McNutt, 2016 ), die mich ausnutzen? Wie kann ich mich in einem hart umkämpften Arbeitsmarkt gegen die Konkurrenz durchsetzen? Die aktuell vorliegenden Fragen beziehen sich zu einem großen Teil auf die zweite Perspektive. Das ist bei Nachwuchsforschenden legitim und nachvollziehbar. Uns ist es aber wichtig, bei allen Karriereunsicherheiten die übergeordnete Frage nicht zu vergessen: Wie können wir gute Wissenschaft umsetzen? Wie können wir dazu beitragen, dass echtes Wissen generiert wird? Das sollte uns als Wissenschaftlerinnen und Wissenschaftlern das wichtigste Anliegen sein. Und entsprechend sollten wir als Fachgemeinschaft auch Karriereanreize so organisieren, dass sie dieses Ziel unterstützen.

Redefine statistical significance
Benjamin D, Berger J, Johannesson M, Nosek B, Wagenmakers E, Berk R, Bollen K, Brembs B, Brown L, Camerer C, Cesarini D, Chambers C, Clyde M, Cook T, Boeck P, Dienes Z, Dreber A, Easwaran K, Efferson C, Fehr E, Fidler F, Field A, Forster M, George E, Gonzalez R, Goodman S, Green E, Green D, Greenwald A, Hadfield J, Hedges L, Held L, Ho T, Hoijtink H, Hruschka D, Imai K, Imbens G, Ioannidis J, Jeon M, Jones J, Kirchler M, Laibson D, List J, Little R, Lupia A, Machery É, Maxwell S, McCarthy M, Moore D, Morgan S, Munafò M, Nakagawa S, Nyhan B, Parker T, Pericchi L, Perugini M, Rouder J, Rousseau J, Savalei V, Schönbrodt F, Sellke T, Sinclair B, Tingley D, Zandt T, Vazire S, Watts D, Winship C, Wolpert R, Xie Y, Young C, Zinman J & Johnson V (2017). Nature Human Behaviour. https://doi.org/10.1038/s41562-017-0189-z

PDF

Citation count: 2715

Uncertainty reigns -
Meaning in numbers shifts -
Science's gentle dance

×

Abstract

NA

Concurrent and longitudinal dyadic polynomial regression analyses of Big Five traits and relationship satisfaction: Does similarity matter?
Weidmann R, Schönbrodt F, Ledermann T & Grob A (2017). Journal of Research in Personality. https://doi.org/10.1016/j.jrp.2017.04.003

PDF

Citation count: 86

Tendencies entwine -
Similar hearts beat in sync -
Joy in harmony

×

Abstract

NA

Enhanced versus simply positive: A new condition-based regression analysis to disentangle effects of self-enhancement from effects of positivity of self-view.
Humberg S, Dufner M, Schönbrodt F, Geukes K, Hutteman R, Zalk M, Denissen J, Nestler S & Back M (2017). Journal of Personality and Social Psychology. https://doi.org/10.1037/pspp0000134

PDF Open Data Open Material

Citation count: 80

Muted pride unmasked -
Shadows of self-love revealed -
Self's subtle deceit

×

Abstract

Despite a large body of literature and ongoing refinements of analytical techniques, research on the consequences of self-enhancement (SE) is still vague about how to define SE effects, and empirical results are inconsistent. In this paper, we point out that part of this confusion is due to a lack of conceptual and methodological differentiation between effects of individual differences in how much people enhance themselves (SE) and in how positively they view themselves (positivity of self-view; PSV). We show that methods commonly used to analyze SE effects are biased because they cannot differentiate between the effects of PSV and the effects of SE. We provide a new condition-based regression analysis (CRA) that unequivocally identifies effects of SE by testing intuitive and mathematically derived conditions on the coefficients in a bivariate linear regression. Using data from 3 studies on intellectual SE (total N = 566), we then illustrate that the CRA provides novel results as compared with traditional methods. Results suggest that many previously identified SE effects are in fact effects of PSV alone. The new CRA approach thus provides a clear and unbiased understanding of the consequences of SE. It can be applied to all conceptualizations of SE and, more generally, to every context in which the effects of the discrepancy between 2 variables on a third variable are examined. (PsycINFO Database Record

Bayes factor design analysis: Planning for compelling evidence
Schönbrodt F & Wagenmakers E (2017). Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-017-1230-y

PDF Open Material

Citation count: 743

Stats woven with care -
Evidence's subtle threads -
Certainty unfolds

×

Abstract

NA

A Bayesian bird's eye view of ‘Replications of important results in social psychology’
Marsman M, Schönbrodt F, Morey R, Yao Y, Gelman A & Wagenmakers E (2017). Royal Society Open Science. https://doi.org/10.1098/rsos.160426

PDF Open Data Open Material

Citation count: 54

Overlapping doubt creeps -
Results in fragile flight -
Science seeks a nest

×

Abstract

We applied three Bayesian methods to reanalyse the preregistered contributions to the Social Psychology special issue 'Replications of Important Results in Social Psychology' (Nosek & Lakens. 2014 Registered reports: a method to increase the credibility of published results. Soc. Psychol.45, 137-141. (doi:10.1027/1864-9335/a000192)). First, individual-experiment Bayesian parameter estimation revealed that for directed effect size measures, only three out of 44 central 95% credible intervals did not overlap with zero and fell in the expected direction. For undirected effect size measures, only four out of 59 credible intervals contained values greater than [Formula: see text] (10% of variance explained) and only 19 intervals contained values larger than [Formula: see text]. Second, a Bayesian random-effects meta-analysis for all 38 t-tests showed that only one out of the 38 hierarchically estimated credible intervals did not overlap with zero and fell in the expected direction. Third, a Bayes factor hypothesis test was used to quantify the evidence for the null hypothesis against a default one-sided alternative. Only seven out of 60 Bayes factors indicated non-anecdotal support in favour of the alternative hypothesis ([Formula: see text]), whereas 51 Bayes factors indicated at least some support for the null hypothesis. We hope that future analyses of replication success will embrace a more inclusive statistical approach by adopting a wider range of complementary techniques.

Der Umgang mit Forschungsdaten im Fach Psychologie: Konkretisierung der DFG-Leitlinien
Schönbrodt F, Gollwitzer M & Abele A (2017). Psychologische Rundschau. https://doi.org/10.1026/0033-3042/a000341

PDF

Citation count: 52

Research meets data cold -
Guidelines unfold like petals -
Data's gentle kiss

×

Abstract

Eine Mglichkeit, die Leistungen datengenerierender Forschung strker zu wrdigen, ist die Schaffung einer neuen Publikationskategorie, die "Datenbereitstellung". Artikel,

2016

Wege aus der Vertrauenskrise
Geukes K, Schönbrodt F, Utesch K, Geukes S & Back M (2016). Zeitschrift für Sportpsychologie. https://doi.org/10.1026/1612-5010/a000167

PDF

Citation count: 12

Faith lost, seeking -
Robust results, transparent -
Trust in every step

×

Abstract

Zusammenfassung. Die Psychologie, und mit ihr auch die Sportpsychologie, befindet sich in einer sogenannten Vertrauenskrise (siehe Titel des Themenhefts). Während die Diskussion ihrer Ursachen und Konsequenzen lebendig geführt wird, fehlt es an der Herausarbeitung individueller Möglichkeiten, Vertrauen wieder kollektiv herzustellen. In diesem Beitrag zeigen wir Wege aus dieser Vertrauenskrise auf und beschreiben insbesondere individuelle Schritte hin zu verlässlichem und offenem Forschen: Verlässliches Forschen bezieht sich auf das Publizieren von robusten Ergebnissen, sowie von direkten und konzeptuellen Replikationsstudien und offenes Forschen auf die Transparenz hinsichtlich der Planung (Präregistrierung), der Durchführung (Open Material) und der Analyse (Open Data, Reproducible Code) wissenschaftlicher Untersuchungen. Das Bekenntnis zu verlässlicher und offener Forschung wird unser Handeln in den verschiedenen Rollen im Wissenschaftssystem (als Forscher, Reviewer, Herausgeber, Betreuer, Kommissionsmitglied etc.) verändern. In diesem Sinne begreifen wir in diesem Beitrag die momentane Diskussion als Chance, die Zuverlässigkeit unserer Befunde nachhaltig zu steigern und langfristig gerechtfertigtes Vertrauen zu schaffen.

Does the experience of presence restore self-control?
Schiestel L & Schönbrodt F (2016). Self and Identity. https://doi.org/10.1080/15298868.2015.1132758

PDF

Citation count: 1

Illusions calm the mind -
Restoring control's gentle sway -
Holiday from self

×

Abstract

A wide range of research suggests that the exertion of self-control is a limited resource or constrained by a motivational shift. Less attention, however, has been given to the question how self-control capacity can be restored. This study examined whether this restoration can be achieved by “presence,” the perceptual illusion of non-mediation. We suggest that the adoption of the mediated reference systems creates a “holiday from self” and restores self-control capacity. Depleted participants who saw an immersive video clip performed better in a subsequent self-control task than participants who saw a non-immersive video clip or were in a waiting group. But the concrete mechanism remains open: Self reported presence did not serve as a mediator.

To Live Among Like-Minded Others
Bleidorn W, Schönbrodt F, Gebauer J, Rentfrow P, Potter J & Gosling S (2016). Psychological Science. https://doi.org/10.1177/0956797615627133

PDF

Citation count: 157

Personas align here -
Self-esteem blooms in kindred -
Vibrant city soul

×

Abstract

Does it matter if your personality fits in with the personalities of the people where you live? The present study explored the links between person-city personality fit and self-esteem. Using data from 543,934 residents of 860 U.S. cities, we examined the extent to which the fit between individuals' Big Five personality traits and the Big Five traits of the city where they live (i.e., the prevalent traits of the city's inhabitants) predicts individuals' self-esteem. To provide a benchmark for these effects, we also estimated the degree to which the fit between person and city religiosity predicts individuals' self-esteem. The results provided a nuanced picture of the effects of person-city personality fit on self-esteem: We found significant but small effects of fit on self-esteem only for openness, agreeableness, and conscientiousness, rather than effects for all Big Five traits. Similar results and effect sizes were observed for religiosity. We conclude with a discussion of the relevance and limitations of this study.

The Peer Reviewers' Openness Initiative: incentivizing open research practices through peer review
Morey R, Chambers C, Etchells P, Harris C, Hoekstra R, Lakens D, Lewandowsky S, Morey C, Newman D, Schönbrodt F, Vanpaemel W, Wagenmakers E & Zwaan R (2016). Royal Society Open Science. https://doi.org/10.1098/rsos.150547

PDF

Citation count: 302

Transparent pages shared -
Collaboration's gentle -
Science's open door

×

Abstract

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism-reviewers withholding comprehensive review-to achieve the goal of creating the expectation of open practices as a matter of scientific principle.

2015

Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences.
Schönbrodt F, Wagenmakers E, Zehetleitner M & Perugini M (2015). Psychological Methods. https://doi.org/10.1037/met0000061

PDF Open Material

Citation count: 567

Small samples suffice -
Flexibility in the test -
Evidence guides path

×

Abstract

Unplanned optional stopping rules have been criticized for inflating Type I error rates under the null hypothesis significance testing (NHST) paradigm. Despite these criticisms, this research practice is not uncommon, probably because it appeals to researcher's intuition to collect more data to push an indecisive result into a decisive region. In this contribution, we investigate the properties of a procedure for Bayesian hypothesis testing that allows optional stopping with unlimited multiple testing, even after each participant. In this procedure, which we call Sequential Bayes Factors (SBFs), Bayes factors are computed until an a priori defined level of evidence is reached. This allows flexible sampling plans and is not dependent upon correct effect size guesses in an a priori power analysis. We investigated the long-term rate of misleading evidence, the average expected sample sizes, and the biasedness of effect size estimates when an SBF design is applied to a test of mean differences between 2 groups. Compared with optimal NHST, the SBF design typically needs 50% to 70% smaller samples to reach a conclusion about the presence of an effect, while having the same or lower long-term rate of wrong inference. (PsycINFO Database Record

Is valuing happiness associated with lower well-being? A factor-level analysis using the Valuing Happiness Scale
Luhmann M, Necka E, Schönbrodt F & Hawkley L (2015). Journal of Research in Personality. https://doi.org/10.1016/j.jrp.2015.11.003

PDF

Citation count: 46

Paradox of delight -
Weighing joy's worth brings less peace -
Happiness's cost

×

Abstract

NA

Should We Stop Looking for a Better Scoring Algorithm for Handling Implicit Association Test Data? Test of the Role of Errors, Extreme Latencies Treatment, Scoring Formula, and Practice Trials on Reliability and Validity
Richetin J, Costantini G, Perugini M & Schönbrodt F (2015). PLoS ONE. https://doi.org/10.1371/journal.pone.0129601

PDF

Citation count: 117

Hidden biases revealed -
Eliminate, replace, compute -
Accuracy's thin line

×

Abstract

Since the development of D scores for the Implicit Association Test, few studies have examined whether there is a better scoring method. In this contribution, we tested the effect of four relevant parameters for IAT data that are the treatment of extreme latencies, the error treatment, the method for computing the IAT difference, and the distinction between practice and test critical trials. For some options of these different parameters, we included robust statistic methods that can provide viable alternative metrics to existing scoring algorithms, especially given the specificity of reaction time data. We thus elaborated 420 algorithms that result from the combination of all the different options and test the main effect of the four parameters with robust statistical analyses as well as their interaction with the type of IAT (i.e., with or without built-in penalty included in the IAT procedure). From the results, we can elaborate some recommendations. A treatment of extreme latencies is preferable but only if it consists in replacing rather than eliminating them. Errors contain important information and should not be discarded. The D score seems to be still a good way to compute the difference although the G score could be a good alternative, and finally it seems better to not compute the IAT difference separately for practice and test critical trials. From this recommendation, we propose to improve the traditional D scores with small yet effective modifications.

Open Peer Commentary and Authors‘ Response
K. M & M. C (2015). European Journal of Personality. https://doi.org/10.1002/per.2005

PDF

Citation count: 12

Shadows of change creep -
Time's slow river shifts the scene -
Personality flows

×

Abstract

Rauthmann, Sherman and Funder have made a landmark contribution to situation research in the target article of this issue. However, we propose that their work overlooks the need to incorporate a developmental perspective. This includes the separate but related issues of time and change. Situations often unfold over long periods of time, can bleed together, and are not time-delimited in the way traditional laboratory experiments define them. Moreover, individuals systematically change over time (lifespan development) and their reactions to situations, as well as their personality-situation transactions, develop in tandem.

Getting the most out of family data with the R package fSRM.
Stas L, Schönbrodt F & Loeys T (2015). Journal of Family Psychology. https://doi.org/10.1037/fam0000058

PDF Open Material

Citation count: 15

Circles of relation -
Families' complex dance found -
In the web of data

×

Abstract

Family research aims to explore family processes, but is often limited to the examination of unidirectional processes. As the behavior of 1 person has consequences that go beyond that individual, family functioning should be investigated in its full complexity. The social relations model (SRM; Kenny & La Voie, 1984) is a conceptual and analytical model that can disentangle family data from a round-robin design at 3 different levels: the individual level (actor and partner effects), the dyadic level (relationship effects), and the family level (family effect). Its statistical complexity may however be a hurdle for family researchers. The user-friendly R package fSRM performs almost automatically those rather complex SRM analyses and introduces new possibilities for assessing differences between SRM means and between SRM variances, both within and between groups of families. Using family data on negative processes, different type of research questions are formulated and corresponding analyses with fSRM are presented.

Affective contingencies in the affiliative domain: Physiological assessment, associations with the affiliation motive, and prediction of behavior.
Dufner M, Arslan R, Hagemeyer B, Schönbrodt F & Denissen J (2015). Journal of Personality and Social Psychology. https://doi.org/10.1037/pspp0000025

PDF

Citation count: 80

Facial smile reveals -
Deep-seated love for human bond -
Social hearts align

×

Abstract

According to classical motive disposition theory, individuals differ in their propensity to derive pleasure from affiliative experiences. This propensity is considered a core process underlying the affiliation motive and a pervasive cause of motivated behavior. In this study, we tested these assumptions. We presented participants with positive affiliative stimuli and used electromyography to record changes in facial muscular activity that are indicative of subtle smiling. We were thus able to physiologically measure positive affect following affiliative cues. Individual differences in these affective contingencies were internally consistent and temporally stable. They converged with affiliation motive self- and informant reports and picture story exercise scores, indicating that they are partly accessible to the self, observable to outsiders, and overlap with implicit systems. Finally, they predicted affiliative behavior in terms of situation selection and modification across a wide variety of contexts (i.e., in daily life, the laboratory, and an online social network). These findings corroborate the long-held assumption that affective contingencies represent a motivational core aspect of affiliation.

When “together” means “too close”: Agency motives and relationship functioning in coresident and living-apart-together couples.
Hagemeyer B, Schönbrodt F, Neyer F, Neberich W & Asendorpf J (2015). Journal of Personality and Social Psychology. https://doi.org/10.1037/pspi0000031

PDF

Citation count: 49

Together means too close -
Agentic hearts entwine tight -
Love's suffocating grasp

×

Abstract

The present research addresses the interplay between agency motives and objective dyadic closeness with regard to the functioning of intimate couple relationships. Applying a Person × Situation approach, we hypothesized (a) that partners' implicit and explicit agency motives predict their selection of dyadic living arrangements characterized by high or low objective closeness (coresidence or living-apart-together), (b) that agency motives have more negative effects on relationship functioning in coresident couples, (c) that agency motives predict agentic motivational states in coresident couples, and (d) that agentic states predict day-to-day changes in relationship satisfaction under conditions of high objective closeness. We found support for these between- and within-couple hypotheses in cross-sectional and prospective analyses of an age-heterogeneous sample of 548 heterosexual couples, and in a 2-week diary study with a subsample of 106 couples. Most notably, agentic motive dispositions and motivational states related to relationship functioning more negatively under conditions of high objective closeness. The overall positive effect of objective closeness on relationship functioning was diminished by strong agentic motivation. Perspectives for future research on agency motives in couple relationships are discussed.

2014

Not afraid to trust you: Trustworthiness moderates gaze cueing but not in highly anxious participants
Süßenbach F & Schönbrodt F (2014). Journal of Cognitive Psychology. https://doi.org/10.1080/20445911.2014.945457

PDF Open Data Open Material

Citation count: 43

Trusting gaze follows -
Safety in the eyes of friend -
Fearful hearts resist

×

Abstract

AbstractGaze cueing (i.e., the shifting of person B's attention by following person A's gaze) is closely linked with human interaction and learning. To make the most of this connection, researchers need to investigate possible moderators enhancing or reducing the extent of this attentional shifting. In this study we used a gaze cueing paradigm to demonstrate that the perceived trustworthiness of a cueing person constitutes such a moderator for female participants. Our results show a significant interaction between perceived trustworthiness and the response time trade-off between valid and invalid gaze cues [gaze cueing effect (GCE)], as manifested in greater following of a person's gaze if this person was trustworthy as opposed to the following of an untrustworthy person's gaze. An additional exploratory analysis showed potentially moderating influences of trait-anxiety on this interaction (p = .057). The affective background of the experiment (i.e., using positive or negative target stimuli) had no influence.Keywords: Affective backgroundAnxietyGaze cueingModeratorTrustworthiness Supplementary materialSupplementary content is available via the 'Supplementary' tab on the article's online page (http://dx.doi.org/10.1080/20445911.2014.945457).Notes1 This requirement was due to an additional hypothesis in which we predicted that two familiar faces (Aragorn from The Lord of the Rings and Hannibal Lecter from The Silence of the Lambs) would be naturally and more strongly primed trustworthy or untrustworthy, respectively, and thus participants should show an even bigger difference in GCEs when following – or not following – the gaze of these faces. As we had problems finding enough participants who had seen both movies and as this was just an additional exploratory hypothesis, we relaxed this requirement. Thus, we ended up with 48 out of 60 participants who had seen both movies. However, results regarding these characters are not reported here, as they can only be regarded exploratory since these character's faces could neither be counterbalanced with their corresponding trustworthiness nor could we efficiently control for effects of different degrees of familiarity, which might have had an effect (Deaner et al., Citation2007). Nonetheless, the interested reader finds all the obtained results and our complete dataset in the online supplementary material.

Loneliness and social behaviours in a virtual social environment
Luhmann M, Schönbrodt F, Hawkley L & Cacioppo J (2014). Cognition & Emotion. https://doi.org/10.1080/02699931.2014.922053

PDF

Citation count: 21

Reconnecting in space -
Loneliness fuels distant hearts -
Love turns to disconnection

×

Abstract

Feeling lonely motivates people to reconnect with others, but it can also trigger a vicious cycle of cognitions and behaviours that reinforces their loneliness. In this study, we examined the behavioural consequences of loneliness in a virtual social environment. A total of 176 participants navigated a character (protagonist) through a two-dimensional browser game and rated the character's loneliness multiple times during the game. In the first part of the game, another character is introduced as the protagonist's spouse. At one point, the spouse leaves for an undetermined period of time but later returns. Immediately before this separation, higher ascribed loneliness of the protagonist was associated with more frequent interactions with the spouse. After the reunion, however, higher ascribed loneliness was associated with less frequent interactions with the spouse. Ascribed loneliness was not significantly related to the frequency of interactions with others nor to the frequency of solitary activities. These patterns held after controlling for ascribed positive affect. Participants' levels of loneliness were related to the level of ascribed loneliness only when the spouse was present but not when the spouse was absent. In sum, these findings suggest that the conditions that trigger the vicious cycle of loneliness are person- and situation-specific.

Validation of direct and indirect measures of preference for sexualized violence.
Larue D, Schmidt A, Imhoff R, Eggers K, Schönbrodt F & Banse R (2014). Psychological Assessment. https://doi.org/10.1037/pas0000016

PDF

Citation count: 33

Battered desires dark -
Shadows of consent's fine line -
Violence in the heart

×

Abstract

Individuals differ in the extent to which they are interested in sexualized violence, as displayed in the frequent but not ubiquitous sexual interest in consensual acts of violent sexual role play and violent pornographic media in the normal population. The present research sought to develop and validate a multi-method assessment battery to measure individual differences in the preference for sexualized violence. Three indirect measures (Implicit Association Test, Semantic Misattribution Procedure, Viewing Time) were combined in an online study with 107 men and 103 women. Participants with and without an affiliation with sadomasochistic sexual interest groups were recruited on corresponding Internet platforms. Results revealed that all 3 indirect measures converged in predicting self-reported sexual interest in non-consensual sexuality. Specifically, for men all indirect measures were related to non-consensual sadistic sexual interest, whereas for women an association with masochistic sexual interest was found. Stimulus artefacts versus genuine gender differences are discussed as potential explanations of this dissociation. An outlook on the usability of the assessment battery in applied settings is delivered.

2013

At what sample size do correlations stabilize?
Schönbrodt F & Perugini M (2013). Journal of Research in Personality. https://doi.org/10.1016/j.jrp.2013.05.009

PDF Open Material

Citation count: 2244

Statistical waves -
Growing stability at -
Large enough the sea

×

Abstract

NA

2012

An IRT analysis of motive questionnaires: The Unified Motive Scales
Schönbrodt F & Gerstenberg F (2012). Journal of Research in Personality. https://doi.org/10.1016/j.jrp.2012.08.010

PDF

Citation count: 213

Scale measures unfold -
Intrinsic and extrinsic -
Motivation's dance

×

Abstract

NA

2011

TripleR: An R package for social relations analyses based on round-robin designs
Schönbrodt F, Back M & Schmukle S (2011). Behavior Research Methods. https://doi.org/10.3758/s13428-011-0150-4

PDF Open Material

Citation count: 187

Relations round the floor -
TripleR measures all sides -
Social bonds revealed

×

Abstract

NA

Attachment Dynamics in a Virtual World
Schönbrodt F & Asendorpf J (2011). Journal of Personality. https://doi.org/10.1111/j.1467-6494.2011.00736.x

PDF

Citation count: 23

Electronic heartache -
Internal models unfold here -
Love's digital dance

×

Abstract

The study investigated the transfer of attachment working models onto autonomous agents in a virtual social environment (VSE). Participants from a community sample (N = 422; mean age = 29 years; mean relationship duration = 4.8 years) encountered three attachment-related key scenes in a VSE: a separation with a subsequent reunion, a conflict, and an illness. In these scenarios, participants gave instructions to the main character (the "protagonist"), who had a romantic relationship with one of the other agents (the "virtual spouse"). Recorded were numerous behaviors as well as emotions participants ascribed to the protagonist. Generally, participants' attachment styles correlated as predicted with their behavior toward the virtual spouse, with βs up to.42. Thus, the study demonstrates that internal working models are transferred to the virtual relationship. This approach opens new ways to investigate attachment-related behavior by using VSEs that allow experimental variations of the virtual partner's reactions and other situational variables.

PERSOC: A Unified Framework for Understanding the Dynamic Interplay of Personality and Social Relationships
Back M, Baumert A, Denissen J, Hartung F, Penke L, Schmukle S, Schönbrodt F, Schröder–Abé M, Vollmann M, Wagner J & Wrzus C (2011). European Journal of Personality. https://doi.org/10.1002/per.811

PDF

Citation count: 291

Social bonds entwine -
Personality's gentle dance -
Relationships bloom

×

Abstract

The interplay of personality and social relationships is as fascinating as it is complex and it pertains to a wide array of largely separate research domains. Here, we present an integrative and unified framework for analysing the complex dynamics of personality and social relationships (PERSOC). Basic principles and general processes on the individual and dyadic level are outlined to show how personality and social relationships influence each other and develop over time. PERSOC stresses the importance of social behaviours and interpersonal perceptions as mediating processes organized in social interaction units. The framework can be applied to diverse social relationships such as first encounters, short–term acquaintances, friendships, relationships between working group members, educational or therapeutic settings, romantic relationships and family relationships. It has important consequences for how we conceptualize, understand, and investigate personality and social relationships. Copyright © 2011 John Wiley & Sons, Ltd.

The Challenge of Constructing Psychologically Believable Agents
Schönbrodt F & Asendorpf J (2011). Journal of Media Psychology Theories Methods and Applications. https://doi.org/10.1027/1864-1105/a000040

PDF

Citation count: 24

Artificial minds ache -
Fleshing out emotions cold -
Simulating soul

×

Abstract

Embodied conversational agents (ECAs) are designed to provide natural and intuitive communication with a human user. One major current topic in agent design consequently is to enhance their believability, often by incorporating internal models of emotions or motivations. As psychological theories often lack the necessary details for direct implementation, many agent modelers currently rely on models that are rather marginal in current psychological research, or models that are created ad hoc with little theoretical and empirical foundation. The goal of this article is both to raise psychologists’ awareness of the central challenges in the process of creating psychologically believable agents, and to recommend existing psychological frameworks to the virtual agents community that seem particularly useful for implementation in ECAs. Special attention is paid to a computationally detailed model of basic social motives that seems particularly useful for implementation: the Zurich model of social motivation.

2010

Antecedents and Consequences of Peer–Rated Intelligence
Denissen J, Schönbrodt F, Zalk M, Meeus W & Aken M (2010). European Journal of Personality. https://doi.org/10.1002/per.799

PDF

Citation count: 43

Colloquial judgment -
Shadows truth, yet influences -
Academic fate lies

×

Abstract

The current study investigated the antecedents and consequences of peer–rated intelligence in a longitudinal round robin design, following previously unacquainted members of small student work groups. Results indicated that peer–reputations of intelligence were reliable, stable and weakly correlated with objective intelligence. Bias was shown by correlations with interpersonal liking (decreasing across time) and idiosyncratic rating tendencies (increasing across time). Agreement between self–ratings and peer–reputations increased over time but was not based on increasing accuracy but on reciprocal associations between self–ratings and peer–reputations in the beginning of the acquaintanceship process, and on peer–reputations predicting changes in self–ratings later on. Finally, it was shown that peer–rated intelligence reputations predict academic achievement across two 4–month periods (even when tested intelligence was controlled) and dropout from university after 8 months. Overall, the pattern of results demonstrates the utility of a socioanalytic perspective in analysing personality and social processes. Copyright © 2010 John Wiley & Sons, Ltd.