Kurate by Kivira
Research quality assurance at portfolio scaleThe SNSF already judges the rigor a project proposes. Now it can measure the rigor it delivered.
The SNSF selects the best proposals through competitive peer review and, since adopting DORA, judges research output by its content rather than by journal metrics. Methodological rigor — whether a funded study used a valid comparator, held to its prespecified endpoints, and drew conclusions its design could support — is a distinct axis, and one no funder has been able to measure at scale, because appraising it took expert-hours per study. When Kurate graded a corpus of trials drawn from higher-quality domains, the spread across that axis was wide.
Even in a sample drawn from higher-quality domains, grades span the full range — the same wide spread found across the published literature generally, not a property of any one funder's choices. What's new isn't the spread; it's that it's now measurable. Once it is, a funder can see which work clears the bar for confident interpretation (A–C) and direct more of the portfolio toward it.
Kurate's premise doesn't rest on our numbers alone. A large, independent metascience literature has documented that questionable research practices are common — outcomes switched between registration and publication, selective reporting, underpowered designs, p-hacking and HARKing — leaving a public record in which a study can look authoritative and still not support its own conclusions. Public and philanthropic funding flows into that record. Kurate turns decades of this evidence into a check that runs at portfolio scale.
Kurate is built by researchers who publish in statistics, research methodology and metascience — including in Nature, NeurIPS and ICLR. Members of the team have separately built the peer-reviewed precursors to Kurate's core method — automated comparison of trial registrations against published results. That background shapes how the tool handles fragile claims, uncertain comparisons and noisy literature.
Research in causal inference, deep generative modelling and multimodal ML; 50+ peer-reviewed publications including ICLR and NeurIPS.
Two doctorates — Engineering (Vision, Speech & Signal Processing), University of Surrey; and Applied Mathematics for the Human & Social Sciences, University of Lausanne. Affiliations: University of Lausanne, University of Surrey, and the Sense Center for Innovation and Research.
Contributes to Kurate's evidence-evaluation methodology and clinical rubric development; 50+ peer-reviewed publications, including three metascience papers in Nature.
Expertise in research-integrity assessment and LLM-workflow evaluation; author of published registration-versus-report comparison tools.
How Kurate works
Kurate reconstructs each study's intended design from its own paper trail, then measures the published report against it — the work a methodologist and a statistician would do together, applied uniformly at scale. Two documents in; one auditable verdict out.
One rubric, applied identically to every study — reproducible, auditable, and fast enough to run across a whole portfolio.
Where Kurate fits
The SNSF already scores rigor at evaluation and, under DORA, judges output by content rather than metrics. The one question neither answers: did the funded work deliver the rigor it proposed? Kurate is built for exactly that gap — complementing the mechanisms already in place, not replacing them.
Is the proposed science rigorous?
SNSF peer review rates each proposal on scientific criteria — including the suitability and feasibility of the methods — on a nine-point scale, following the SNSF's principles and the DORA declaration. Assessed on the proposal, before any data exists.
Is the output more than a metric?
Having adopted DORA, the SNSF stopped using journal impact factors in evaluation in 2020 — rightly, since they measure attention, not method. That removes the flawed proxy, but leaves the quality of the delivered work unmeasured.
Did the funded work deliver the rigor it proposed?
No portfolio-scale measure closes this loop today. Kurate compares each published study against its own pre-registration, across the whole portfolio — the delivered-rigor counterpart to the rigor the SNSF scores at evaluation, and DORA-aligned by construction: it reads method, not metrics.
The closest existing checks — clinical-trial registration in the Swiss National Clinical Trials Portal (SNCTP) under the Human Research Act, and the SNSF's Open Research Data policy, which requires a Data Management Plan for every funded project — already require that trials, protocols and data be registered and shared. Neither verifies that reported outcomes match what was prespecified, and the SNSF notes its Data Management Plan undergoes no scientific evaluation. That comparison is exactly what Kurate performs — automatically, across the whole portfolio.
What it lets the SNSF do
Across roughly 2,400 grants a year — a CHF 1.2 billion portfolio spanning 6,000-plus active projects and 23,000 researchers — Kurate grades every study on one rubric in a single pass, turning rigor from something you could only sample by hand into a portfolio-wide layer alongside the evaluation the SNSF already runs. What that puts within reach:
See which schemes and calls actually deliver rigor
Rank funding schemes, programmes and calls by the methodological quality of the work they produce — so the SNSF can direct more funding toward the ones that consistently deliver, and give the ones that lag the evidence to sharpen their guidance or evaluation criteria. The same lens resolves to the researcher level where that's useful — surfacing where rigor support would help most, not a blacklist. Grounded in method, not reputation, and defensible because every grade is auditable.
Track portfolio quality over time
Watch methodological quality across a funding scheme, a discipline, or a decade — and test whether a given call raises or lowers the rigor of the work it produces.
Strengthen the schemes that need it
See which funding schemes and evaluation panels produce the most rigorous work — and where adjusted guidance would raise quality the most.
Report defensible rigor metrics
Give the Research Council, the federal government, and the public a reproducible measure of research quality that complements existing evaluation instead of competing with it.
See a field's quality evolve
Track whether rigor in a domain is improving or eroding over years — the kind of question that shapes where a new call or initiative should be aimed.
How to read the rigor
- Registration-anchored. Quality is judged against the study's own pre-data-collection commitments, not a reviewer's taste.
- GRADE-like, without pool contamination. Comparable in spirit to a Cochrane/GRADE appraisal, but dealbreaker studies are excluded rather than downweighted — so one flawed study can't quietly dilute a synthesis.
- Cross-domain in one pass. Clinical, methodological and statistical checks applied uniformly — precisely where human review is scarce, slow, and inconsistent.
- Reproducible & auditable. The same rubric, applied the same way, every time — a property no distributed panel of reviewers can guarantee, and the basis for defensible portfolio metrics.
- Scope-honest. Strongest in medication, treatment and RCT-relevant domains, where the registration-versus-report comparison is most decisive.
See Kurate for yourself
The figures in this brief are illustrative. To see the real thing, ask us for a demo — we'll walk you through Kurate on live published research: the methodological grade, the dealbreaker flags, and the detailed analysis behind each grade.
About the figures. The grade distribution is Kurate's own result on a corpus drawn from higher-quality research domains; it is presented as a floor, not a random portfolio sample, and does not yet establish how SNSF-funded output is distributed. The SNSF budget (CHF 1.2 billion invested in 2025 across roughly 2,400 projects; ~6,200 active projects and ~23,000 researchers) is from SNSF reporting. SNSF mechanisms referenced — competitive peer-review evaluation on a nine-point scale, the adoption of the DORA declaration (journal impact factors dropped from evaluation in 2020), the Open Research Data policy and its Data Management Plan requirement, and clinical-trial registration in the Swiss National Clinical Trials Portal (SNCTP) under the Human Research Act — are per SNSF and kofam / Federal Office of Public Health documentation. The Project Funding cap (CHF 250,000 per applicant per year) is from SNSF; the savings floor scales the CHF 1.2 billion base by an illustratively small share and is a lower-bound sketch, not a projection. Kurate is decision-support for research quality assessment and does not replace clinical, regulatory, or institutional review. Kurate is built by Kivira Health — kivira.health.
Selected related publications
Methodological and meta-scientific work associated with the Kurate team.
- Vowels, M. J. (2023). Misspecification and unreliable interpretations in psychology and social science. Psychological Methods, 28(3), 507–526. DOI
- Vowels, M. J., Vowels, L. M., & Wood, N. D. (2023). Spectral and cross-spectral analysis: A tutorial for psychologists and social scientists. Psychological Methods, 28(3), 631–650. DOI
- Vowels, M. J. (2023). Prespecification of structure for the optimization of data collection and analysis. Collabra: Psychology, 9(1), Article 71300. DOI
- Vowels, M. J. (2024). Trying to outrun causality with machine learning: Limitations of model explainability techniques for exploratory research. Psychological Methods. DOI
- Vowels, M. J. (2025). A causal research pipeline and tutorial for psychologists and social scientists. Psychological Methods. DOI
- Vowels, M. J. (2024). Typical yet unlikely and normally abnormal: The intuition behind high-dimensional statistics. Statistics, Politics and Policy, 15(1), 87–113. DOI
- Aczel, B., Szaszi, B., Clelland, H. T., Kovacs, M., Holzmeister, F., et al. (2026). Investigating the analytical robustness of the social and behavioural sciences. Nature, 652(8108), 135–142. DOI
- Higgins, W. C., Clarke, B., Elson, M., & Cummins, J. (2026). Recommendations for incorporating LLMs into psychological research: A commentary on Austin and colleagues (2026). PsyArXiv. DOI
- Ahnström, L., Bruckner, T., Aspromonti, D. A., Caquelin, L., Cummins, J., et al. (2026). TrialScout links published results to trial registrations using a large language model. medRxiv. DOI
- Elson, M., Hussey, I., Clarke, B., Norwood, S. F., Grinschgl, S., Arslan, R. C., et al. (2026). Against anonymising meta-scientific data. PsyArXiv. DOI
- Cummins, J., Clarke, B., Hussey, I., & Elson, M. (2026). RegCheck: A tool for automating comparisons between study registrations and papers. arXiv. DOI
- Miske, O., Abatayo, A. L., Daley, M., Dirzo, M., Fox, N., Haber, N., Hahn, K. M., et al. (2026). Investigating the reproducibility of the social and behavioural sciences. Nature, 652(8108), 126–134. DOI
- Cummins, J. (2025, September 1). Psychology needs… an AI revolution. The Psychologist. Article
- Röseler, L., Kaiser, L., Doetsch, C., Klett, N., Seida, C., Schütz, A., Aczel, B., et al. (2024). The Replication Database: Documenting the replicability of psychological science. Journal of Open Psychology Data, 12(1), Article 8. DOI
- Tierney, W., Hardy, J. H., III, Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., Hiring Decisions Forecasting Collaboration, & Uhlmann, E. L. (2020). Creative destruction in science. Organizational Behavior and Human Decision Processes, 161, 291–309. DOI
- Van Dessel, P., Cummins, J., Hughes, S. J., Kasran, S., Cathelyn, F., & Moran Yorovich, T. (2020). Reflecting on twenty-five years of research using implicit measures: Recommendations for their future use. Social Cognition, 38(Supplement), S223–S242. DOI