In the United States nearly 21M adults suffer from depression each year [1], with depression serving as the nation’s leading cause of disability [2]. Despite this, less than 4% of Americans receive mental health screenings from their primary care physicians during annual wellness visits. The pandemic and public campaigns of late have made strides toward positively increasing awareness of mental health struggles, but there remains a persisting stigma around depression and other mental health conditions. The influence of this stigma is especially marked in older adults. People aged 65 and older are less likely than any other age group to seek mental health support. Older adults – for whom depression significantly increases the risk of disability and morbidity – also tend to underreport mental health symptoms [3].

In the US, this outlook becomes even more troubling when coupled with the rate at which the country’s population is aging: 1 out of every 6 people will be 60 years or over by 2030 [4]. As widespread and prevalent as depression is, identifying and treating depression and other mental health conditions remains challenging and there is limited objectivity in the screening processes.

DAM is a clinical-grade, speech-based model designed to screen for signs of depression and anxiety using voice biomarkers. To the best of our knowledge, it is the first model developed explicitly for clinical-grade mental health assessment from speech without reliance on linguistic content or transcription. A predecessor model has been peer-reviewed in the largest voice biomarker study by the Annals of Family Medicine, a leading U.S. Primary Care Journal [5]. The model operates exclusively on the acoustic properties of the speech signal, extracting depression- and anxiety-specific voice biomarkers rather than semantic or lexical information. Numerous studies [6–8] have demonstrated that paralinguistic features – such as spectral entropy, pitch variability, fundamental frequency, and related acoustic measures – exhibit strong correlations with depression and anxiety. Building on this body of evidence, DAM extends prior approaches by leveraging deep learning to learn fine-grained vocal biomarkers directly from the raw speech signal, yielding representations that demonstrate greater predictive power than hand-engineered paralinguistic features. DAM analyzes spoken audio to estimate depression and anxiety severity scores which can be subsequently mapped to standardized clinical scales, such as PHQ-9 (Patient Health Questionnaire-9) for depression and GAD-7 (Generalized Anxiety Disorder-7) for anxiety.

The model was trained and evaluated on a large-scale speech dataset collected from approximately 35,000 individuals via phone, tablet, or web app, which corresponds to ~863 hours of speech data. Ground-truth labels were derived from both clinician-administered and self-reported PHQ-9 and GAD-7 questionnaires, ensuring strong alignment with established clinical assessment standards. The data consists predominantly of American English speech. However, a broad range of accents is represented, providing robustness across diverse speaking styles.

The audio data itself cannot be shared for privacy reasons. Demographic statistics, model scores, and associated metadata for each audio stream are available for threshold tuning at https://huggingface.co/datasets/KintsugiHealth/dam-dataset.

Foundation model: OpenAI Whisper-Small EN

Training approach: Fine-tuning + Multi-task learning

Downstream tasks: Depression and anxiety severity estimation

Whisper serves as the backbone for extracting voice biomarkers, while multi-task head is fine-tuned jointly on depression and anxiety prediction tasks to leverage shared representations across mental health conditions.

Preferred minimum audio length: 30 seconds of speech after Voice Activity Detector

Input modality: Audio only

Shorter audio samples may lead to reduced prediction accuracy.

The model outputs a dictionary of the following form {"depression":score, "anxiety": score}.

If quantized=False (see the Usage section below), the scores are returned as raw float values which correlate monotonically with PHQ-9 and GAD-7.

If quantized=True the scores are converted into integers representing the severity of depression and anxiety.

0 – no depression (PHQ-9 <= 9)

1 – mild to moderate depression (10 <= PHQ-9 <= 14)

2 – severe depression (PHQ-9 >= 15)

0 – no anxiety (GAD-7 <= 4)

1 – mild anxiety (5 <= GAD-7 <= 9)

2 – moderate anxiety (10 <= GAD-7 <= 14)

3 – severe anxiety (GAD-7 >= 15)

The output will resemble a dictionary, for example {'depression': 2, 'anxiety': 3}, indicating that the analyzed audio sample exhibits voice biomarkers consistent with severe depression and severe anxiety.

As mentioned in the Data section above, the raw audio data cannot be shared, but validation and test sets of model scores associated with ground truth and demographic metadata are available for threshold tuning. This way thresholds can be tuned for traditional binary classification, ternary classification with an indeterminate output, and multi-class classification of severity. Two modules are provided for this in the model code's tuning package, as illustrated below.

This module implements a generalization of ROC curve analysis wherein ground truth is binary, but model output can be negative (score below lower threshold), positive (score above upper threshold), or indeterminate (score between thresholds). For the purpose of metric calculations such as sensitivity and specificity, examples marked indeterminate do not count towards either the numerator or denominator. The budget for fraction of examples to be marked indeterminate is configurable as shown below.

The depression and anxiety models were each trained with ordinal regression to predict a scalar score monotonically correlated with the underlying PHQ-9 and GAD-7 questionnaire ground truth sums. As such there are efficient dynamic programming algorithms to select optimal thresholds for multi-class numeric labels under a variety of decision criteria.

This model was created through equal contributions by Oleksii Abramenko, Noah Stein, and Colin Vaz during their work at Kintsugi Health. It builds on years of prior modeling, data collection, clinical research, and operational efforts by a broader team. A full list of contributors is available on the Kintsugi Health organization card at https://huggingface.co/KintsugiHealth.