CardioNetV2
The latest multi-modal model in the Cardio Sonix line.
Built on the basis of models whose architectures were originally
intended for computer vision tasks
(like a modified ResNet) or for NLP (like LSTM).
The model works with audio signal and tabular data.
The model works with the input audio signal as with tokens:
a mel-kesprogram with time samples is extracted from the audio,
where each time sample has N-mel-cepstral coefficients.
At the very beginning, the LSTM takes a mel-cepstrogram as input
and produces an output tensor that goes into ResNet (Residual Neural Network).
ResNet is a modified audio signal processing model from the family of residual networks.
In this implementation, residual blocks with pre-activation were used.
The data then goes to the DenseMixer input.
This model performs inference separately for audio and tabular features,
then concatenates the outputs into a dense feature vector and performs inference on it,
after which we get a prediction based on audio and tabular data
