← Search

Table · dataset · 2026

<p>Five-run results on CMU-MOSI (mean sample SD).</p>

Listed in HKU DataHub and figshare and Loughborough Research Repository and UP Research Data Repository — shown once because both records carry DOI 10.1371/journal.pone.0359425.t005

<div><p>Language use, vocal behavior, and facial activity provide complementary indicators of affective state, but multimodal models can be affected by noisy temporal observations and insufficient global guidance before cross-modal interaction.

Description

We developed a framework combining bidirectional gated recurrent unit encoders, attention pooling, dynamic hypernode injection, and graph attention fusion. Textual, acoustic, and visual sequences were mapped into a shared latent space and compressed into modality-level representations.

A sample-dependent hypernode and a learnable static prior were then injected through gated residual connections before graph propagation. The model was evaluated on CMU-MOSI and CMU-MOSEI using five random seeds and validation-MAE checkpoint selection. On CMU-MOSI, the model obtained an MAE of , a correlation of , and an Acc-7 of .

Read the rest (2 more)

On CMU-MOSEI, the corresponding results were , , and . A five-run comparison of pre-GAT, post-GAT, and no injection showed that pre-GAT injection provided the strongest overall regression-oriented balance, although post-GAT was slightly higher on MOSEI Acc-7. Gate, graph-attention, representation, and balanced case diagnostics showed that the injected prior was sample dependent and modality selective.

These findings support pre-propagation hypernode injection under complete, word-aligned tri-modal inputs, without establishing universal superiority or robustness to missing and corrupted modalities.</p></div>

Links

Where it is published

Catalogue records · 1

Topics

Provenance · 4 source records, 42 field assertions
SourceKeyLast seenRaw
HKU DataHuboai:figshare.com:article/3401741612 d agoJSON v1
figshareoai:figshare.com:article/3401741611 d agoJSON v1
Loughborough Research Repositoryoai:figshare.com:article/3401741611 d agoJSON v1
UP Research Data Repositoryoai:figshare.com:article/3401741611 d agoJSON v1
FieldAssertionExtractorEvidence
access_levelsource · datahub hku hkconnector:datahub_hku_hk@1.0.0
concepts[field].anzsrc:field:440710mapping · figshare comvocabulary-mapper@1.0.0keywords['Science Policy']
concepts[field].anzsrc:field:440710mapping · researchdata up ac zavocabulary-mapper@1.0.0keywords['Science Policy']
concepts[field].anzsrc:field:440710mapping · datahub hku hkvocabulary-mapper@1.0.0keywords['Science Policy']
concepts[field].anzsrc:field:440710mapping · repository lboro ac ukvocabulary-mapper@1.0.0keywords['Science Policy']
concepts[field].local:field:astronomymapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:chemistrymapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:chemistrymapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:computer-science-aimapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:computer-science-aimapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:earth-environmentalmapping · datahub hku hkconnector:datahub_hku_hk@1.0.0
concepts[field].local:field:earth-environmentalmapping · researchdata up ac zaconnector:researchdata_up_ac_za@1.0.0
concepts[field].local:field:earth-environmentalmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:earth-environmentalmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:economics-financemapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:economics-financemapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:engineeringmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:engineeringmapping · researchdata up ac zaconnector:researchdata_up_ac_za@1.0.0
concepts[field].local:field:engineeringmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:humanitiesmapping · researchdata up ac zaconnector:researchdata_up_ac_za@1.0.0
concepts[field].local:field:humanitiesmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:humanitiesmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:life-sciencesmapping · researchdata up ac zaconnector:researchdata_up_ac_za@1.0.0
concepts[field].local:field:life-sciencesmapping · datahub hku hkconnector:datahub_hku_hk@1.0.0
concepts[field].local:field:life-sciencesmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:life-sciencesmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:materials-sciencemapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:mathematics-statisticsmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:medicine-healthmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:medicine-healthmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:ocean-atmosphericmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:psychology-behavioralmapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:psychology-behavioralmapping · figshare comconnector:figshare_com@1.0.0
concepts[field].local:field:psychology-behavioralmapping · datahub hku hkconnector:datahub_hku_hk@1.0.0
concepts[field].local:field:psychology-behavioralmapping · researchdata up ac zaconnector:researchdata_up_ac_za@1.0.0
concepts[field].local:field:social-sciencemapping · repository lboro ac ukconnector:repository_lboro_ac_uk@1.0.0
concepts[field].local:field:social-sciencemapping · researchdata up ac zaconnector:researchdata_up_ac_za@1.0.0
concepts[field].local:field:social-sciencemapping · figshare comconnector:figshare_com@1.0.0
descriptionsource · datahub hku hkconnector:datahub_hku_hk@1.0.0/metadata/dc/description
licensesource · datahub hku hkconnector:datahub_hku_hk@1.0.0/metadata/dc/rights
publication_datesource · datahub hku hkconnector:datahub_hku_hk@1.0.0
titlesource · datahub hku hkconnector:datahub_hku_hk@1.0.0/metadata/dc/title