Table · dataset · 2026
<p>Five-run results on CMU-MOSI (mean sample SD).</p>
Listed in HKU DataHub and figshare and Loughborough Research Repository and UP Research Data Repository — shown once because both records carry DOI 10.1371/journal.pone.0359425.t005
<div><p>Language use, vocal behavior, and facial activity provide complementary indicators of affective state, but multimodal models can be affected by noisy temporal observations and insufficient global guidance before cross-modal interaction.
Description
We developed a framework combining bidirectional gated recurrent unit encoders, attention pooling, dynamic hypernode injection, and graph attention fusion. Textual, acoustic, and visual sequences were mapped into a shared latent space and compressed into modality-level representations.
A sample-dependent hypernode and a learnable static prior were then injected through gated residual connections before graph propagation. The model was evaluated on CMU-MOSI and CMU-MOSEI using five random seeds and validation-MAE checkpoint selection. On CMU-MOSI, the model obtained an MAE of , a correlation of , and an Acc-7 of .
Read the rest (2 more)
On CMU-MOSEI, the corresponding results were , , and . A five-run comparison of pre-GAT, post-GAT, and no injection showed that pre-GAT injection provided the strongest overall regression-oriented balance, although post-GAT was slightly higher on MOSEI Acc-7. Gate, graph-attention, representation, and balanced case diagnostics showed that the injected prior was sample dependent and modality selective.
These findings support pre-propagation hypernode injection under complete, word-aligned tri-modal inputs, without establishing universal superiority or robustness to missing and corrupted modalities.</p></div>
Links
Where it is published
- DOI doi.org/10.1371/journal.pone.0359425.t005 ↗
DOI / persistent id · from datahub hku hk
Catalogue records · 1
- OAI-PMH record api.figshare.com/v2/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=oai%3Af… ↗
metadata API · from datahub hku hk
Topics
- From keywords
- Astronomy & Astrophysics · Chemistry · Chemistry · Computer Science & AI · Computer Science & AI · Earth & Environmental Science · Earth & Environmental Science · Earth & Environmental Science · Earth & Environmental Science · Economics & Finance · Economics & Finance · Engineering · Engineering · Engineering · Humanities · Humanities · Humanities · Life Sciences · Life Sciences · Life Sciences · Life Sciences · Materials Science · Mathematics & Statistics · Medicine & Health · Medicine & Health · Ocean & Atmospheric Science · Psychology & Behavioral Science · Psychology & Behavioral Science · Psychology & Behavioral Science · Psychology & Behavioral Science · Research, science and technology policy · Research, science and technology policy · Research, science and technology policy · Research, science and technology policy · Social Science · Social Science · Social Science
Provenance · 4 source records, 42 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| HKU DataHub | oai:figshare.com:article/34017416 | 12 d ago | JSON v1 |
| figshare | oai:figshare.com:article/34017416 | 11 d ago | JSON v1 |
| Loughborough Research Repository | oai:figshare.com:article/34017416 | 11 d ago | JSON v1 |
| UP Research Data Repository | oai:figshare.com:article/34017416 | 11 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · datahub hku hk | connector:datahub_hku_hk@1.0.0 | |
| concepts[field].anzsrc:field:440710 | mapping · figshare com | vocabulary-mapper@1.0.0 | keywords['Science Policy'] |
| concepts[field].anzsrc:field:440710 | mapping · researchdata up ac za | vocabulary-mapper@1.0.0 | keywords['Science Policy'] |
| concepts[field].anzsrc:field:440710 | mapping · datahub hku hk | vocabulary-mapper@1.0.0 | keywords['Science Policy'] |
| concepts[field].anzsrc:field:440710 | mapping · repository lboro ac uk | vocabulary-mapper@1.0.0 | keywords['Science Policy'] |
| concepts[field].local:field:astronomy | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:chemistry | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:chemistry | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:computer-science-ai | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:computer-science-ai | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · datahub hku hk | connector:datahub_hku_hk@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · researchdata up ac za | connector:researchdata_up_ac_za@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:earth-environmental | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:economics-finance | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:economics-finance | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:engineering | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:engineering | mapping · researchdata up ac za | connector:researchdata_up_ac_za@1.0.0 | |
| concepts[field].local:field:engineering | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:humanities | mapping · researchdata up ac za | connector:researchdata_up_ac_za@1.0.0 | |
| concepts[field].local:field:humanities | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:humanities | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · researchdata up ac za | connector:researchdata_up_ac_za@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · datahub hku hk | connector:datahub_hku_hk@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:life-sciences | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:materials-science | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:mathematics-statistics | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:medicine-health | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:medicine-health | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:ocean-atmospheric | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:psychology-behavioral | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:psychology-behavioral | mapping · figshare com | connector:figshare_com@1.0.0 | |
| concepts[field].local:field:psychology-behavioral | mapping · datahub hku hk | connector:datahub_hku_hk@1.0.0 | |
| concepts[field].local:field:psychology-behavioral | mapping · researchdata up ac za | connector:researchdata_up_ac_za@1.0.0 | |
| concepts[field].local:field:social-science | mapping · repository lboro ac uk | connector:repository_lboro_ac_uk@1.0.0 | |
| concepts[field].local:field:social-science | mapping · researchdata up ac za | connector:researchdata_up_ac_za@1.0.0 | |
| concepts[field].local:field:social-science | mapping · figshare com | connector:figshare_com@1.0.0 | |
| description | source · datahub hku hk | connector:datahub_hku_hk@1.0.0 | /metadata/dc/description |
| license | source · datahub hku hk | connector:datahub_hku_hk@1.0.0 | /metadata/dc/rights |
| publication_date | source · datahub hku hk | connector:datahub_hku_hk@1.0.0 | |
| title | source · datahub hku hk | connector:datahub_hku_hk@1.0.0 | /metadata/dc/title |