You can deploy this model and start earning money today!

phuongnm94/
BiosERC

Name: BiosERC
Availability: InStock

safetensors

Views

No views yet

BiosERC: Integrating Biography Speakers Supported by LLMs for ERC Tasks

In the Emotion Recognition in Conversation task, recent investigations have utilized attention mechanisms exploring relationships among utterances from intra- and inter-speakers for modeling emotional interaction between them. However, attributes such as speaker personality traits remain unexplored and present challenges in terms of their applicability to other tasks or compatibility with diverse model architectures. Therefore, this work introduces a novel framework named BiosERC, which investigates speaker characteristics in a conversation. By employing Large Language Models (LLMs), we extract the ``biographical information'' of the speaker within a conversation as supplementary knowledge injected into the model to classify emotional labels for each utterance. Our proposed method achieved state-of-the-art (SOTA) results on three famous benchmark datasets: IEMOCAP, MELD, and EmoryNLP, demonstrating the effectiveness and generalization of our model and showcasing its potential for adaptation to various conversation analysis tasks.

Full paper here: https://link.springer.com/chapter/10.1007/978-3-031-72344-5_19

Results

Performance comparison between our proposed method and previous works on the test sets.


Methods	IEMOCAP	EmoryNLP	MELD
HiTrans	64.50	36.75	61.94
DAG	68.03	39.02	63.65
DialogXL	65.94	34.73	62.14
DialogueEIN	68.93	38.92	65.37
SGED + DAG-ERC	68.53	40.24	65.46
S+PAGE	68.93	40.05	64.67
InstructERC +(ft LLM)	71.39	41.39	69.15

Intra/inter ERC (baseline) ${[AccWR]}_{MLP}$	67.65	39.33	64.58
BiosERC $_{ BERT-based}$	67.79	39.89	65.51
BiosERC +ft LLM $_{Llama-2-7b}$	69.02	41.44	68.72
BiosERC +ft LLM $_{Llama-2-13b}$	71.19	41.68	69.83

Data

unzip the file data.zip to extract data.

IEMOCAP Data structure examples:
json
1{ 2 # this is first conversation 3 "Ses05M_impro03": { 4 "labels": [ 5 4, 6 2, 7 4, 8 4 9 ], 10 "sentences": [ 11 "Guess what?", 12 "what?", 13 "I did it, I asked her to marry me.", 14 "Yes, I did it." 15 ], 16 "genders": [ 17 "M", 18 "F", 19 "M", 20 "M", 21 "F", 22 ] 23 }, 24 25 # this is second conversation 26 "Ses05M_impro03": { 27 "labels": [ 28 4, 29 2, 30 ], 31 "sentences": [ 32 "Guess what?", 33 "what?", 34 ], 35 "genders": [ 36 "M", 37 "F", 38 ] 39 } 40}

Python ENV

Init python environment

cmd

    conda create --prefix=./env_py38  python=3.9
    conda activate ./env_py38 
    pip install -r requirements.txt

Run

Init environment follow the above step.

Data peprocessing.

Put all the raw data to the folder data/. The overview of data structure:

    .
    ├── data/
    │   ├── meld.valid_spdescV2_Llama-2-70b-chat-hf.json # speaker biography will be generated by run `python src/llm_bio_extract.py`
    │   ├── meld.train_spdescV2_Llama-2-70b-chat-hf.json # speaker biography will be generated by run `python src/llm_bio_extract.py`
    │   ├── meld.test_spdescV2_Llama-2-70b-chat-hf.json  # speaker biography will be generated by run `python src/llm_bio_extract.py`
    │   ├── meld.test.json
    │   ├── meld.train.json
    │   ├── meld.valid.json
    │   ├── ...
    │   ├── iemocap.test.json
    │   ├── iemocap.train.json
    │   └── iemocap.valid.json
    ├── src/
    ├── finetuned_llm/
    └── ...

Train
Run following command to train a new model.
bash
1python src/llm_bio_extract.py # to extract speaker bio 2bash scrips/train_llm.sh # to train a llm model

Note: Please check this scripts to check the setting and choose which data you want to run.

Citation

bibtex

@InProceedings{10.1007/978-3-031-72344-5_19,
    author="Xue, Jieying
    and Nguyen, Minh-Phuong
    and Matheny, Blake
    and Nguyen, Le-Minh",
    editor="Wand, Michael
    and Malinovsk{\'a}, Krist{\'i}na
    and Schmidhuber, J{\"u}rgen
    and Tetko, Igor V.",
    title="BiosERC: Integrating Biography Speakers Supported by LLMs for ERC Tasks",
    booktitle="Artificial Neural Networks and Machine Learning -- ICANN 2024",
    year="2024",
    publisher="Springer Nature Switzerland",
    address="Cham",
    pages="277--292",
    abstract="In the Emotion Recognition in Conversation task, recent investigations have utilized attention mechanisms exploring relationships among utterances from intra- and inter-speakers for modeling emotional interaction between them. However, attributes such as speaker personality traits remain unexplored and present challenges in terms of their applicability to other tasks or compatibility with diverse model architectures. Therefore, this work introduces a novel framework named BiosERC, which investigates speaker characteristics in a conversation. By employing Large Language Models (LLMs), we extract the ``biographical information'' of the speaker within a conversation as supplementary knowledge injected into the model to classify emotional labels for each utterance. Our proposed method achieved state-of-the-art (SOTA) results on three famous benchmark datasets: IEMOCAP, MELD, and EmoryNLP, demonstrating the effectiveness and generalization of our model and showcasing its potential for adaptation to various conversation analysis tasks. Our source code is available at https://github.com/yingjie7/BiosERC.",
    isbn="978-3-031-72344-5"
}

Licensing Information

MELD: Licensed under GPL-3.0.
EMORYNLP: Licensed under Apache 2.0.
IEMOCAP: Licensed under a non-commercial research license. Refer to the official IEMOCAP website for terms of use.

phuongnm94/BiosERC