Airin-chan/

LatenConnectionTransformers

LCT is Transformers variant model that replaced Feed forward network with LCM , how its work ? LCM looking attention as upgraded signal by step1 step2 for looking attention. and then, LCM will do residural connection (attention (input), laten (output)) for making reached attention signal.

LCT look processing following:

embedding => LCT Block (Attention + LCM ) xN sub model => FFN as decoder => Linear

bash

  import LCT_architecture
  model = keras.models.load(LCt-Tiny-version.keras)

About LCM: https://zenodo.org/records/17501400?token=eyJhbGciOiJIUzUxMiJ9.eyJpZCI6Ijk2ZmJmNDg3LWI3MTYtNDVlNy05OWEzLTRiOTZkNGFhOTkzMyIsImRhdGEiOnt9LCJyYW5kb20iOiJiZTNhZjBmMGJmN2NmN2EyNWYyMzRiZWI3MjJkMjcwZCJ9.1Tcsiz_aRDDHbR2MmUdf2MkcUPbyKsI88dRGsv1O3MpA-dxBMk7B4JiSvfwk0RKG9SBzV7WGHY3mnth_iEwhTg

support LCT: https://ko-fi.com/alpin92578

Candra Alpin gunawan

this model trained by 3K Conversations Dataset for ChatBot from kaggle user Kreesh Rajani datasets link: https://www.kaggle.com/datasets/kreeshrajani/3k-conversations-dataset-for-chatbot