This model is designed to generate icebreaking quizzes. Icebreaking refers to activities conducted before meetings to help participants break the ice and ease tension by solving simple quizzes. By inputting easy-to-answer questions along with users' responses into the model, it generates new multiple-choice questions with four options and the correct answer. These customized questions can help participants engage with each other and reduce awkwardness in the room.
1. Data Collection
We collected questions and answers from blogs via web crawling and used state-of-the-art models like ChatGPT to generate new four-option multiple-choice questions. We then manually reviewed and edited the generated questions to correct any errors in answer matching and adjusted the difficulty level to make the multiple-choice options more challenging. This process resulted in a well-structured dataset.
Example Data
blog_id
question_crawling
question_number
answer_crawling
question_generated
multiple_choice_generated
answer_generated
gemma
살면서 가장 기억에 남는 사람
95
미술학원 선생님! 내가 미술 쪽에 흥미를 가질 수 있게 도와주신 분이야. 학교 끝나고 학원에 가면 맛난 것도 많이 주시고 좋은 기억들만 만들어주신 분이야.
이 사람이 초등학교 때 가장 기억에 남는 사람은 누구인가요?
1. 초등학교 담임 선생님 2. 중학교 동창 3. 미술학원 선생님 4. 대학교 교수
3
2. Fine-tuning Process
This model was fine-tuned using the Google Gemma 2B instruct model, with LoRA applied to make the training process lighter and faster on TPU. LoRA reduces the number of parameters updated during training, allowing efficient fine-tuning without consuming excessive memory resources. Early stopping was also applied to prevent overfitting.
The training was conducted on a JAX backend using TPU acceleration, distributing the workload across multiple TPU cores for improved efficiency. The model was optimized using the AdamW optimizer, and the loss function was calculated using sparse categorical cross-entropy.
Key Fine-tuning Details:
Sequence Length: 512
Batch Size: 16
LoRA Rank: 8
Learning Rate: 2e-5
Epochs: 20
Train Data Rate: 0.9
Early Stopping Step: 5
Training Method:
The training data consists of "question_crawling" and "answer_crawling" columns as inputs, with "question_generated", "multiple_choice_generated", and "answer_generated" columns as outputs. The data was formatted to fit the fine-tuned model's input structure for training.
The model is built using Keras_NLP's pre-built GemmaCausalLM and applied to the Gemma 2B instruct architecture. We activated LoRA on the decoder blocks and managed the model efficiently across TPU devices via model parallelization. Training was conducted for 20 epochs using TPU resources on Kaggle.
Training Time and Results:
Training ran for 20 epochs with a batch size of 16.
Time per epoch: ~5 minutes (with TPU)
Total training time: ~100 minutes
The model demonstrated high accuracy in quiz generation, achieving approximately 81% accuracy on the validation data.
Input
<instruction>
Using the text: 좋아하는 히어로? 나의 히어로는 우리 아빠, create a new multiple-choice question with 4 answer options.
Model Output
<Response>
가장 좋아하는 히어로는 누구인가요?
1. 나의 아빠 2. 슈퍼맨 3. 캡틴 아메리카 4. 헐크
1
Expected Answer
<Response>
좋아하는 히어로는 누구인가요?
1. 우리 엄마 2. 스파이더맨 3. 우리 아빠 4. 아이언맨
3
When using this model, data must be formatted according to a specific template. The following structure is required for input:
plaintext
1<instruction>
2Using the text: {question_crawling} {answer_crawling}, create a new multiple-choice question with 4 answer options.
The model takes the question_crawling and answer_crawling data to generate new multiple-choice questions. The expected format for the generated output is:
This template includes the newly generated question, multiple-choice options, and the correct answer. When using this template for testing, the fields for question_generated, multiple_choice_generated, and answer_generated are left blank for the model to fill in.
This model has been developed as a service using the Flask framework and Google Survey, allowing real users to utilize it for icebreaking. Users respond to simple questions on Google Survey, and the model generates new questions based on their input. Flask processes the question generation, while a 'conversation topic guide' is provided to facilitate discussions while waiting for the generated questions.
