This model is a fine-tuned version of Qwen/Qwen1.5-MoE-A2.7B optimized for generating synthetic survey responses across multiple domains. It has been instruction-tuned using a custom dataset of survey responses, with each response reflecting a specific persona.

Training Data

Dataset Size: ~3,000 examples
Domains: 10 domains including healthcare, education, etc.
Format: ChatML instruction format with system and user prompts

Training Details

Base Model: Qwen/Qwen1.5-MoE-A2.7B
Training Method: Parameter-Efficient Fine-Tuning with LoRA
LoRA Parameters: r=16, alpha=32, dropout=0.05
Training Setup:
- Batch Size: 8
- Learning Rate: 0.0002
- Epochs: 5

Usage

This model is specifically designed for generating synthetic survey responses from different personas. It works best when provided with:

A detailed persona description
A specific survey question

Python Example

python

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("aryashah00/survey-finetuned-Qwen1.5-MoE-A2.7B", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("aryashah00/survey-finetuned-Qwen1.5-MoE-A2.7B", trust_remote_code=True)

# Define persona and question
persona = "A nurse who educates the child about modern medical treatments and encourages a balanced approach to healthcare"
question = "How often was your pain well controlled during this hospital stay?"

# Prepare prompts
system_prompt = f"You are embodying the following persona: {{persona}}"
user_prompt = f"Survey Question: {{question}}\n\nPlease provide your honest and detailed response to this question."

# Create message format
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_prompt}
]

# Apply chat template
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# Tokenize
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to(model.device)

# Generate response
import torch
with torch.no_grad():
    output_ids = model.generate(
        input_ids=input_ids,
        max_new_tokens=256,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )

# Decode
output = tokenizer.decode(output_ids[0], skip_special_tokens=True)

# Extract just the generated response
response_start = output.find(input_text) + len(input_text)
generated_response = output[response_start:].strip()

print(f"Generated response: {{generated_response}}")

Inference API Example

python

import requests

API_URL = "https://api-inference.huggingface.co/models/aryashah00/survey-finetuned-Qwen1.5-MoE-A2.7B"
headers = {"Authorization": "Bearer YOUR_API_KEY"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

messages = [
    {"role": "system", "content": "You are embodying the following persona: A nurse who educates the child about modern medical treatments and encourages a balanced approach to healthcare"},
    {"role": "user", "content": "Survey Question: How often was your pain well controlled during this hospital stay?\n\nPlease provide your honest and detailed response to this question."}
]

output = query({"inputs": messages})
print(output)

Limitations

The model is optimized for survey response generation and may not perform well on other tasks
Response quality depends on the clarity and specificity of the persona and question
The model may occasionally generate responses that don't fully align with the given persona

License

This model follows the license of the base model Qwen/Qwen1.5-MoE-A2.7B.