Overview

⚙️ Model Configuration

• Model: Llama-3.3-70B-Instruct
• Parameters: 70 billion
• Quantization: 4-bit GPTQ
• Source: Hugging Face (hf)
• Precision: torch.float16
• Hardware: NVIDIA A100 80GB PCIe
• CUDA Version: 12.4
• PyTorch Version: 2.6.0+cu124
• Batch Size: 1

• The evaluation was performed on a high-performance GPU (A100 80GB).
• The model is significantly larger than the previous 8B version, with GPTQ 4-bit quantization reducing memory footprint.
• A single-sample batch size was used, which might slow evaluation speed.