You can deploy this model and start earning money today!

poisson-fish/
Marcoroni-70B-v1-AWQ

Name: Marcoroni-70B-v1-AWQ
Availability: InStock

Views

No views yet

This is AIDC-ai-business/Marcoroni-70B-v1 quantized to LMDeploy 4bit AWQ with the following config:

bash

python3 -m lmdeploy.lite.apis.auto_awq \
  --model ./Marcoroni-70B-v1 \
  --w_bits 4 \
  --w_group_size 128 \
  --work_dir ./quant

Original Model Card:

Trained by: trained by AIDC AI-Business.
Model type: Marcoroni-70B is an auto-regressive language model based on the Llama 2 transformer architecture.
Language(s): English
License for Marcoroni-70B base weights: Non-Commercial Creative Commons license (CC BY-NC-4.0)

### Instruction:

<prompt> (without the <>)

### Response: