K2-geoscience-7B-4bit-mlx
This is a 4-bit
MLX quantization of
K2 (
daven3/k2), the first open-source foundation language model for the
geoscience domain, converted for fast local inference on Apple Silicon.
K2 is a 7B model built by further-pretraining LLaMA-7B on ~5.5B tokens of geoscience literature (earth-science papers and Wikipedia) and instruction-tuning with the GeoSignal dataset. See the original work for full details:
Deng et al.,
K2: A Foundation Language Model for Geoscience Knowledge
Understanding and Utilization, WSDM 2024.
Paper:
https://arxiv.org/abs/2306.05064 · Code:
https://github.com/davendw49/k2
Conversion details
- Method:
mlx_lm.convert with -q --q-bits 4
- Result: ~4.5 bits per weight
- Precision of base load: float16
- Converted on: Apple Silicon (MLX)
- Tokenizer: the
daven3/k2 repo does not ship a tokenizer. K2 uses the unmodified LLaMA-7B SentencePiece tokenizer (vocab size 32000), so the standard LLaMA tokenizer files are bundled here for a self-contained model. Credit to the LLaMA authors for the tokenizer.
Usage
1from mlx_lm import load, generate
2
3model, tokenizer = load("xquantize/K2-geoscience-7B-4bit-mlx")
4
5# K2 (this release) is a BASE model — prompt it with completion-style text,
6# not chat turns.
7prompt = "Plate tectonics is the scientific theory that"
8print(generate(model, tokenizer, prompt=prompt, max_tokens=120, verbose=True))
Note on usage style
This is the base / further-pretrained K2 model, not an instruction-tuned chat model. It continues text rather than answering questions conversationally. For best results, give it a sentence to continue (e.g. "P-waves and S-waves differ in that...") rather than a chat-style question. Output can be repetitive, as is typical for 7B base models of this generation.
Performance
On an Apple Silicon Mac, this 4-bit build runs at ~34 tokens/sec with a peak memory footprint of ~4 GB, making a geoscience-specialized 7B usable on consumer hardware.
Limitations
K2 is a 2023/2024-era LLaMA-7B derivative; its general fluency and knowledge reflect that generation, not current frontier models. It may produce inaccurate or outdated statements and should not be relied on for authoritative scientific or safety-critical decisions. Quantization to 4 bits may introduce additional minor quality degradation relative to the full-precision model.
License & attribution
The base model K2 is distributed under the LLaMA license (LLaMA-2 community license terms apply to redistribution of LLaMA-derived weights). All credit for the model belongs to the original K2 authors (Deng et al., davendw49/k2). This repository only provides an MLX-format 4-bit quantization for local inference and claims no ownership of the underlying model.
