Views
No views yet
emrekuruu/job-searcher-qwen3-8B
(the safetensors original). Distilled from DeepSeek V4 Pro onto Qwen3-8B.| File | Size | Purpose |
|---|---|---|
Qwen3-8B-Q4_K_M.gguf | ~5 GB | Base model, q4_K_M quantized |
query_gen.lora.gguf | ~100 MB | LoRA: resume → LinkedIn search queries |
fit_eval.lora.gguf | ~100 MB | LoRA: (resume, job) → 5×20-pt fit score + reasoning |
llama-cpp-python)1from llama_cpp import Llama
2from huggingface_hub import hf_hub_download
3
4repo = "emrekuruu/job-search-gguf"
5base = hf_hub_download(repo, "Qwen3-8B-Q4_K_M.gguf")
6q_lora = hf_hub_download(repo, "query_gen.lora.gguf")
7e_lora = hf_hub_download(repo, "fit_eval.lora.gguf")
8
9llm = Llama(
10 model_path=base,
11 n_gpu_layers=-1,
12 n_ctx=16384,
13 lora_paths=[q_lora, e_lora],
14 lora_scales=[1.0, 0.0], # query_gen ON, fit_eval OFF
15 verbose=False,
16)
17
18# Switch tasks by re-scaling the adapters:
19# llm.set_lora_scale(0, 0.0); llm.set_lora_scale(1, 1.0) # → fit_eval
20
21resp = llm.create_chat_completion(
22 messages=[
23 {"role": "system", "content": "..."},
24 {"role": "user", "content": "..."},
25 ],
26 response_format={"type": "json_object"},
27 max_tokens=4096,
28)
29print(resp["choices"][0]["message"]["content"])Qwen/Qwen3-8B, quantized to q4_K_M via llama.cpp's convert_hf_to_gguf.py.emrekuruu/job-search-distill,
then converted with llama.cpp's convert_lora_to_gguf.py.emrekuruu/job-search-lora.