You can deploy this model and start earning money today!

RichardErkhov/
knowledgator_-_Qwen-encoder-1.5B-gguf

Name: knowledgator_-_Qwen-encoder-1.5B-gguf
Availability: InStock

Views

No views yet

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Qwen-encoder-1.5B - GGUF

Model creator: https://huggingface.co/knowledgator/
Original model: https://huggingface.co/knowledgator/Qwen-encoder-1.5B/

Name	Quant method	Size
Qwen-encoder-1.5B.Q2_K.gguf	Q2_K	0.63GB
Qwen-encoder-1.5B.IQ3_XS.gguf	IQ3_XS	0.68GB
Qwen-encoder-1.5B.IQ3_S.gguf	IQ3_S	0.71GB
Qwen-encoder-1.5B.Q3_K_S.gguf	Q3_K_S	0.71GB
Qwen-encoder-1.5B.IQ3_M.gguf	IQ3_M	0.72GB
Qwen-encoder-1.5B.Q3_K.gguf	Q3_K	0.77GB
Qwen-encoder-1.5B.Q3_K_M.gguf	Q3_K_M	0.77GB
Qwen-encoder-1.5B.Q3_K_L.gguf	Q3_K_L	0.82GB
Qwen-encoder-1.5B.IQ4_XS.gguf	IQ4_XS	0.84GB
Qwen-encoder-1.5B.Q4_0.gguf	Q4_0	0.87GB
Qwen-encoder-1.5B.IQ4_NL.gguf	IQ4_NL	0.88GB
Qwen-encoder-1.5B.Q4_K_S.gguf	Q4_K_S	0.88GB
Qwen-encoder-1.5B.Q4_K.gguf	Q4_K	0.92GB
Qwen-encoder-1.5B.Q4_K_M.gguf	Q4_K_M	0.92GB
Qwen-encoder-1.5B.Q4_1.gguf	Q4_1	0.95GB
Qwen-encoder-1.5B.Q5_0.gguf	Q5_0	1.02GB
Qwen-encoder-1.5B.Q5_K_S.gguf	Q5_K_S	1.02GB
Qwen-encoder-1.5B.Q5_K.gguf	Q5_K	1.05GB
Qwen-encoder-1.5B.Q5_K_M.gguf	Q5_K_M	1.05GB
Qwen-encoder-1.5B.Q5_1.gguf	Q5_1	1.1GB
Qwen-encoder-1.5B.Q6_K.gguf	Q6_K	1.19GB
Qwen-encoder-1.5B.Q8_0.gguf	Q8_0	1.53GB

Original model description:

license: apache-2.0 datasets:

wikimedia/wikipedia language:
en library_name: transformers tags:
LLM2Vec
encoder
LLM
classification
NER
question-answering

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

LLM2Vec is a simple recipe to convert decoder-only LLMs into text encoders. It consists of 3 simple steps: 1) enabling bidirectional attention, 2) masked next token prediction, and 3) unsupervised contrastive learning. The model can be further fine-tuned to achieve state-of-the-art performance.

Repository: https://github.com/McGill-NLP/llm2vec
Paper: https://arxiv.org/abs/2404.05961

Overview:

This is a bi-directional version of Qwen2-1.5B trained with masked token prediction on the Wikipedia dataset. Modern decoder models offer several advantages over classical encoders like BERT:

They are pre-trained on more recent textual corpora They are trained on larger and more diverse datasets Modern decoders have better support for long-context windows Flash-attention support is available for these models

Considering these benefits, we are excited to release a series of decoder models tuned to work in a bi-directional setting. This approach combines the strengths of modern decoder architectures with the versatility of bi-directional context understanding, potentially opening up new possibilities for various natural language processing tasks, such as NER.

In comparison to original LLM2Vec we trained all weights of LLama model, it potentially improve bi-directional abilities of the model.

Installation

pip install llm2vec

Usage

python

from llm2vec.models import Qwen2BiModel

import torch
from transformers import AutoTokenizer

# Loading base Mistral model, along with custom code that enables bidirectional connections in decoder-only LLMs. MNTP LoRA weights are merged into the base model.
tokenizer = AutoTokenizer.from_pretrained(
    "knowledgator/Qwen-encoder-1.5B"
)

model = Qwen2BiModel.from_pretrained("knowledgator/Qwen-encoder-1.5B")

text = "The quick brown fox jumps over the lazy dog."

inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=512)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
inputs = {k: v.to(device) for k, v in inputs.items()}

with torch.no_grad():
    outputs = model(**inputs)

last_hidden_states = outputs.last_hidden_state

Here's an improved and expanded version of the README snippet:

Adapting for Different Discriminative Tasks

Our bi-directional LLaMA model can be easily adapted for various discriminative tasks such as text classification, question answering, and token classification. To use these specialized versions, we provide a fork of LLM2Vec with additional functionality.

Installation

To get started, clone our fork of LLM2Vec and install it:

bash

git clone https://github.com/Knowledgator/llm2vec.git
cd llm2vec
pip install -e .

Using -e flag installs the package in editable mode, which is useful for development.

Usage

Here's how to import and use the models for different tasks:

python

from llm2vec import (
    AutoLLMEncoderForSequenceClassification,
    AutoLLMEncoderForQuestionAnswering,
    AutoLLMEncoderForTokenClassification
)

# Load models for different tasks
classification_model = AutoLLMEncoderForSequenceClassification.from_pretrained('knowledgator/Qwen-encoder-1.5B')
question_answering_model = AutoLLMEncoderForQuestionAnswering.from_pretrained('knowledgator/Qwen-encoder-1.5B')
token_classification_model = AutoLLMEncoderForTokenClassification.from_pretrained('knowledgator/Qwen-encoder-1.5B')

Example: Text Classification

Here's a basic example of how to use the model for text classification:

python

from transformers import AutoTokenizer

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained('knowledgator/Qwen-encoder-1.5B')

# Prepare input
text = "This movie is great!"
inputs = tokenizer(text, return_tensors="pt")

# Get classification logits
outputs = classification_model(**inputs)
logits = outputs.logits

# The logits can be used with a softmax function to get probabilities
# or you can use torch.argmax(logits, dim=1) to get the predicted class

Fine-tuning

To fine-tune these models on your specific task:

Prepare your dataset in a format compatible with HuggingFace's datasets library.
Use the Trainer class from HuggingFace's transformers library to fine-tune the model.

Here's a basic example:

python

from transformers import Trainer, TrainingArguments
from datasets import load_dataset

# Load your dataset
dataset = load_dataset("your_dataset")

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    warmup_steps=500,
    weight_decay=0.01,
    logging_dir="./logs",
)

# Initialize Trainer
trainer = Trainer(
    model=classification_model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
)

# Fine-tune the model
trainer.train()

Contributing

We welcome contributions! If you have suggestions for improvements or encounter any issues, please open an issue or submit a pull request on our GitHub repository.

RichardErkhov/knowledgator_-_Qwen-encoder-1.5B-gguf

Original model description:

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

Overview:

Installation

Usage

Adapting for Different Discriminative Tasks

Installation

Usage

Example: Text Classification

Fine-tuning

Contributing

RichardErkhov/
knowledgator_-_Qwen-encoder-1.5B-gguf