This is a Floating Point 32 upscale, where all components and merges were remastered to floating point 32. This includes all the merges (recreated with master files), and where possible subbing full FP32 models.

The goal: Carry forward maximum precision right up to the point where it is "GUFFed".

This includes F32 master file for GGUF too... at a whopping 78 GBs.

WHY?

Because the difference between F32 vs BF16 is... over 8 DECIMAL places.

And as each merge / model is modified there are "losses" along the way.

These losses are carried forward and in turn lead to more losses.

And decimal points are critical to model performance.

SMALL?

Yes... but multiplied by each merge(s), and compression(s): 20 billion times.

The result:

At Q2K an impressive drop of 533 points in perplexity. (lower is better) (VS: Q2K original base model: PPL = 9.8077 +/- 0.06821 )

At Q4KM a whopping drop of 976 points in perplexity. (VS: Q4km original base model -> PPL = 8.7858 +/- 0.06074)

At Q6 an awesome drop of 234 points in perplexity. (VS: Q6 original base model -> PPL = 8.6070 +/- 0.05907 )

To put this in perspective "Q6" now operates ABOVE the original full precision version of "Psyonic-Cetacean-20b" and Q4KM operates at close to Q6 level quality.

This because at "Q6" the quant / compressed model is considered to be accurate within "+0.0008 ppl" of the full, uncompressed / unquanted model and it exceeds this threshold by over 200 points.

Imatrix quants take this even further in most cases DOUBLING the "drop" in perplexity realized in the reg quants.

Q4km-imatrix :

Final estimate: PPL = 8.6095 +/- 0.05898

(Non imatrix: Final estimate: PPL = 8.6902 +/- 0.05985 )

(VS: Q4km base model -> PPL = 8.7858 +/- 0.06074)

(VS: Q6 BASE model -> Final estimate: PPL = 8.6070 +/- 0.05907 Q6)

But... what about Q8?

The mountain moved:

150 points better: PPL = 8.5850 +/- 0.05881 VS: BASE/ORGINAL: PPL = 8.6012 +/- 0.05900

THE RESULTS ARE IN:

AS per Jeb Carter, original creator of the model:

- instruction following has improved dramatically.
- new abilities have emerged.
- he had to REDUCE the instructions sets used because the model no longer needed as specific instructions.
- prose, nuance and depth have all improved.
- known issues with the original model have disappeared.

This is not "something for nothing" ; it is method of ensuring maximum precision at every step just before "ggufing" the model.

The methods employed only ensure precision loss is minimized or eliminated.

It is mathematical and theory sound.

The bottom line here is this:

Higher quality instruction following and output.

Likewise you can use a smaller compression, with higher token per second and still get great quality.

Same great model... turbo charged.

This is the first group of remasters.

Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:

In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;

Set the "Smoothing_factor" to 1.5 to 2.5

: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"

: in text-generation-webui -> parameters -> lower right.

: In Silly Tavern this is called: "Smoothing"

NOTE: For "text-generation-webui"

-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)

Source versions (and config files) of my models are here:

https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be

OTHER OPTIONS:

Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.

Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers

This a "Class 2" model:

For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:

[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]

The FOUR Horsemen:

This repo will be followed by a "reg quant plus" repo, which added additional components into the GGUF (all levels) at floating point 32 precision to further increase the sheer creativity and raw AI horsepower.

This process shaves at extra 50-100 points off perplexity... again.

Following this group will be a full float 32 precision Imatrix (including reg quants "imatrixed").

Test results VS org and "ultra" regular quants will be posted when they come in.

Imatrix Plus repo (with the same floating 32 enhancement at "reg quant plus") that will push the limit even more.

Details of all methods (and pitfalls to avoid) employed to make this high precision remasters will be posted shortly along with comparisons of original model and new ultra remaster.

Thanks again to Jeb Carter, the original creator of "Psyonic-Cetacean 20B"

[ https://huggingface.co/jebcarter/psyonic-cetacean-20B ]

Special Thanks:

Special thanks to all the following, and many more...

All the model makers, fine tuners, mergers, and tweakers:

Provides the raw "DNA" for almost all my models.
Sources of model(s) can be found on the repo pages, especially the "source" repos with link(s) to the model creator(s).

Huggingface [ https://huggingface.co ] :

The place to store, merge, and tune models endlessly.
THE reason we have an open source community.

LlamaCPP [ https://github.com/ggml-org/llama.cpp ] :

The ability to compress and run models on GPU(s), CPU(s) and almost all devices.
Imatrix, Quantization, and other tools to tune the quants and the models.
Llama-Server : A cli based direct interface to run GGUF models.
The only tool I use to quant models.

Quant-Masters: Team Mradermacher, Bartowski, and many others:

Quant models day and night for us all to use.
They are the lifeblood of open source access.

MergeKit [ https://github.com/arcee-ai/mergekit ] :

The universal online/offline tool to merge models together and forge something new.
Over 20 methods to almost instantly merge model, pull them apart and put them together again.
The tool I have used to create over 1500 models.

Lmstudio [ https://lmstudio.ai/ ] :

The go to tool to test and run models in GGUF format.
The Tool I use to test/refine and evaluate new models.
LMStudio forum on discord; endless info and community for open source.

Text Generation Webui // KolboldCPP // SillyTavern:

Excellent tools to run GGUF models with - [ https://github.com/oobabooga/text-generation-webui ] [ https://github.com/LostRuins/koboldcpp ] .
Sillytavern [ https://github.com/SillyTavern/SillyTavern ] can be used with LMSTudio [ https://lmstudio.ai/ ] , TextGen [ https://github.com/oobabooga/text-generation-webui ], Kolboldcpp [ https://github.com/LostRuins/koboldcpp ], Llama-Server [part of LLAMAcpp] as a off the scale front end control system and interface to work with models.

DavidAU/Psyonic-Cetacean-Ultra-Quality-20b-GGUF-imatrix

Ultra Quality High Remaster of the incredible: Psyonic-Cetacean-20b - Imatrix Plus.

Special Thanks:

DavidAU/
Psyonic-Cetacean-Ultra-Quality-20b-GGUF-imatrix