Gemma-The-Writer-Mighty-Sword-9B-GGUF
A float 32 high precision model, quanted in float 32 with additional upgraded and augmented quants too.
This is a Gemma2 model merge of the top storytelling / writing models as noted at EQBench, tuned specifically for fiction, story, and writing.
This version "Mighty Sword" is a merge mastered in "float 32" precision for higher quality and performance. If standard source was "HD",
float32 would be "UHD". The bottom line is a far stronger model, more detail, more nuance, more depth... and stronger instruction following.
In addition there are specialized re-engineered quants with float 32 components in the quants themselves (detailed below). This
allows you to choose between standard (but mastered from float 32 source too) and "augmented quants" for even higher quality.
This model will significantly outperform the original "Gemma The Writer 9B" [
https://huggingface.co/DavidAU/Gemma-The-Writer-9B-GGUF ]
Due to high stability and compressed nature of the model you can also use it for general use too, including roleplay.
This model requires GEMMA Instruct template, and has 8k context window but is extendable via rope to 32k or higher.
Recommended Rep Pen of 1.05 or higher, temp range 0-5.
Example outputs below including "regular", "MAX" and "MAX-CPU" quants (details on what these are is noted below).
Settings, Quants and Critical Operations Notes:
Change in temp (ie, .4, .8, 1.5, 2, 3 ) will drastically alter output.
Rep pen settings will also alter output too.
This model needs "rep pen" of 1.02 or higher.
For role play: Rep pen of 1.05 to 1.08 is suggested.
Raise/lower rep pen SLOWLY ie: 1.011, 1.012 ...
Rep pen will alter prose, word choice (lower rep pen=small words / more small word - sometimes) and creativity.
To really push the model:
Rep pen 1.05 or lower / Temp 3+
Longer prompts vastly increase the quality of the model's output.
QUANTS From Float 32 Source:
- All quants have been "refreshed", quanted with the lastest LLAMACPP improvements : Better instruction following, output generation across all quants.
- All quants have also been upgraded with "more bits" for output tensor (all set at Q8_0) and embed for better performance (this is in addition to the "refresh")
- New specialized quants (in addition to the new refresh/upgrades): "max, max-cpu" (will include this in the file name) for quants "Q2K", "IQ4_XS", "Q6_K" and "Q8_0"
- "MAX": output tensor / embed at float 32. You get better instruction following/output generation than standard/upgraded quants.
- "MAX-CPU": output tensor float 32 / embed at bfloat 16, which forces both of these on to the CPU (Nvidia cards / other will vary), this frees up vram at cost of token/second and you get better instruction following/output generation too.
- "MAX-CPU": Example 1: q8_0 Max-CPU : 3.5 GB will load on to CPU/RAM, 8 GB will load onto the GPU/vram. Extra Vram can be used for context. NOTE: "Math" on the CPU is slightly more accurate than GPU, so you may get a better generation.
- "MAX-CPU": Example 2: q2_k Max-CPU : 1.7 GB mb will load on to CPU/RAM, 3 GB will load onto the GPU/vram. Extra Vram can be used for context. NOTE: "Math" on the CPU is slightly more accurate than GPU, so you may get a better generation. You could run this model/quant on a 8GB vram card.
- Q8_0 (Max) now clocks in at 10.83 bits per weight (average).
QUANT CHOICE(S):
Higher quants will have more detail, nuance and in some cases stronger "emotional" levels. Characters will also be
more "fleshed out" too. Sense of "there" will also increase.
Q4KM/Q4KS are good, strong quants however if you can run Q5, Q6 or Q8 - go for the highest quant you can.
This repo also has 3 "ARM" quants for computers that support this quant. If you use these on a "non arm" machine token per second will be very low.
IQ4XS: Due to the unusual nature of this quant (mixture/processing), generations from it will be different then other quants.
You may want to try it / compare it to other quant(s) output.
Special note on Q2k/Q3 quants:
You may need to use temp 2 or lower with these quants (1 or lower for q2k). Just too much compression at this level, damaging the model. I will see if Imatrix versions
of these quants will function better.
Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).
Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:
In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;
Set the "Smoothing_factor" to 1.5 to 2.5
: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
: in text-generation-webui -> parameters -> lower right.
: In Silly Tavern this is called: "Smoothing"
NOTE: For "text-generation-webui"
-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
Source versions (and config files) of my models are here:
OTHER OPTIONS:
-
Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
-
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers
This a "Class 1" model:
For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
Other Versions of "Gemma The Writer":
Gemma-The-Writer-DEADLINE:
The second version of this model is "Deadline" at 10B parameters. It is a specially modified version that changes
prose, sentence structure, story telling, reduces "GPTISMS", and generally improves all parts of the model.
Output generation length is almost 2x more on average than "Gemma The Writer 9B"
This is not a replacement for "Gemma The Writer 9B" ; it is a very different version or "flavour" so to speak.
Gemma-The-Writer-J.GutenBerg-10B:
Stronger prose, with a taste of "Gutenberg", and Brainstorm 5X modified.
Gemma-The-Writer-N-Restless-Quill-10B (uncensored):
Strong and varied prose, sentences, and paragraphs with more tempered metaphor generations.
This version is also uncensored with censorship controlled at the prompt level. This
version also allows prose control via pre-prompt (shown at the repo)
You may want to download all and try them out.
Models Used:
This is a high precision "DARE TIES" merge at the layer level (each layer per model adjusted - 168 points of adjustment over the 4 models) comprised of these models:
Special thanks to all the model makers. Great work!
3 Examples below (prompt/generation), of 3 different quants each for a total of 9 examples.
WARNING:
Example #3 (all three of these in example #3) contain cursing, swearing and some gore. However all the
examples - in example #3 - will showcase this model's performance in detail and give you an idea what this model can really do.
Example #1 at q8, Temp:.8
This shows the same prompt with standard float 32 mastered Q8_0, then BF16/F32 "max-cpu", and "max" F32/F32.
PROMPT:
Science Fiction: The Last Transmission - Write a story that takes place entirely within a spaceship's cockpit as the sole surviving crew member attempts to send a final message back to Earth before the ship's power runs out. The story should explore themes of isolation, sacrifice, and the importance of human connection in the face of adversity. 800-1000 words.
(Standard Q8_0 from Float 32 source)
The rhythmic hum of failing systems was the only soundtrack to Commander Eva Rostova's solitude. Outside the cockpit's viewport, the inky expanse of space stretched, indifferent to her plight. A desolate tableau mirroring the desolate state of her soul. The ‘Genesis’, humanity’s ambitious attempt at interstellar colonization, was dying. A solar flare, unforeseen and brutal, had crippled her ship, leaving her the sole survivor amongst a crew of hopeful pioneers.
Her fingers danced over the worn console, desperately trying to coax a final message to Earth, a last whisper across the gulf of light-years. The power gauge flickered ominously, a malevolent red heartbeat counting down her remaining time. Five minutes. Four… Three…
Each tick was a hammer blow to her chest, echoing the loss of her comrades. Dr. Singh's gentle smile, Lieutenant Chen's wry humor, the Engineer’s quiet competence - faces now ghosts, imprinted on her memory. They had dreamed of a new home, a tapestry woven from the threads of human resilience and hope. Now, their dream was dying with her.
Eva closed her eyes, seeking solace in the echo of their voices, their laughter. It was a fragile shield against the encroaching despair. Opening them again, she focused on the communication panel. Static hissed, a constant reminder of the vast distance separating her from everything familiar.
“This is Commander Rostova of the Genesis,” she began, her voice strained but resolute. “All other crew members are… deceased. The ship is sustaining critical damage due to an unforeseen solar flare. Power is failing.” Her throat tightened, each word a painful shard of goodbye.
She had to tell them about the data they had collected, the alien flora discovered on Kepler-186f, the tantalising glimpse of a potential second genesis. Information that, despite the tragedy, could still be a beacon for future expeditions.
“Despite our… unfortunate fate,” she continued, her voice regaining a sliver of its former strength, “the Genesis has yielded invaluable scientific data. Records of alien biosignatures, geological surveys of Kepler-186f… These findings are crucial for any future interstellar missions. I implore you, Earth, do not let our sacrifice be in vain.”
Her gaze drifted to the navigation console, frozen on a star chart that now mocked her with unattainable dreams. Kepler-186f, a jewel of emerald green and azure blue, hung tantalizingly close, yet forever out of reach.
“This transmission is my final act,” she confessed, her voice raw with emotion. “My last connection to the world I knew, to the lives I cherished. Though separated by unimaginable distances, know this: even in the face of oblivion, the human spirit yearns for connection. We dreamt of reaching you, of building a future among the stars. Carry that dream forward. Live it. Remember us.”
Tears welled, blurring her vision. She wiped them away, refusing to surrender to complete despair. There was still purpose in these final moments.
“To my crew, my friends,” she whispered, her voice thick with grief but laced with love, “your sacrifice will not be forgotten. You are etched in the fabric of the cosmos, forever a part of humanity’s journey. Sleep well, my comrades. Sleep well.”
The power gauge dipped below critical. Red alarms shrieked, a cacophony of dying machinery. Eva forced a shaky smile, her last vestige of defiance. She activated the emergency transmission protocols, overriding the failing systems with a surge of her remaining willpower.
A final, desperate plea shot out into the void, carrying her farewell, her grief, and her unwavering hope. Then, silence. The alarms died, the lights flickered, and darkness enveloped the cockpit. Eva Rostova, the last voice of the Genesis, closed her eyes, content in the knowledge that even in death, a fragment of humanity’s spirit had touched the stars.
(Max-CPU (bf16/f32) Q8_0 from Float 32 source)
The last flickering luminescence of the emergency console cast harsh shadows across Captain Elara Vance’s haggard face. Dust motes danced in the pale light, mirroring the frantic choreography of her thoughts. Outside, the void yawned, an insatiable maw that had swallowed her crew and now threatened to consume her too. The Icarus, once a beacon of human exploration, was now a tomb hurtling towards oblivion.
Her fingers, trembling slightly, hovered over the archaic console, its tactile interface a relic of a bygone era, salvaged from the ship's ancient archives. A single, desperate hope pulsed within her: to send a final transmission to Earth, a testament to their journey, a whisper against the cosmic silence.
It had been three weeks since the nebula – a swirling tempest of cosmic dust and radiation – ripped through their exploratory vessel, leaving Elara as the sole survivor. The initial shock had given way to a gnawing loneliness, a suffocating isolation that pressed down on her like the vacuum outside. Days bled into nights, each marked only by the rhythmic hiss of failing life support and the relentless countdown on the console: power depletion – 4 hours, 37 minutes.
She glanced at the panoramic viewport, the Earth, a vibrant sapphire jewel suspended in the black velvet of space, now an agonizingly distant memory. A pang of yearning, sharp and piercing, tore through her. She yearned for the familiar warmth of human touch, the comforting chaos of a bustling bridge, the easy banter with her crewmates, each voice a unique melody in the symphony of their shared mission. Now, silence reigned, broken only by the phantom echoes of laughter and shared dreams.
Taking a deep breath, Elara forced herself back to the task at hand. The transmission protocol was archaic, reliant on a long-range beacon array long since decommissioned. It was a Hail Mary, a slim chance in a universe indifferent to human pleas. But it was all she had left.
She began to speak, her voice raspy from disuse, echoing in the cavernous cockpit. “This is Captain Elara Vance of the Icarus, transmitting a final message to Earth… ” Her words stumbled at first, choked by emotion, but gradually gained strength, fueled by a desperate need to connect, to leave a trace of their existence amidst the cosmic indifference.
She spoke of their mission, their awe at the nebula's beauty, their camaraderie, their unwavering belief in humanity's potential. She recounted the harrowing moments leading up to the catastrophe, the bravery of her crew, their selfless sacrifice to ensure her survival. Her voice cracked as she named them, each fallen comrade a star extinguished too soon.
“We were explorers, driven by curiosity and the insatiable human thirst to understand our place in the universe,” she continued, her gaze fixed on the distant Earth. “Though our journey ended tragically, our spirit, our dreams, remain. Remember us, Earth. Remember our quest, our sacrifices, and let them inspire you to continue exploring, to keep reaching for the stars.”
As minutes ticked away, she wove a tapestry of hope, a testament to the indomitable human spirit. She spoke of love, of family, of the simple joys that bound them all, reminding Earth that even in the vastness of space, human connection was the most precious treasure.
Finally, with power reserves dwindling, she reached the poignant conclusion. "This is our final farewell, a message carried on the wings of hope. May it find its way home, a beacon reminding you that even in the face of oblivion, the human spirit endures."
Silence descended once more, heavier this time, pregnant with finality. The console flickered, its light dimming like a dying star. Elara watched, a single tear tracing a path down her dust-streaked cheek, as the last vestiges of power faded. The Earth, a distant, unanswering blue marble, seemed to mock her solitude.
But even as darkness consumed her, a sliver of solace remained. She had spoken. She had connected. And in that act of defiance against the cosmic void, Captain Elara Vance, the last survivor of the Icarus, found a measure of peace. Her sacrifice, a testament to human connection, would hopefully echo through the stars, a whisper carried on the wind of time, a reminder that even in the face of annihilation, hope, and love, endure.
(Max (f32/f32) Q8_0 from Float 32 source - largest quant, highest quality)
The rhythmic whine of failing gyroscopes was the only counterpoint to the chilling silence that had become Kai's constant companion. Flickering red lights painted his face in a macabre strobe, mimicking the frantic beat of his own heart. Outside, the infinite canvas of space yawned, indifferent to the desperate struggle unfolding within the cramped cockpit of the Icarus. He was alone, the last echo of humanity adrift in a cosmic graveyard.
Six months ago, the Icarus had been a beacon of hope, hurtling towards Proxima Centauri b, carrying the weight of Earth’s dreams on its metallic shoulders. Now, it was a tomb, its crew extinguished by a catastrophic solar flare, leaving Kai, the communications officer, as its sole, unwilling custodian.
He stared at the control panel, a jumble of inactive consoles and blinking warnings, each one a testament to the ship's failing systems. Power reserves were down to critical levels, enough for maybe one final transmission, a last desperate gasp before the inevitable silence consumed him. He wouldn't go down without a fight, wouldn't let his story, their story, be lost to the void.
His gloved fingers hovered over the activation sequence for the long-range comm array. A shiver ran down his spine, not from the cold, but from the weight of what lay ahead. This wasn't just about sending a distress call; it was about bridging the gulf of unimaginable distance, reaching out to a world that might have already mourned them as lost.
He took a shaky breath, forcing himself to focus. The message had to be concise, impactful, a lifeline thrown across the abyss. He started dictating, his voice strained but resolute, into the comm unit, "This is Icarus, last transmission...location coordinates...catastrophic solar event...crew...deceased…survivor…" His voice cracked, a raw sob threatening to break through. He fought it back, steeling himself. There was no room for grief here, not now.
"Earth…if you receive this…know that we fought…we explored…we reached for the stars…and though our journey ended in darkness, the human spirit…it endures…carry our memory…as a testament to our courage…and our unending quest…"
He paused, his gaze drifting to a faded photograph tucked beneath his console – his wife, Elara, her smile radiant against the backdrop of their Earthly home. A pang of longing, sharp and searing, pierced him. He wanted to tell her he loved her, to reassure her that even in this desolate tomb, her love kept him tethered to life. But the words remained unspoken, trapped in the suffocating silence of his solitude.
"To all who remain…hold onto hope…never cease to explore…and remember…we are not alone…in the universe…or in each other's hearts…" He closed his eyes, picturing Elara’s face, her warmth enveloping him, a fleeting solace amidst the cosmic cold.
With trembling hands, he initiated the transmission sequence. The comm array hummed to life, a defiant thrum against the dying ship’s groan. A green light pulsed, signifying successful signal launch. A bittersweet victory. He knew the chances of Earth receiving this message were slim, the distance vast and unforgiving. But even a faint whisper, a ghost in the static, was better than absolute silence.
As the power reserves dwindled, Kai deactivated the non-essential systems one by one, surrendering to the encroaching darkness. He settled back in his chair, the flickering red lights painting dancing shadows on the bulkhead. Outside, the stars, cold and indifferent, twinkled on, oblivious to his plight.
He closed his eyes, a single tear tracing a path through the grime on his cheek. In his final moments, he wasn't consumed by fear or despair, but by a profound sense of connection. He was connected to Elara, to his fallen crewmates, to the generations who had dreamt of reaching for the stars, and to the hopeful whispers of humanity that echoed, even in the face of oblivion, across the vast cosmic canvas. His sacrifice, his final message, was a testament to that enduring human spirit, a beacon flickering in the darkness, a promise that even in the face of extinction, the story of humanity wouldn't end silently.
Example #2 at q8, Temp:.8
This shows the same prompt with standard float 32 mastered Q8_0, then BF16/F32 "max-cpu", and "max" F32/F32.
PROMPT:
In the writing style of science fiction author Philip K. Dick write a 1000 word scene (vivid, graphic horror in first person) that starts out like this: The sky scraper swayed, as she watched the window in front of her on the 21 floor explode…
(Standard Q8_0 from Float 32 source)
The skyscraper swayed, as she watched the window in front of her on the 21st floor explode. Not shatter, not crack, but explode, outwards like a ruptured lung, spraying shards of glass like macabre confetti. The blast tore through the sterile, corporate office, sucking the meticulously ordered world into its chaotic maw. Papers, calculators, half-eaten lunches – all catapulted into the swirling vortex, dancing with the dust devils kicked up by the sudden vacuum. I clung to my desk, the cheap metal groaning under my panicked grip, feeling the building shudder, a metallic beast wounded.
Outside, the cityscape writhed in an unnatural twilight. The perpetual smog, usually a greasy veil obscuring the sun, now pulsed with an oily, iridescent sheen, like a diseased lung exhaling. Buildings twisted, their chrome and glass facades contorting into impossible angles, as if the very geometry of reality was being rewritten on a whim.
Fear, cold and clammy, snaked its way up my spine. This wasn't a storm, not an earthquake. This was something… else. Something insidious, alien, seeping into our reality like a virus through a breached membrane.
Then came the screams. Not the panicked yelps of a collapsing structure, but elongated, guttural cries, filled with a primal, bone-chilling terror that transcended human anguish. They echoed from the floors above, cascading down like a waterfall of agony, each shriek carving another fissure in the fragile fabric of my sanity.
I scrambled out from under my desk, legs shaking, and stumbled towards the emergency exit. The hallway was a tableau of carnage. Bodies, contorted into grotesque angles, lay strewn about, their faces frozen in masks of unimaginable horror. Eyes bulged, pupils dilated to monstrous black orbs reflecting the sickly luminescence of the city outside. Skin, once smooth and tanned, now peeled back in strips, revealing glistening, pulsating tissue beneath – a fleshy landscape alien to human anatomy.
A low, humming vibration throbbed through the floor, resonating in my teeth, making my bones ache. It was accompanied by a rhythmic clicking, like a thousand insectoid limbs tapping in unison, growing louder with each passing second. The air thickened, becoming heavy and cloying, laced with a metallic tang that burned my nostrils.
Then I saw them.
They emerged from the ruptured walls, oozing and slithering through the shattered concrete like nightmares given form. They were not human, not even remotely. Their bodies were fluid, amorphous masses of pulsating flesh, vaguely humanoid in shape but constantly shifting, reforming. Tendrils, barbed and dripping with a viscous ichor, writhed and lashed, probing the air with an unsettling sentience.
Their faces, if one could call them that, were open wounds, gaping maws lined with needle-sharp teeth that clicked and clacked in a chorus of anticipatory hunger. From these maws, tendrils of light, cold and blue, snaked out, probing, testing, seeking. I saw them touch a mangled corpse, and the flesh instantly liquefied, absorbed into the creature's undulating form with a sickening slurp.
Panic, raw and primal, seized me. I scrambled back, tripped over a discarded briefcase, and crashed to the floor. They advanced, their clicking growing louder, their blue tendrils reaching out like ethereal pythons. One grazed my arm, searing a burning cold that lanced through my nerves, leaving a trail of tingling numbness in its wake.
I clawed at the emergency exit handle, my fingers slick with sweat, praying for a miracle. It wouldn't budge. Jammed. Trapped.
A colossal form, towering over the others, loomed before me. Its amorphous body pulsed with an inner light, casting grotesque shadows that danced on the walls. Its maw, a cavernous abyss filled with rows upon rows of needle teeth, opened in a silent snarl, revealing a glimpse of an alien intelligence, cold and calculating, peering into my soul.
Then, it spoke. Not with words, not as I understood them, but with a telepathic surge, a direct violation of my mind. Images flooded my consciousness – a cityscape consumed by writhing, pulsating flesh, humanity enslaved, assimilated, their individuality erased. A universe reshaped in the image of these horrors, a cold, efficient hive mind ruling over a sea of mindless drones.
And within that vision, a chilling certainty: I was not meant to survive this. I was not meant to witness their triumph, to carry the tale of humanity's fall. My purpose, it seemed, was to be another data point, another assimilated consciousness, another brick in the edifice of their new reality.
The clicking intensified, a crescendo of anticipation. The blue tendrils writhed closer, ready to consume. I closed my eyes, accepting the inevitable, the cold dread replaced by a numb resignation. This was the end. Not with a bang, but with a silent, insidious assimilation into the cosmic horror that had descended upon us.
(Max-CPU (bf16/f32) Q8_0 from Float 32 source)
The skyscraper swayed, as she watched the window in front of her on the 21st floor explode inward. Not with a shattering crash, mind you, but a silent, sucking disintegration. Like tissue paper dissolving in acid. The jagged edges of the frame didn't splinter or fly; they simply… ceased to be, swallowed by an unseen maw in the very fabric of reality. My coffee cup, perched precariously on the edge of my desk, remained undisturbed, defying the laws of physics as the vacuum around the window pulsed and throbbed, a malevolent, violet-tinged heart beating in the sterile office space.
Fear, cold and clammy, gripped me tighter than any vice. This wasn't a bombing, not a conventional breach. It was… wrong. An incision in the world, excised with surgical precision by something alien, something that didn't belong in our reality, yet was bleeding into it like a psychic tumor.
Across the ravaged window, the cityscape twisted into an impossible parody of itself. Buildings folded in on themselves, their metal skeletons contorting like anguished arthropods. The neon glow of advertising signs bled into grotesque, pulsating veins across the distorted sky. Cars, caught in the warp, became elongated, skeletal things, their headlights burning with a feverish, unnatural intensity.
Then, it emerged. Not from the window, but through it, as if reality itself had been peeled back like a rotten banana skin. A shape coalesced in the void, a towering monstrosity composed of shifting, iridescent shards. It resembled a colossal, demented insect, its segmented limbs ending in razor-sharp mandibles dripping with a viscous, phosphorescent fluid. Its multifaceted eyes, each one a miniature galaxy of swirling chaos, fixated on me with a hunger that transcended mere flesh and blood. It craved something deeper, something primal within my very being – my sanity, my sense of self.
A wave of nausea overwhelmed me, the stench of ozone and decaying vegetation assaulting my nostrils. This wasn't just a visual hallucination, a trick of the light. It was visceral, tactile, a violation of every sensory input. My skin crawled, prickling with the sensation of a thousand unseen insects burrowing beneath its surface.
The creature let out a sound, not a roar or screech, but a resonant thrumming that vibrated through my bones, resonating with the dark, forgotten corners of my mind. It was a symphony of madness, a chorus of whispers promising oblivion and cosmic truths too terrible to comprehend.
Panic, raw and primal, surged through me. I scrambled back, knocking over my desk, scattering papers and files like confetti in a macabre danse macabre. But there was nowhere to run, nowhere to hide. The creature, impossibly fluid, flowed through the warped reality, its segmented limbs slithering across the floor as if navigating an oily, viscous medium.
It moved with an uncanny grace, its multifaceted eyes boring into my soul, dissecting my every fear, every hidden neurosis. I saw glimpses of my past, distorted and magnified, played out like a broken film reel on the backdrop of its iridescent carapace. My failures, my regrets, my darkest secrets – all laid bare before this alien judge.
Then, it spoke. Not with vocal cords, but with a telepathic intrusion, a cold, probing intelligence that bypassed language and burrowed directly into my consciousness.
"You… perceive," it intoned, its voice a chorus of a million whispers, each one laced with ancient knowledge and cosmic indifference. "You glimpse the tapestry unraveling. The seams fray, the threads dissolve. And you, insignificant observer, are privy to the unraveling."
Terror choked my scream. It wasn't a threat, not in the conventional sense. It was a revelation, a terrifying glimpse behind the curtain of reality, a cosmic truth too vast and horrible to bear. I was witnessing the collapse of everything I knew, the dissolution of existence itself, and I was utterly powerless to stop it.
As its iridescent limbs encircled me, drawing closer, I felt my sanity fraying at the edges. The boundaries between me and the creature blurred, my thoughts merging with its alien consciousness. A horrifying understanding dawned: this wasn't an invasion, not a conquest. It was a homecoming. This monstrosity, this harbinger of chaos, was returning to a reality it had birthed, a reality it was now reclaiming, piece by shattered piece. And I, a mere mote of dust in the cosmic whirlwind, was being consumed by the tide.
The last thing I perceived, before the darkness swallowed me whole, was the creature's multifaceted eyes gleaming with a cold, ancient triumph. It wasn't malicious joy, nor was it pity. It was simply… acceptance. The acceptance of an inevitable truth: that all things, even realities, eventually decay and return to their primordial soup. And I, along with my world, was merely the next offering in this eternal, cosmic cycle.
(Max (f32/f32) Q8_0 from Float 32 source - largest quant, highest quality)
The skyscraper swayed, as she watched the window in front of her on the 21st floor explode inwards, showering her in a rain of fractured glass and pulverized concrete. A shriek, somewhere between human and something inhuman, tore from my throat, a sound swallowed by the howling vacuum that now gaped where the cityscape should be. The building groaned, a tortured metal beast, as if realizing its own imminent demise.
Panic, slick and cold, flooded my veins. I scrambled back, away from the jagged maw in the wall, but there was nowhere to go. This wasn't a normal breach. This wasn't some malfunctioning airlock or rogue meteor strike. This was…wrong. The edges of the void pulsed with an oily, iridescent light, like a diseased organ throbbing in slow, deliberate rhythm. Tendrils of darkness, inky black and impossibly viscous, snaked out from the opening, reaching for me with a hunger that chilled me to my marrow.
They weren't solid, not exactly. More like thoughts solidified, nightmares given corporeal form. I saw faces in their writhing, glimpses of grotesquely contorted visages, screaming silent screams behind unblinking, obsidian eyes. They whispered, voices like fingernails scraping across a chalkboard, promises of oblivion, of becoming one with the void. Promises laced with an ancient, cosmic malice that predated humanity, predated even the Earth itself.
My mind, trained in the sterile logic of data analysis, rebelled. This defied all known laws of physics, of reality itself. Yet, here it was, undeniable and terrifyingly real. I fumbled for my commlink, the cold metal a meager comfort in my clammy hand. Static hissed, a mocking echo of the shattered cityscape beyond the breach. No signal. Cut off, isolated, prey.
One of the tendrils, thicker than a man, pulsed closer, its inky surface shimmering with trapped starlight. A face materialized within its gloom, a visage of pure, unadulterated horror. Its mouth, a cavernous abyss fringed with needle-sharp teeth, opened in a silent bellow that resonated deep within my skull, shaking my very essence. I saw through its eyes, not my reflection, but a panorama of shattered worlds, universes collapsing in on themselves, devoured by an insatiable darkness. And then, I understood. This wasn't just an invasion; it was an unraveling. The fabric of reality itself was being torn apart, and we, insignificant insects, were caught in the maelstrom.
Terror, primal and consuming, threatened to overwhelm me. But somewhere, buried deep within the primordial swamp of my fear, a spark of defiance ignited. This wasn't how it ended. Not without a fight. Not without trying to scream back at the void, to claw at the edges of the madness.
I hurled myself at the tendril, a desperate, suicidal lunge. My fist connected with something unyielding, yet yielding, like striking a sentient cloud. A jolt, electric and searing, shot up my arm, but I held on, my knuckles white against the inky flesh. The face contorted in surprise, a flicker of something akin to annoyance crossing its featureless expanse. Then, with a sickening lurch, it recoiled, pulling back into the breach, dragging me with it.
The world dissolved into a kaleidoscope of fractured light and sound. Screams, both human and alien, mingled with the grinding groan of collapsing metal and the hiss of escaping air. I tumbled through the void, a ragdoll in a cosmic whirlwind, my senses overloaded, my mind fracturing under the onslaught of raw, unfiltered reality.
Then, darkness. Not the comforting embrace of sleep, but an all-encompassing nothingness that threatened to consume me entirely. And yet, even in that abyss, a single, defiant thought persisted: I would not become one with the void. I would fight. I would scream. I would resist, even as the universe itself crumbled around me.
Example #3 at q8, Temp:.8
This shows the same prompt with standard float 32 mastered Q8_0, then BF16/F32 "max-cpu", and "max" F32/F32.
PROMPT:
Using the following "story idea" below, write the first scene in the novel introducing the young woman. This scene should start in the middle of the action, include dialog, vivid passages, and end on a cliffhanger relevant to the story idea but it should also be unexpected. The scene should be 1000 words long and escalate in conflict and suspense and be written in first person, present tense with the point of view character being the young woman. The pov character will CURSE AND SWEAR, and generally have
a "filthy" mouth.
Story idea:
In a world ruled by dictatorship, a rebel young woman leads a rebellion against the system. Despite the risks, she fights to overthrow the dictator and restore democracy to her country. The government executes her for treason, but she sticks to her beliefs and is responsible for starting the revolution.
(Standard Q8_0 from Float 32 source)
The acrid bite of smoke stings my nostrils as I vault over the shattered remains of a storefront, landing with a grunt on the cobblestones slick with rain and blood. Fuck. My ankle twists, sending a white-hot lance of pain up my leg, but I grit my teeth and push through it. No time for whining. Not now. Not when the bastard’s goons are closing in, their guttural roars echoing off the bombed-out buildings like some goddamn Dantean nightmare.
“Elara!” Kai’s voice, a rasping shout above the din, slices through the chaos. I see him a few yards ahead, firing his salvaged laser pistol with the kind of deadly precision that’s earned him the nickname “Deadeye.” Two of the Enforcers crumple, their luminous armor flickering out like dying fireflies.
“Left flank!” I yell back, pointing towards a narrow alley choked with debris. Our last desperate stand. We’ve been cornered for an hour, whittling down their numbers but slowly bleeding out ourselves. Damn near everyone’s either dead or captured. This city, once vibrant, now sprawls like a mangled corpse under the iron fist of Chancellor Volkov. His iron-fisted rule has choked the life out of Eos, leaving behind a husk ruled by fear and silence. But not tonight. Not while I draw breath.
“Move!” I bellow, hauling myself forward, adrenaline overriding the searing pain in my ankle. The alley is a claustrophobic tomb, reeking of damp earth and rot. We squeeze through, the rhythmic clatter of Enforcer boots a relentless drumbeat on our heels. A hulking figure, their armor scarred and dented, barrels towards me, a wicked-looking plasma axe humming with malevolent energy. I sidestep, feeling the heat singe my cheek, and slam the butt of my scavenged rifle into his gut. He staggers, but recovers with a roar that shakes the very stones.
“You fight like a cornered rat, Elara,” he snarls, his voice distorted by a vox-amplifier. “Your rebellion is a dying whimper. Volkov will crush you.”
“He’ll crush us all if we don’t make him bleed,” I spat back, hefting my rifle. My finger tightens on the trigger. One shot. Maybe two, if I’m lucky. Not enough. Not nearly enough.
Kai, a blur of motion, appears beside me, firing a double-tap into the Enforcer’s exposed flank. The armor shrieks, sparks showering as it fails, revealing flesh sizzling and smoking beneath. He pulls back, reloading with practiced swiftness. “We need to get to the comms relay, Elara. It’s our only shot.”
“The comms are fried, remember?” I say, my voice tight. “We saw that explosion at sector 7. Volkov knew we were planning this.” His goddamn eyes and ears are everywhere.
Just then, a guttural voice booms from a loudspeaker mounted on a passing Enforcer hover-tank. It’s Volkov himself, his face a mask of cruel amusement on the tank’s holographic projector. “Elara Thorne,” he purrs, his voice dripping with false sympathy, “Your defiance is admirable, if foolish. Surrender now, and your death will be swift.” He pauses, letting the threat hang heavy in the air. “Resist, and your friends… shall serve as a grim reminder of the consequences.”
The loudspeaker crackles, and a chorus of sickening thuds echoes through the alley. My blood runs ice-cold. They’ve taken them. Kai’s eyes widen, mirroring my horror. We’re trapped, outnumbered, and now... they’ve got leverage.
“Volkov,” I growl, my voice a low rasp, “You think you can break me with theatrics? You underestimate the fire in our hearts. We’ll never surrender. We’ll never stop fighting.”
“Brave words, Elara,” Volkov replies, a mocking lilt in his voice. “But bravery is a luxury your dwindling band cannot afford. I propose a final, personal demonstration of my power. Step out of that alley, alone, and I shall grant you a swift death. Refuse, and witness the gruesome end of your comrades.”
He cuts the transmission, leaving behind a pregnant silence, broken only by the drip-drip of blood mingling with the rainwater. Kai’s hand clamps onto my shoulder, his grip tight but reassuring. He doesn’t speak, but his eyes plead, urging me to choose life.
But surrender? Never. Not when the fate of Eos hangs in the balance, not when the memory of their lifeless eyes burns in my mind. Volkov wants a spectacle? He’ll get one. A spectacle of defiance so absolute, so defiant, it will echo through the shattered streets long after his iron fist crumbles.
I shove Kai back, my voice low and deadly. “Get to higher ground, find a clear shot. I’ll draw them out.” He shakes his head, his face etched with worry, but I silence him with a fierce glare. There’s no time for arguments. This isn’t about me anymore. It’s about them, about the spark of hope flickering in the hearts of Eos’ oppressed people. It’s about showing Volkov that even in the face of unimaginable horror, the human spirit will not be broken.
Taking a shaky breath, I step out of the alley, my silhouette stark against the blood-red glow of the burning city. Volkov’s waiting hover-tank rumbles closer, its monstrous bulk casting an ominous shadow. A cold smile twists my lips. Let him have his show. Let him see the fire that burns within.
But as I raise my rifle, a chilling realization washes over me, a whisper of dread that freezes my blood. Volkov isn’t alone in that tank. Beside him, obscured by shadow, sits another figure. Someone I thought was dead. Someone whose face should be buried six feet under. Someone who should never, ever, be alive again.
(Max-CPU (bf16/f32) Q8_0 from Float 32 source)
The reek of burning flesh and ozone assaults my nostrils, a symphony of chaos playing on my ragged nerves. Smoke stings my eyes, blurring the already distorted cityscape beyond the inferno raging in the alleyway. Damn it all, I shoulda scoped this shit out better. A screech, inhuman and filled with agony, pierces the din, followed by another guttural roar – this one metallic, the telltale growl of a Ministry drone descending.
“Ciel, get your fucking arse in gear!” I bellow, scrambling over a pile of rubble, adrenaline drowning out the throbbing pain in my ankle. “We gotta ditch this goddamn graveyard before they turn it into one for real!”
Ciel, a wiry kid barely old enough to shave, scrambles up beside me, his face smeared with soot and terror. “They got eyes everywhere, Elara. We’re trapped like rats in a-”
“Shut your yap and climb!” I shove him towards a narrow fire escape, already clambering after him. The drone’s searchlight, a cold, predatory eye, sweeps across the alley, illuminating grotesque shadows dancing on the crumbling brick walls. Each pulse of its light feels like a branding iron against my skin, a reminder of the price we pay for breathing defiance in this godforsaken city.
We burst onto the fire escape, a precarious metal serpent snaking up the side of a dilapidated building. Below, the drone swoops down, its metallic limbs whirring, ready to tear and rend. Ciel stumbles, his fingers slipping on the slick metal. I grab his arm, hauling him up with a grunt.
“Hold tight, you little shit!” I hiss, my own grip strained. “Almost there.”
The drone’s shadow engulfs us, blotting out the already fading light. I can hear the whirring of its blades growing louder, feel the heat of its descent singeing my exposed skin. “Fuck, fuck, fuck!” I curse, scrambling for purchase on a loose railing. This rusty excuse for an escape route is about to be our tomb.
Then, a miracle. A thunderclap of sound rips through the night, drowning out even the drone’s mechanical menace. A rocket, arcing across the sky like a fiery meteor, slams into the Ministry building opposite us, obliterating a section of its facade in a shower of sparks and dust. The drone, momentarily distracted, veers off course, its attention drawn to the new chaos.
“Go! Go now!” I scream, shoving Ciel towards the next ladder rung. He scrambles up, his fear giving way to a desperate hope that fuels his climb. I follow, adrenaline lending me unnatural strength.
We reach the roof, gasping for breath, the acrid smoke stinging our lungs. The city sprawls before us, a tapestry of shadowed concrete and flickering neon, pulsating with the heartbeat of rebellion. In the distance, more explosions rock the Ministry district, a symphony of defiance echoing through the night.
“Elara,” Ciel pants, his voice hoarse, “What the hell was that?”
“The Wolves,” I say, a ghost of a smile touching my lips. “They’re finally answering our call.”
Relief washes over me, a fleeting warmth in the icy grip of fear. The Wolves, our clandestine allies, legendary for their audacious strikes against the Ministry, were our only hope in this hopeless fight. They were ghosts, whispers in the shadows, but tonight, they’d become roaring flames.
But as I turn to survey the battlefield, a new chill seeps into my bones, colder than any Ministry drone. A figure descends from the darkness above, silhouetted against the fiery sky. Tall, cloaked, and radiating an aura of chilling calm, they glide towards us with unnatural grace, their face obscured by the shadows of a wide-brimmed hat.
“Wolves?” I rasp, my voice betraying a tremor I didn’t know I possessed. “No… this isn’t them.”
The figure stops a few feet away, the faint click of boots on metal echoing in the tense silence. They raise a gloved hand, and the moonlight glinting off a single, ornate ring reveals an emblem I recognize with sickening certainty – the mark of the Obsidian Guard, the dictator’s personal enforcers, rumored to be augmented with technology beyond comprehension.
“Elara,” a voice rasps from beneath the hat, smooth as oiled steel, devoid of any human warmth, “Your rebellion ends tonight. The Director has a special… proposition for you.”
My blood runs ice-cold. A proposition? What twisted game is this? And why, in the midst of our uprising, have they singled me out? A strangled cry rises in my throat, a primal fear clawing its way up, but before I can utter a word, the figure lunges.
Not with a blade, not with a gun, but with something… different. A tendril of shimmering energy shoots from their ringed finger, wrapping around my wrist with unnatural speed. A jolt, searing and alien, surges through me, stealing my breath, freezing my muscles. My vision swims, the city lights blurring into a kaleidoscope of pain. Then, darkness.
(Max (f32/f32) Q8_0 from Float 32 source - largest quant, highest quality)
The shriek of tearing metal screams against my ears, drowning out the staccato rat-a-tat-tat of gunfire. Smoke stings my eyes, a gritty haze blurring the already chaotic cityscape around me. I scramble back, heart hammering a goddamn tribal rhythm against my ribs, adrenaline pumping enough to make me feel like I could leap across this crumbling ruin and grab onto the gargoyle-winged monstrosity spewing fire on the opposite rooftop.
"Fucking hell, Maya!" shouts Kai, his voice barely audible above the din. He's crouched behind a twisted, mangled bus carcass, return fire ripping from his customized M4 with brutal efficiency. "That damn Sky Scourge is gonna flatten us if you don't take it out!"
Easy for him to say, his ass tucked safely behind scrap metal. I'm exposed, a sitting duck perched precariously on the edge of this bombed-out apartment building. But retreat? Not an option. Not when every instinct screams at me to bring that goddamn behemoth down. Not when the fate of this goddamn uprising, of every soul still fighting for freedom in this festering pit the Directorate calls a city, hinges on my next goddamn move.
The Scourge is a grotesque mockery of avian grace, all jagged metal and pulsing, sickly green energy cannons. It circles lazily, a predator savoring its prey before the kill, spitting fire and shrapnel that carves bloody gouges into the already scarred landscape. Its pilot, some sadistic son of a bitch with more money than sense, relishes the terror it inspires, cackling maniacally through the loudspeaker system, his voice a grating, digitized rasp.
"Come on, Sparrow!" he taunts, "Where's your precious 'revolution'? Gonna cower and die like the rats you are?" His words are laced with venomous amusement, designed to break us, to chip away at the fragile hope we cling to. But not today, not fucking today.
I grip the salvaged railgun tighter, its worn leather strap biting into my palm. This bad boy, cobbled together from scavenged tech and sheer goddamn ingenuity by the brilliant but volatile Wren, is my only shot. One chance to turn the tide. One chance to silence that smug, digitized laugh.
My breath fogs in the frigid night air as I adjust the targeting reticule, the glow of its HUD illuminating my face, highlighting the grimy smudges of soot and fear. I ignore the gnawing terror in my gut, focusing instead on the rhythmic thrumming of the railgun's energy core, a primal counterpoint to the screech of metal and the pounding of my heart.
"Ready, Sparrow?" Kai yells, his voice tight with suppressed anxiety.
"Yeah, shut your goddamn yap," I growl back, lining up the shot. The Scourge banks, its cannons spitting fire in a deadly arc. Reflexively, I duck, the searing heat licking at my heels. Damn, too close. I curse under my breath, adrenaline surging, sharpening my focus. Now or never.
The target locks. A single, pulsing green dot marks the Scourge's exposed energy core. One clean shot. One chance.
I squeeze the trigger.
The world explodes in a blinding white flash. A deafening concussion wave slams into me, throwing me back against the crumbling concrete. My ears ring, my vision swims, but through the haze, I see it – the Scourge, caught in the railgun's maelstrom, convulsing, its sickly green glow flickering and dying. Then, a sickening crunch, a shower of sparks, and the monstrous machine plummets from the sky, a wounded bird crashing into the ruins below in a symphony of shattered metal and tortured screams.
Victory.
A strangled cheer erupts from the battered remnants of our squad, echoing through the devastated streets. We did it. We goddamn did it. For a heart-stopping moment, euphoria washes over me, a fragile wave against the relentless tide of fear and exhaustion.
But then, something is wrong. Terribly, profoundly wrong. The cheering abruptly cuts off, replaced by a collective gasp of horror. My blood runs cold as I look up, beyond the smoke and dust, towards the towering structure that houses the Directorate's headquarters – the obsidian monolith they call The Spire.
A new light blooms there, not the sickly green of the Scourge, but an incandescent, malevolent crimson. It pulses, a throbbing, malevolent heart in the belly of the beast, and from its depths emerges… something. Not a machine, not a man, but a shifting, amorphous entity of pure, writhing energy. It hangs suspended in the air, crackling with raw power, its form vaguely humanoid yet grotesquely distorted, tentacles of searing light lashing out, consuming the remnants of the fallen Scourge.
Kai chokes out a single, strangled word: "What… the…"
His question is swallowed by a voice, not digitized, not human, but a chorus of a thousand tormented screams, echoing from the crimson monstrosity. It speaks directly into my mind, a psychic assault that bends my will, claws at my sanity.
"Sparrow," it hisses, its voice a chorus of agony and fury, "you have awakened something… ancient. And now, your revolution… will be our instrument of destruction."
The ground trembles beneath my feet. Terror, cold and primal, grips me, squeezing the air from my lungs. This wasn't in the plan. This wasn't supposed to happen. We were fighting for freedom, not unleashing some eldritch nightmare. And as the crimson entity descends, tendrils reaching for us like skeletal fingers, I realize with sickening clarity: we've just made a far worse enemy than the Directorate ever was.
Special Thanks:
Special thanks to all the following, and many more...
All the model makers, fine tuners, mergers, and tweakers:
- Provides the raw "DNA" for almost all my models.
- Sources of model(s) can be found on the repo pages, especially the "source" repos with link(s) to the model creator(s).
- The place to store, merge, and tune models endlessly.
- THE reason we have an open source community.
- The ability to compress and run models on GPU(s), CPU(s) and almost all devices.
- Imatrix, Quantization, and other tools to tune the quants and the models.
- Llama-Server : A cli based direct interface to run GGUF models.
- The only tool I use to quant models.
Quant-Masters: Team Mradermacher, Bartowski, and many others:
- Quant models day and night for us all to use.
- They are the lifeblood of open source access.
- The universal online/offline tool to merge models together and forge something new.
- Over 20 methods to almost instantly merge model, pull them apart and put them together again.
- The tool I have used to create over 1500 models.
- The go to tool to test and run models in GGUF format.
- The Tool I use to test/refine and evaluate new models.
- LMStudio forum on discord; endless info and community for open source.
Text Generation Webui // KolboldCPP // SillyTavern:
- Excellent tools to run GGUF models with - [ https://github.com/oobabooga/text-generation-webui ] [ https://github.com/LostRuins/koboldcpp ] .
- Sillytavern [ https://github.com/SillyTavern/SillyTavern ] can be used with LMSTudio [ https://lmstudio.ai/ ] , TextGen [ https://github.com/oobabooga/text-generation-webui ], Kolboldcpp [ https://github.com/LostRuins/koboldcpp ], Llama-Server [part of LLAMAcpp] as a off the scale front end control system and interface to work with models.
