What If I told you I found 5 layers out of the 32 that do absolutely nothing.
Cutting these layers out do not change the output for my model at all individually, but cutting all 5 at once breaks the model completely.
Lets take 4 layers out of 5 out and see if we can recover after removing this redundancy.
Please Give my Mermaid-Llama-3-8B and This Mermaid-Llama-3-Pruned-7B and the rest of the pruned Mermaids.
28/32 Layers Model.
license: cc-by-4.0
Mermaid-Llama-3-7B
Introducing Mermaid-LLama-3-7B, a robust language model designed for Python code understanding and crafting captivating story flow maps.
Pruned down to 7 billion parameter.
See MergeKit Notes And Try Triming my model yourself and explore my world of trimming models to fit SMARTER Models with lower requirements f
or specific tasks. Mermaid is just a start, Hire me to solve your problem and I will build the smallest footprint model that solves just that problem.
I wish to specialize in packing models on Edge Devices.
Open For Hire See my links to my Linkedin for more.
MermaidLlama GIF
Key Features
Code Understanding:
Masters Python intricacies with finesse.
Generates clear and accurate Mermaid Diagram Flow Charts.
Ideal for developers seeking visual representations of their code logic.
Storytelling Capabilities:
Converts narrative inputs into captivating Mermaid Diagrams.
Maps character interactions, plot developments, and narrative arcs.
Unmatched Performance:
Surpasses GPT-4 in generating well-organized Mermaid Diagrams.
Training Insights:
Trained on a diverse dataset, including 800 unique, hand-curated Mermaid Graph examples utilizing 478 complete Python programs.
Exhibits emergent properties in story-to-flow map translations and step-by-step instruction flow maps.
Collaboration
Interested in enhancing Mermaid's capabilities? Contact troydoesai@gmail.com for collaboration opportunities.
Example Use Cases
Retrieval-Augmented Generation (RAG): Utilize Mermaid-LLama-3-8B to create condensed knowledge graphs. This model excels in generating flow diagrams that enhance the retrieval process. These knowledge graphs are stored in a vector database, which allows for quick and efficient retrieval of contextually relevant information. When a query is received, the system retrieves a pertinent knowledge graph, appending it as context to the model. This enriched context enables Mermaid-LLama-3-8B to deliver more accurate and nuanced responses. This approach is particularly beneficial in applications requiring deep, context-aware interactions, such as sophisticated Q&A systems, dynamic data analysis, and complex decision-making tasks.
Code Documentation: Automatic visual flow charts from Python code.
Storyboarding: Visually appealing diagrams for storytelling.
Project Planning: Visual project flow maps for effective team communication.
Learning Python: Helps students visually understand Python code structures.
Game Design: Visualizing game storylines for coherent narrative structure.
Proof of Concept
Stay tuned for the release of the VSCode Extension that displays the Live Flow Map every time a user stops typing for more than 10 seconds.
Training Specifications
LoRA Rank: 2048
LoRA Alpha: 4096
Batch Size: 1
Micro Batch Size: 1
Cutoff Length: 4096
Save every n steps: 1000
Epochs: 3
Learning Rate: 1e-6
LR Scheduler: Cosine
Target Modules:
Enable q_proj
Enable v_proj
Enable k_proj
Enable o_proj
Enable gate_proj
Enable down_proj
Enable up_proj
Getting Started
Start by downloading one of my models.
0 TroyDoesAI GIF
Load the model.
1 Load Model in 4-bit Show Example Use GIF
Use my prompt template to generate a Mermaid code block, which can be viewed in the Mermaid Live Editor or using the Mermaid CLI tool.
2 Loaded Model in Full Precision 16-bit Show Inference and Mermaid Live Editor GIF
Here we open the VLLM GUI Program while still running in Vram the Mermaid-Llama-8B to compare the flow diagram to the actual program and show the lightweight capabilites of small models on consumer hardware.
3 Open The Program VLLM Program With Full Precision Mermaid-Llama-8B Running to Evaluate Flow Map GIF
Note: This model should be treated as an Auto-Complete Model, Do not try talking to it in chat you are gonna get garbage, those layers have been pruned and replaced, that is all you will hear of my secret sauce on training on small < 1000 entry datasets.
