A quick fine tune on this model to try to make it actually speak english.
WARNING: IT CAN'T GENERATE COHERENT RESPONSES AND JUST OUTPUTS GIBBERISH
================================================================================ BASIC MODEL SANITY CHECK ================================================================================ Model device: cuda:0 Tokenizer vocab size: 129280 EOS token: <｜end▁of▁sentence｜> 1 PAD token: <｜end▁of▁sentence｜> 1 Input shape: torch.Size([1, 29]) Logits shape: torch.Size([1, 29, 129280]) Finite logits ratio: 1.0 ================================================================================ TOP NEXT TOKEN DEBUG ================================================================================ Prompt: Explain what a neural network is in one short paragraph. 01. id=223 prob=0.031604 token=' ' 02. id=14 prob=0.023512 token=',' 03. id=344 prob=0.018435 token=' is' 04. id=270 prob=0.017867 token=' the' 05. id=304 prob=0.016909 token=' to' 06. id=294 prob=0.010203 token=' of' 07. id=260 prob=0.009461 token=' a' 08. id=579 prob=0.007778 token=' we' 09. id=20 prob=0.007129 token='2' 10. id=16 prob=0.006803 token='.' 11. id=21 prob=0.006707 token='3' 12. id=305 prob=0.006125 token=' and' 13. id=588 prob=0.005795 token=' can' 14. id=24 prob=0.005749 token='6' 15. id=438 prob=0.005410 token=' =' 16. id=90 prob=0.005206 token='x' 17. id=671 prob=0.005185 token='The' 18. id=28 prob=0.005000 token=':' 19. id=295 prob=0.004803 token=' in' 20. id=603 prob=0.004534 token='.\n' ================================================================================ GREEDY TESTS ================================================================================ ================================================================================ GREEDY TEST 1 ================================================================================ PROMPT: Explain what a neural network is in one short paragraph. OUTPUT: , the Ministry song trying term the the background Week ubo the reinforce problem problem Qu problem the computer problem the lobster browse the never theWeek banking song toWeekWeekWeekoiWeekWeekWeekWeekWeek of theWeekWeek. theWeekWeekWeekWeekWeekWeekWeekWeekWeekWeekWeekWeekWeekDATE to theWeekWeekWeekWeek conjugateWeekWeekWeekWeekWeekWeekWeekWeekWeekWeek ================================================================================ GREEDY TEST 2 ================================================================================ PROMPT: Write one vivid sentence about a pirate ship in a storm. OUTPUT: Ministry to is browse a the to to to to to is a a, a a2 of the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the the = ================================================================================ GREEDY TEST 3 ================================================================================ PROMPT: Give me three tips for debugging Python code. OUTPUT: , is of Ministry to the trying a term song Cal reinforce browse problem problemWeek Qu background never withdrawal may songoi problemrackerubo problem song conjugate ================================================================================ GREEDY TEST 4 ================================================================================ PROMPT: What is photosynthesis? OUTPUT: Ministry tounders, the 一线的 trying song ubo the Cal dysfunctional the ofoi the reinforce may theWeek conjugate the the term the Qu withdrawal browse the background the lobster banking theWeek theWeek theWeek a theWeek theWeek the the the the theWeek theWeek the adventure never the the the the the the the theWeek the problem theWeekWeek theWeekWeekWeekWeekWeek theWeek theWeek theWeek theWeek ================================================================================ GREEDY TEST 5 ================================================================================ PROMPT: Write a short friendly message to someone who helped me. OUTPUT: a Ministry is, the ,, of the the the the the the the trying , song term problem problem background Qu never reinforce mayWeek song computer song withdrawaloi songubo song song browse . theWeek song the song ================================================================================ GREEDY TEST 6 ================================================================================ PROMPT: Complete this sentence: The old robot opened its eyes and OUTPUT: Ministry, the to of term song problem Qu never problem background song song trying a mayoi reinforce
