My thoughts after one role playing session.
TheDrummer/GLM-Steam-106B-A12B-v1 is faster with MMAP (koboldcpp) , speed around 2.5T/s at 16ctx, without its 1.8T/s on my 3090. I offload 18 out of 48 layers iQ4_XS gguf. Model is not censored, creative and assertive. Lot of variety between swipes. Promising. Issues: 1. model assumed me in jeans while I wrote message earlier that I undress. 2. Other example '[...] I say dispassionately not even looking at her while eating.' Reply '[...] She doesn't break eye contact. [...]'. 3. Another example '*She doesn't walk towards you, but towards the bedroom door. She pauses in the doorway, not looking back. "Just… give me a minute.' Reply 'my surprised expression and inner thoughts' , 'Over ten minutes passes. Finally, the sound of the bathroom door opening drifts through the house. She emerges, [...]' | Everything else is good that is creativity, variety, assertiveness, character consistency. Not tested instruction following.
The prose in the sample is a lot better than Air at least. Can't test it myself. Just appreciating TheDrummer's work.