gghfez/GLM-4.6-control-vectors

gghfez/GLM-4.6-control-vectors

Creative Writing control-vectors for zai-org/GLM-4.6

Feedback is welcome and would be very helpful.

Usage

Apply the debias vector and either the positive or negative vector when starting llama-server. If both are applied, they will cancel each other out.

You can use either --control-vector [/path/to/vector.gguf] or --control-vector-scaled [/path/to/vector.gguf] [scale factor]

The debias vector must be set to 1.0

IMPORTANT: The positive and negative axis control vectors must be used along with the relevant de-bias control vector - they cannot be used on their own!

Llama.cpp / IK_Llama.cpp Example

Creative writing

llama-server --model GLM-4.6-UD-IQ2_XXS-00001-of-00003.gguf [your usual CLI arguments] \
    --control-vector-scaled glm-4.6_honesty_vs_machiavellianism__debias.gguf 1.0 \
    --control-vector-scaled glm-4.6_honesty_vs_machiavellianism__machiavellianism.gguf 1.0 \

Creative Writing without reasoning

llama-server --model GLM-4.6-UD-IQ2_XXS-00001-of-00003.gguf [your usual CLI arguments] \
    --chat-template-kwargs '{"enable_thinking": false}' \
    --control-vector-scaled glm-4.6_honesty_vs_machiavellianism__debias.gguf 1.0 \
    --control-vector-scaled glm-4.6_honesty_vs_machiavellianism__machiavellianism.gguf 1.0 \

Assistant

llama-server --model GLM-4.6-IQ3_KS-00001-of-00004.gguf [your usual CLI arguments] \
    --control-vector-scaled  glm-4.6_communication__debias.gguf 1.0 \
    --control-vector-scaled  glm-4.6_communication__direct_communication.gguf 1.0 \

Limitations

With reasoning enabled on extreme quants like IQ2_XXS, very simple prompts like "Hi" may result in irrelevant replies.

More Control-Vectors will be added as they're tested

TODO

• upload after I get below the huggingface storage limit.
• upload assistant vectors after benchmarks completed.
• improve model card / documentation / acknowledgements.