Spaces:
Running
on
A10G
Running
on
A10G
Arm optimized quants
#113
by
SaisExperiments
- opened
Is it possible to have the
Q4_0_4_4
Q4_0_4_8
Q4_0_8_8
formats added?
Thanks for the feature request; let's wait a bit to see if we get more requests for this.
reach-vb
changed discussion status to
closed
reach-vb
changed discussion status to
open
Afaik there is no support for those quants anymore... Instead there is something called online repacking.