VRAM required to run inference

by niktheod - opened Sep 22

Sep 22

•

Hi. I have 16GBs of VRAM available but when I try to run the inference code you provide on hugging face I run out of memory. Any idea why this could be? Usually for models of similar size 16GBs of VRAM are more than enough.

niktheod

Sep 24

Turns out that the implementation of Ovis2_5 simply doesn't have a @torch .no_grad() decorator (I don't know why I guess it should have it?). So either adding it to the source code or simply adding it when calling the method solves the problem.

niktheod changed discussion status to closed Sep 24

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment