GPU poor - No CUDA
#1
by
jeanbaptdzd
- opened
Hey,
Trying to run it in a GPU poor setup. Can you please mention which libs are CUDA/nvcc/triton dependent ? Is quantization an option ?
Hi, thanks for your comment, we improved the readme, hope it helps !
We provide a script to load and run the model using the PyTorch eager path : https://gist.github.com/alxndrTL/3f98ccbf671c014e7a66002c4ff177ec
No matter your environment, it will use default PyTorch operation and thus should be runnable on a lot of different setups