sentence transformers usage instructions unclear
This might be a noob question but the model card simply copies over the model usage instructions from google/embeddinggemma-300m. It does not actually describe how to actually use the unsloth/embeddinggemma-300m embedding model.
Are we expected to manually download the model and pass in the path to that model to sentence transformers
@ruddbanga7 you can use the GGUF embeddings via llama.cpp and e.g. expose it as API or use a GUI tool like LM Studio. https://huggingface.co/ChristianAzinn/mxbai-embed-large-v1-gguf#example-usage-with-lm-studio has an instruction for the latter, though it may be a bit outdated screenshot-wise.
Using llama-cpp-python also fails to load this model. It encounters the error 'LlamaModel' object has no attribute 'sampler'. Is this an issue with the model parameters or with llama-cpp-python?
@taotaoche
the LlamaModel object has no attribute sampler error is because the llama-cpp-python binding hasn't added support for Gemma Models yet, but the upstream llama.cpp has.
I have been using a fork by inference-sh which is maintained and resolves this error, but there I can't do batch processing of embeddings and I have already opened an issue for that. Link to fork: https://github.com/inference-sh/llama-cpp-python