ONNX conversion
@Xenova
@JasonMayes
Could you please help in converting this model to ONNX so that this can be used with Transformers.js? Looking at the output from the model, it seems a solid bet for in-browser ML use-cases.
I mean just think about the improvements for any in-browser AI agent by feeding the output from this model, into phi-3.5-vision (or any other vision model that is already present in ONNX format)!!
I've converted the caption model (Florence-2) here: https://huggingface.co/onnx-community/OmniParser-v2.0_icon_caption, and the icon_detect model can be converted using the original ultralytics/yolo converter (I haven't done myself, but hopefully someone else wants to and can upload to the onnx-community org).
I've converted the caption model (Florence-2) here: https://huggingface.co/onnx-community/OmniParser-v2.0_icon_caption, and the icon_detect model can be converted using the original ultralytics/yolo converter (I haven't done myself, but hopefully someone else wants to and can upload to the onnx-community org).
Hi!
@Xenova
, I've been trying to use the onnx-community/OmniParser-v2.0_icon_caption model with transformers.js for a few weeks already but with no success. I've been using the example for v1 here https://scrimba.com/s08johf0et, but the v2 model seems to be failing silently on the generate step.
Is there any example or guidance you could provide on how to get this model working with transformers.js?
Thank you in advance!