How to run it on TGI
Is it possible to run this model on TGI?
TypeError: tgi_flash_attention_forward() missing 8 required positional arguments: 'kv_cache', 'kv_head_mapping', 'slots', 'cu_seqlen_prefill', 'seqlen', 'block_tables', 'max_s', and 'kv_scales'
2025-11-27T17:20:46.511566Z ERROR warmup { max_input_length Some(14336) max_prefill_tokens 11484 max_total_tokens None max_batch_size None } : warmup : text_generation_router_v3::client : backends/v3/src/client/mod.rs : 45: Server error: tgi_flash_attention_forward() missing 8 required positional arguments: 'kv_cache', 'kv_head_mapping', 'slots', 'cu_seqlen_prefill', 'seqlen', 'block_tables', 'max_s', and 'kv_scales'
Error: Backend(Warmup(Generation("tgi_flash_attention_forward() missing 8 required positional arguments: 'kv_cache', 'kv_head_mapping', 'slots', 'cu_seqlen_prefill', 'seqlen', 'block_tables', 'max_s', and 'kv_scales'")))