raxtemur
/

sonar-llm-1.3b

@@ -12,7 +12,7 @@ library_name: transformers
 pipeline_tag: text-generation
 ---
-# SONAR-LLM (1.3M) -- Text summarization checkopoint
 We present SONAR-LLM, a decoder-only transformer that "thinks" in the same continuous SONAR embedding space, yet is supervised through token-level cross-entropy propagated via the frozen SONAR decoder. This hybrid objective retains the semantic abstraction of LCM while eliminating its diffusion sampler and restoring a likelihood-based training signal. Across model sizes from 39M to 1.3B parameters, SONAR-LLM attains competitive generation quality.

 pipeline_tag: text-generation
 ---
+# SONAR-LLM (1.3B) -- Text summarization checkopoint
 We present SONAR-LLM, a decoder-only transformer that "thinks" in the same continuous SONAR embedding space, yet is supervised through token-level cross-entropy propagated via the frozen SONAR decoder. This hybrid objective retains the semantic abstraction of LCM while eliminating its diffusion sampler and restoring a likelihood-based training signal. Across model sizes from 39M to 1.3B parameters, SONAR-LLM attains competitive generation quality.