Upgrade to 1m context
Any plans to update this to the new 1m context version?
Correction: We need to verify whether config_1m.json can run properly on the quantized model.
We will conduct testing and provide an updated config.json for 1M context in a later update.
We will conduct testing and provide an updated config.json for 1M context in a later update.
How is the testing going I was testing by directly changing the config.json with the addition of the config_1m.json. it is really slow.
Correction: We need to verify whether config_1m.json can run properly on the quantized model.
Any news?
We tested the 1M-context configuration, but the results were unsatisfactory; therefore, we currently have no plans to roll out this update.
We tested the 1M-context configuration, but the results were unsatisfactory; therefore, we currently have no plans to roll out this update.
Thank you for the quick response. Is this for all the quantized models? I wanted to try them to in the AWQ version. Also can you more specific about what were the results?