Share your ideas about our platform

Offer separate short-term and long-term support LLM endpoints

I understand that there might be different needs among users of the token factory: build stable applications with reliable LLM performance → requires long-term support (LTS) for the model endpoint (even if the model is somewhat outdated) use the latest and greatest model as soon as it has been released, because it offers the best performance per dollar/euro → only needs short-term support (STS) for the model endpoint Public model endpoints should clearly state “STS” or “LTS”. This way the users can pick depending on individual needs. On Nebius’ side this would allow superseding STS model endpoints without further notice as soon as a superior version of that model gets released - ultimately offering a better service and keeping the “zoo” of model endpoints manageable in the long run. LTS model endpoints on the other hand could have a predetermined EOL date, so developers can plan for necessary model updates.

Jan Schaller 4 months ago

💡 Feature request

Support Implicite/Explicite Prompt Caching

Many other providers offer prompt-caching with discounted pricing for cache hits, either implicitly (e.g., OpenAI, DeepSeek, Google, DeepInfra, NovitaAI, Fireworks) or explicitly (most notably Anthropic). This capability can significantly reduce costs in agentic workflows, where a single session often re-sends the same context repeatedly (for example, when the model performs multiple tool calls in sequence and the shared conversation/context is included each time). Today, Nebius Token Factory is at a cost disadvantage in these repeated, input-token-heavy scenarios compared to providers that support prompt caching and pass the savings through to customers. Please add support for prompt caching (implicit or explicit), including discounted pricing for cached prompt tokens, to improve cost-efficiency for agentic and tool-using applications.

Lukas Kreussel 5 months ago

8

💡 Feature request

Support Latest Top 5 LLM Models

Out of the top 5 models (based on LLM and artificalanalysis) token factory still lacks: - MiniMax M2.5 - GLM 5 - Qwen3.5-397B-A17B - Step 3.5 Flash Additionally MiMo-V2-Flash would be welcome as Step 3.5 Flash only supports 64k token window. I’m quite confident that providing these models in the token factory catalog would provide users frontier proprietary level models, which would greatly help adoption. Additional note: On the main website (nebius.com) when someone hovers over the token factory menu the models that appear in the popup are all relatively old. K2.5 is already provided in token factory, but only K2 is listed. (1) https://llm-stats.com/leaderboards/open-llm-leaderboard (SWE-bench Verified) (2) https://artificialanalysis.ai/models/open-source (intelligence ranking)

davidhidvegi 6 months ago

1

💡 Feature request

/v1/responses OpenAI compatible endpoint

The /v1/responses endpoint from OpenAI is used by tools like Kilo Code and Opencode, and it is particularly useful for agent development and handling large files. I noticed that Nebius does not currently support this, but I believe it would be a valuable addition. Maintaining compatibility with the official OpenAI API would enable Nebius to serve as a backend for tools like Opencode and similar applications.

Roberto Sánchez 6 months ago

💡 Feature request

Token Factory API: Lowercase model IDs

In the Token Factory API, the model selector should be able to be specified as lowercase. For example: Required today: {"model":"Qwen/Qwen3-Coder-480B-A35B-Instruct", … } Wanted: {"model":"qwen/qwen3-coder-480b-a35b-instruct", … } Why? Currently, OpenCode does not work with any Nebius model, even if OpenCode have a built-in integration with Nebius. Nothing works. Any request says “Not found”, and that is an error from your API because they (mistakenly) force lowercase on all model IDs. If I edit the OpenCode config file and Pascal Case the Nebius model IDs, then your API accepts it. This is also easy to test with curl. This is really an OpenCode issue, but maybe you could also edit your API and allow lowercased IDs, for us developers (customers) sake 🙏 Regards, Christoffer

uninjured2875 7 months ago

5

💡 Feature request

Standalone applications support in OpenTofu/Terraform

Currently only Applications for Managed Service for Kubernetes has OpenTofu/Terraform support https://docs.nebius.com/terraform-provider/reference/resources/applications_v1alpha1_k8s_release. It would be great to have OpenTofu/Terraform support for standalone applications such as vLLM as well. Thanks! 😃

hongbo-miao 12 months ago

💡 Feature request