Skip to main content

One post tagged with "router"

View all tags

Maybe We Can Do More with the Router

· 5 min read

Since the release of claude-code-router, I've received a lot of user feedback, and quite a few issues are still open. Most of them are related to support for different providers and the lack of tool usage from the deepseek model.

Originally, I created this project for personal use, mainly to access claude code at a lower cost. So, multi-provider support wasn't part of the initial design. But during troubleshooting, I discovered that even though most providers claim to be compatible with the OpenAI-style /chat/completions interface, there are many subtle differences. For example:

  1. When Gemini's tool parameter type is string, the format field only supports date and date-time, and there's no tool call ID.

  2. OpenRouter requires cache_control for caching.

  3. The official DeepSeek API has a max_output of 8192, but Volcano Engine's limit is even higher.

Aside from these, smaller providers often have quirks in their parameter handling. So I decided to create a new project, musistudio/llms, to deal with these compatibility issues. It uses the OpenAI format as a base and introduces a generic Transformer interface for transforming both requests and responses.

Once a Transformer is implemented for each provider, it becomes possible to mix-and-match requests between them. For example, I implemented bidirectional conversion between Anthropic and OpenAI formats in AnthropicTransformer, which listens to the /v1/messages endpoint. Similarly, GeminiTransformer handles Gemini <-> OpenAI format conversions and listens to /v1beta/models/:modelAndAction.

When both requests and responses are transformed into a common format, they can interoperate seamlessly: