You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Low-Memory Linux-Friendly LLM & Embedding API Suggestion #4
We need LLM and embedded model API more, and most of the Linux environment cannot support GPU with large memory, so it is difficult to deploy/include embedded models locally through vllm. Thank you. Very good product, promising future.
我们更需要 LLM 和嵌入式模型 API,而且大多数 Linux 环境无法支持大内存 GPU,因此通过 vllm 在本地部署/包含嵌入式模型比较困难成本较高。谢谢,很好的产品,未来可期
Activity
RaymondTang2003 commentedon Sep 17, 2025
We will support external APIs in the next update. Thanks for your suggestion! 🙂
RaymondTang2003 commentedon Sep 19, 2025
In the latest version, we have support the external API for LLM, VLM and Embedding Model. Thanks again for your suggestion!!!