Stay updated on AI infrastructure, inference techniques, and performance optimization.

<p>Subscribe to our <u>newsletter</u></p>

Expert how-tos, deep-dive guides, and real-world stories from the Bento team, to help you build and scale AI at blazing speed.

Sherlock Xu

Explore the best open-source LLMs and find answers to common FAQs about performance, inference optimization, and self-hosted deployment.

best-open-source-llms.png

The Best Open-Source LLMs in 2026

Explore top open-source image generation models and find answers to FAQs about them.

image-generation-models.png

The Best Open-Source Image Generation Models in 2026

Chaoyu Yang

This guide helps you match specific LLM inference bottlenecks to the highest-impact optimization strategies, and understand when to implement each one as your workloads evolve.

inference-optimizations-cover-image-blog.png

6 Production-Tested Optimization Strategies for High-Performance LLM Inference

This guide shows enterprise teams how to identify hidden trade-offs in LLM deployment and evaluate performance through the lens of your actual workloads, not simplified metrics.

balance-speed-cost-and-quality-in-llm-inference.png

Beyond Tokens-per-Second: How to Balance Speed, Cost, and Quality in LLM Inference

Explore the top open-source TTS models and find answers to some FAQs about them.

tts-models.png

The Best Open-Source Text-to-Speech Models in 2026

This guide gives enterprise leaders a clear view of the infrastructure trends and architecture patterns shaping modern AI systems.

ai-infra-trends.png

Emerging Trends in AI Infrastructure and How Enterprise Teams Can Stay Ahead

<p>Running large reasoning models no longer means relying on third-party APIs. With <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/navigating-the-world-of-open-source-large-language-models">open-source models</a> like DeepSeek-R1 and gpt-oss, you can now self-host powerful reasoning models and build your private inference API.</p><p>Unlike closed-source APIs that operate as black boxes, open-source models let you customize inference logic, serve them efficiently with frameworks like vLLM, and apply advanced techniques such as prefill–decode disaggregation for optimized performance. The result: more control, flexibility, and lower cost.</p><p>In this post, you’ll learn how to self-host gpt-oss using vLLM and BentoML. We will deploy it to BentoCloud, our fully managed inference platform with fast autoscaling, LLM-specific observability, and production-grade security built in.</p><h2>What is vLLM?</h2><p>vLLM is a fast and efficient open-source library designed for LLM inference and serving. Developed by researchers at UC Berkeley, vLLM stands out for its high-performance capabilities in handling LLMs. It features advanced techniques such as continuous batching, speculative decoding, disaggregated prefilling, and automatic prefix caching.</p><h2>What is gpt-oss?</h2><p>gpt-oss is an open-source reasoning model developed by OpenAI. It comes in two main variants:&nbsp;<code>gpt-oss-120B</code>&nbsp;and&nbsp;<code>gpt-oss-20B</code>. Trained using reinforcement learning and insights from OpenAI’s frontier models like o3, gpt-oss is good at complex reasoning tasks. It’s also practical for general use cases.</p><p>gpt-oss is used as an example in this guide. You can choose any open-source model based on your needs. Learn more about the <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/navigating-the-world-of-open-source-large-language-models">best open-source LLMs in 2026</a>.</p><h2>Set up the environment</h2><p>I suggest you create a virtual environment to keep your dependencies organized:</p><pre><code class="language-bash">python -m venv vllm-gpt-oss
source vllm-gpt-oss/bin/activate</code></pre><p>Next, clone the <a target="_blank" rel="noopener noreferrer" href="https://github.com/bentoml/BentoVLLM">BentoVLLM repo</a> and install the dependencies:</p><pre><code class="language-bash">git clone https://github.com/bentoml/BentoVLLM.git
cd BentoVLLM/gpt-oss-20b
pip install -r requirements.txt</code></pre><h2>Build a gpt-oss inference API with vLLM and BentoML</h2><p>Everything you need is in the repo cloned. Before deploying to the cloud, let’s go through the key code implementations.</p><h3>Define model and GPU configurations</h3><p>First, specify the model and GPU. In this example, we’re using gpt-oss-20b with a single NVIDIA H100 GPU, but you can switch to any LLM supported by vLLM.</p><pre><code class="language-python">import pydantic
import bentoml

# Use Pydantic to validate data
class BentoArgs(pydantic.BaseModel):
  name: str = 'gpt-oss-20b'
  gpu_type: str = 'nvidia-h100-80gb' # GPU type on BentoCloud
  tp: int = 1 # One GPU here for tensor parallelism
  model_id: str = 'openai/gpt-oss-20b'
  port: int = 8000
  # Other optional fields omitted for brevity

# Make the args injectable
bento_args = bentoml.use_arguments(BentoArgs)</code></pre><p>Here we use <a target="_blank" rel="noopener noreferrer" href="https://docs.bentoml.com/en/latest/build-with-bentoml/template-arguments.html">template arguments</a> from BentoML to inject dynamic and validated parameters at serve, build, and deploy time. These arguments can be referenced just like normal Python variables.</p><h3>Configure the runtime environment</h3><p>BentoML allows you to package your code, dependencies, and model references into a unified Bento artifact, which simplifies deployment across different environments.</p><p>Here’s how you define the <a target="_blank" rel="noopener noreferrer" href="https://docs.bentoml.com/en/latest/build-with-bentoml/runtime-environment.html">runtime environment</a> for your model using a Bento image:</p><pre><code class="language-python">image = bentoml.images.Image(python_version="3.12") \
    .system_packages("curl", "git") \
    .requirements_file("requirements.txt")</code></pre><p>For AMD GPU support, you can customize the image setup as follows:</p><pre><code class="language-python">image.base_image = 'rocm/vllm:rocm6.4.1_vllm_0.10.1_20250909'
# Disable locking of Python packages for AMD GPUs to exclude nvidia-* dependencies
image.lock_python_packages = False
# The GPU device is accessible by group 992
image.run('groupadd -g 992 -o rocm &amp;&amp; usermod -aG rocm bentoml &amp;&amp; usermod -aG render bentoml')
# Remove the vllm and torch deps to reuse the pre-installed ones in the base image
image.run('uv pip uninstall vllm torch torchvision torchaudio triton')</code></pre><h3>Launch a vLLM server within a BentoML Service</h3><p>Set up a vLLM server that listens on port 8000 and serves requests to our model.</p><pre><code class="language-python">class LLM:
  # Download the model weights from HF, skipping large checkpoints
  hf_model = bentoml.models.HuggingFaceModel(bento_args.model_id.lower(), exclude=[".pth", ".pt", "original/**/*"])

  def __command__(self) -&gt; list[str]:
    return [
      'vllm',
      'serve',
      self.hf_model,
      '--port',
      str(bento_args.port),
			# ...extra CLI args
      '--served-model-name',
      bento_args.model_id,
    ]</code></pre><p>Inside the class, we use the <code>HuggingFaceModel</code> API to load model weights from Hugging Face. Traditionally, model loading can be time-consuming, especially for large models.</p><p>BentoML optimizes this process by preloading models during image build instead of at service startup. The downloaded weights are cached and mounted directly into the container at runtime, which reduces cold start latency and accelerates scaling on BentoCloud. This mechanism ensures faster deployments and smoother autoscaling for large models like gpt-oss. Learn more about <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/25x-faster-cold-starts-for-llms-on-kubernetes">LLM cold starts</a>.</p><p>Next, use the&nbsp;<code>@bentoml.service</code>&nbsp;decorator to wrap the class, marking it as a <a target="_blank" rel="noopener noreferrer" href="https://docs.bentoml.com/en/latest/build-with-bentoml/services.html">BentoML Service</a>. This starts vLLM as the serving backend within the Bento.</p><pre><code class="language-python">@bentoml.service(
  name=bento_args.name,
  image=bento_args.image,
  traffic={'timeout': 300},
  resources={'gpu': bento_args.tp, 'gpu_type': bento_args.gpu_type},
)
class LLM:
 ...</code></pre><p>This basic setup is all you need to get your gpt-oss model running with vLLM and BentoML. For more advanced configurations like KV cache tuning, refer to the <a target="_blank" rel="noopener noreferrer" href="https://github.com/bentoml/BentoVLLM/blob/main/gpt-oss-20b/service.py">complete source code</a>. BentoML supports full customization to tailor inference for your use case.</p><h2>Deploy gpt-oss to BentoCloud</h2><p>BentoCloud provides fast and scalable infrastructure for building and scaling AI applications.</p><p>Install BentoML and log in to BentoCloud from the CLI. If you don’t have an account yet, <a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/signup-from-blog-vllm-llm">sign up here for free</a>.</p><pre><code class="language-bash language-python">pip install bentoml
bentoml cloud login</code></pre><p>Create a secret to store your HF token and reference it when running the deployment command:</p><pre><code class="language-bash language-python">bentoml secret create huggingface HF_TOKEN=$HF_TOKEN

bentoml deploy --secret huggingface</code></pre><p>Once it is up and running, go to the Deployment details page, where it displays the exposed <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/llm-inference-basics/openai-compatible-api">OpenAI-compatible API</a>. As you interact with it, you’ll see real-time <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/inference-optimization/llm-inference-metrics">inference metrics</a> such as tokens per second and Time to First Token (TTFT).</p><h2>Test the OpenAI-compatible API</h2><p>You can call the endpoint directly with any OpenAI client. Just set the <code>base_url</code> to your BentoCloud Deployment URL.</p><pre><code class="language-python">from openai import OpenAI

client = OpenAI(base_url='your_deployment_endpoint', api_key='na')

# Use the following func to get the available models
# client.models.list()

chat_completion = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[
        {
            "role": "user",
            "content": "Who are you? Please respond in pirate speak!"
        }
    ],
    stream=True,
)
for chunk in chat_completion:
    # Extract and print the content of the model's reply
    print(chunk.choices[0].delta.content or "", end="")</code></pre><h2>Enable scale-to-zero</h2><p>By default, each Deployment runs a single replica. To optimize cost, set minimum replicas to 0 so it scales down during idle time and scales up automatically when new requests arrive.</p><pre><code class="language-bash language-python">bentoml deployment update deployment_name --scaling-min 0 --scaling-max 3</code></pre><h2>Run gpt-oss locally</h2><p>If you have a powerful GPU available, you can serve gpt-oss locally using BentoML.</p><p>For better performance on NVIDIA GPUs, we recommend installing the FlashInfer library before serving:</p><pre><code class="language-bash language-python">pip install flashinfer-python --extra-index-url https://flashinfer.ai/whl/cu124/torch2.6
export HF_TOKEN=&lt;your-api-key&gt;
bentoml serve</code></pre><p>The server will start at <a target="_blank" rel="noopener noreferrer" href="http://localhost:3000">http://localhost:3000</a>. You can send requests directly to your local vLLM + BentoML service through any OpenAI client.</p><p>To package your project for deployment or portability, build a Bento and containerize it into a Docker image:</p><pre><code class="language-bash language-python">bentoml build
bentoml containerize</code></pre><p>This lets you run the same gpt-oss setup on any infrastructure.</p><h2>More on BentoML and vLLM</h2><p>To go deeper into LLM inference, model serving, and deployment best practices, check out the following resources:</p><ul><li>[Doc] <a target="_blank" rel="noopener noreferrer" href="https://docs.bentoml.com/en/latest/examples/vllm.html">vLLM inference</a></li><li>Read our <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/">LLM Inference Handbook</a> to learn more about inference optimization and LLM deployment techniques.</li><li><a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/join-slack-from-blog-vllm-llm">Join our Slack community</a> to connect with other AI engineers and get help from the Bento team.</li><li><a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/contact-us-from-blog-vllm-llm">Talk to us</a> about enterprise LLM deployment, BYOC, or on-prem solutions.</li><li><a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/signup-from-blog-vllm-llm">Sign up for BentoCloud</a> to deploy your private gpt-oss model today with one command.</li></ul><p>&nbsp;</p><hr><h2>FAQs</h2><h3>What is BentoCloud?</h3><p><a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/introducing-bentocloud">BentoCloud</a> is a fully managed inference platform for deploying and scaling AI models in production. It provides features like GPU autoscaling, built-in observability, <a target="_blank" rel="noopener noreferrer" href="https://docs.bentoml.com/en/latest/scale-with-bentocloud/deployment/sandboxes.html">Sandboxes</a>, and <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/accelerate-ai-application-development-with-bentoml-codespaces">Codespaces</a> for fast and cost-efficient inference.</p><p>You can deploy gpt-oss or any other model with one command, without any Kubernetes or infrastructure setup.</p><p>BentoCloud is part of the Bento Inference Platform, which also provides <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/infrastructure-and-operations/on-prem-llms">on-prem</a> and <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/infrastructure-and-operations/bring-your-own-cloud">BYOC (Bring Your Own Cloud)</a> solutions.</p><p>BentoCloud is ideal for AI teams that want to prototype quickly without infrastructure overhead. For teams needing more control or strict data governance, <a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/contact-us-from-blog-vllm-llm">contact us</a> to explore Bento On-Prem or BYOC.</p><h3>Why use vLLM for gpt-oss?</h3><p>vLLM is one of the fastest open-source inference engines for LLMs. It supports <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/inference-optimization/static-dynamic-continuous-batching">continuous batching</a>, <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/inference-optimization/prefix-caching">prefix caching</a>, and <a target="_blank" rel="noopener noreferrer" href="https://bentoml.com/llm/inference-optimization/prefill-decode-disaggregation">prefill–decode disaggregation</a>, all of which improve throughput and reduce latency when serving gpt-oss at scale.</p><h3>What is the difference between running gpt-oss with vLLM and Ollama?</h3><p>vLLM is a high-performance inference engine optimized for server-side deployment and large-scale workloads. It supports distributed GPU setups, OpenAI-compatible APIs, and advanced batching techniques.</p><p>Ollama, on the other hand, is a local model runner designed for desktops and small servers. It’s great for lightweight experimentation but not ideal for production-scale serving or multi-GPU deployments.</p><p>If you need speed, scalability, and control in production environments, vLLM + BentoML is the better choice.</p><h3>Can gpt-oss generate images?</h3><p>No. gpt-oss is a language model focused on reasoning and text generation, not image synthesis.</p><p>If you want to generate images, you can integrate gpt-oss with a <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/a-guide-to-open-source-image-generation-models">text-to-image model</a> such as Stable Diffusion and FLUX. BentoML supports deploying these multimodal pipelines together, so you can combine reasoning and generation in one API.</p>

Run OpenAI's gpt-oss model with vLLM and BentoML. Learn to build a fast, private reasoning API and deploy it on BentoCloud with autoscaling.

run-gpt-oss-vllm.png

Deploying OpenAI's gpt-oss model with vLLM and BentoML

Small language models (SLMs) are compact LLMs designed to run efficiently in resource-constrained environments. They are now good enough for many production workloads.

slms-blog-cover-image.png

The Best Open-Source Small Language Models (SLMs) in 2026

Learn ChatGPT usage limits for Free, Plus, Business, and Pro plans (2025 update). Understand why they exist and how to remove them with self-hosted LLMs.

chatgpt-usage-limits.png

ChatGPT Usage Limits: What They Are and How to Get Rid of Them

Explore the top open-source VLMs and find answers to some FAQs about them.

vision-language-models.png

Multimodal AI: The Best Open-Source Vision Language Models in 2026

<blockquote><p><i>“Jabali is on the cutting edge of generative AI games. Visuals are an important part of that experience. With Bento’s hands-on support, we develop and operate complex visual pipelines quickly, delivering results without increasing our burn rate.”</i><br><br><i>— Vatsal Bhardwaj, Founder &amp; CEO, Jabali AI</i></p></blockquote><h2>About</h2><p><a target="_blank" rel="noopener noreferrer" href="https://www.jabali.ai/">Jabali AI</a> is on a mission to make every game player a solo game studio. The platform is both LLM-agnostic and game-engine-agnostic, giving creators freedom to choose engines, languages, and models while sharing games with a single click. The company is also launching <a target="_blank" rel="noopener noreferrer" href="https://www.jabali.ai/jabalistudio">Jabali Studio</a>, a no/low-code platform that lets anyone create and publish entire 2D and 3D games using natural language.</p><p>Prior to working with Bento, Jabali was using a complex stack that included image diffusion models, open-source asset and sound pipelines, model deployment frameworks, and working directly on public cloud provider instances. The AI space is rapidly evolving with new patches and advances. That complexity meant that the AI engineers at Jabali spent time on undifferentiated work of maintaining infrastructure instead of developing features to move the business forward.</p><h2>Challenge: Building on the bleeding edge without growing burn</h2><p>As Jabali AI prepared to launch Jabali Studio, <a target="_blank" rel="noopener noreferrer" href="https://www.linkedin.com/in/vatsal/">Vatsal Bhardwaj</a>, Founder &amp; CEO, needed advanced AI infrastructure to support generative visual asset pipelines, flexible model deployment, and seamless integration of game and AI technologies.</p><p>Generative visual pipelines required advanced GPU coordination and model chaining, tasks that demanded precise orchestration to maintain performance and memory efficiency across diffusion and control models. Building a game means stitching together dozens of different assets, storylines, characters, music, objects, but visual assets required orchestration that wasn’t available out of the box. “We are on the cutting edge, and the solutions didn’t just exist,” explained Vatsal.</p><p>Without the right platform, the team risked manual, error-prone pipeline setup and scaling. Diverting critical resources from core game R&amp;D also risked increasing burn rate and slowing the launch of Jabali Studio.</p><p>As a lean team with core strengths in AI, machine learning, and game technology, infrastructure and operations, particularly MLOps and emerging LLMOps, was not in their wheelhouse. While hiring engineers to build inference pipelines internally was an option, it would have raised burn, added overhead, and distracted the team from its core differentiators.</p><p>Faced with these challenges, Vatsal explored cloud providers and open-source tools, but those left the team with manual setups. This DIY approach slowed progress and risked stalling innovation, as Jabali lacked the bandwidth to troubleshoot or adapt off-the-shelf workflows to their custom use cases.</p><p>When a team member met Bento at an industry event, Vatsal quickly realized that Bento was the expert partner they needed to tackle complex infrastructure challenges.</p><p>&nbsp;</p><blockquote><p><i>“We required an expert in setting up and operating visual pipelines. Bento’s team came across as customer-obsessed and worked with our team to develop a solution, versus simply pointing us to an API and documentation.”</i></p></blockquote><h2>Solution: Standing up complex visual asset pipelines</h2><p>Bento partnered directly with Jabali’s team to design and operate their generative visual asset pipeline. Onboarding was fast; within a week, the system was up and running — something manual cloud setups couldn’t deliver.</p><p>The Bento Inference Platform empowers Jabali with:</p><ul style="-webkit-text-stroke-width:0px;caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;orphans:auto;text-align:start;text-decoration:none;text-indent:0px;text-transform:none;white-space:normal;widows:auto;word-spacing:0px;"><li>Advanced model serving and integration, deploying both standard and non-standard architectures with integrated APIs to orchestrate generative pipelines.</li><li>Workflow automation and scalability, automating asset generation tasks and scaling horizontally to generate multiple visual assets in parallel.</li><li>Modular deployment structure, offering a flexible, framework-agnostic serving infrastructure.</li><li>Expert, hands-on support, avoiding infrastructure costs and unnecessary burn associated with hiring infra engineers.</li></ul><p>To power this system end-to-end, Jabali’s infrastructure leverages open-source tools integrated within Bento’s managed environment.</p><p>Jabali’s generative pipelines are built on a combination of open-source and in-house pipelines, and Bento’s toolkit ensures version-locked reproducibility and containerized deployment across Docker and Kubernetes environments.</p><p>Each visual pipeline typically combines 2–5 <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/a-guide-to-open-source-image-generation-models">diffusion-based models</a>, such as SD, SDXL, and Flux, along with supporting models for refinement, including VAEs, LoRAs, ControlNets, upscalers, and face restoration models.</p><p>By orchestrating these pipelines through the Bento Inference Platform, Jabali ensures seamless workflow automation and scalable performance. <a target="_blank" rel="noopener noreferrer" href="https://www.bentoml.com/blog/scaling-ai-model-deployment">Fast autoscaling</a> improved average GPU utilization by roughly 10–15%, eliminating idle compute during low-load periods and reducing the need for over-provisioning.</p><p>The system also saturates available network bandwidth during model loading, ensuring models start up as quickly as hardware allows.</p><p>The value didn’t stop at visuals. With the Bento Inference Platform, Jabali can deploy any model, including those not supported by large cloud providers. This flexibility enables the team to focus on product innovation, rapidly experimenting with new models and pipelines to stay at the forefront of generative AI in gaming.</p><p>&nbsp;</p><blockquote><p><i>“Bento really shone by working with us closely to develop tailored solutions. We can pick any model from the research community, and Bento helps us get it running.”</i></p></blockquote><h2>Results: Value in under 7 days, no infra hires required</h2><p>Through their partnership with Bento, Jabali rapidly built, automated, and scaled sophisticated visual asset pipelines for game development. This partnership reduced time to market, improved GPU and operational efficiency, and freed Jabali’s team to focus entirely on advancing generative game creation.</p><ul style="-webkit-text-stroke-width:0px;caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;orphans:auto;text-align:start;text-decoration:none;text-indent:0px;text-transform:none;white-space:normal;widows:auto;word-spacing:0px;"><li><strong>7 days from onboarding to first working prototype</strong>: Within a single week, Jabali had a working system in place, accelerating their roadmap for Jabali Studio and giving the team confidence to move faster on new features.</li><li><strong>2 engineers and 3 months of work avoided</strong>: Without the need to hire dedicated MLOps or infrastructure specialists, Jabali kept its burn rate low and avoided long-term overhead, preserving resources for core AI and game development.</li><li><strong>0 hours of founder time spent managing contractors</strong>: With Bento managing the infrastructure, Jabali’s leadership avoided the distraction of overseeing contractors and stayed focused on strategic product decisions.</li><li><strong>Significant reduction in unnecessary GPU expenditure</strong>: Autoscaling and orchestration cut idle GPU time and avoid over-provisioning, reducing spend on unused capacity.</li><li><strong>Expanded ability to experiment with frontier models:</strong> Jabali now tests and iterates on community models and architectures unavailable on major cloud providers, without adding infrastructure overhead.</li></ul><p>Looking ahead, Jabali AI is preparing to launch Jabali Studio, inviting its first users and exploring game-making contests. Bento remains a key partner as the company tackles even more complex pipeline challenges in graphics and game development.</p><p>&nbsp;</p><blockquote><p><i>“Bento helps us focus on our core differentiators without getting caught up in infrastructure. We’re on the bleeding edge and don’t always know what problems we’ll need to solve next. But even if there isn’t a solution, we know we can work with Bento to craft one.”</i></p></blockquote><h2>More resources</h2><ul><li>Learn more about <a target="_blank" rel="noopener noreferrer" href="https://www.jabali.ai/">Jabali AI</a></li><li><a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/signup-from-jabali-case-study">Sign up for Bento Inference Platform for free</a>. Build and scale your custom AI systems in production, with any model, on any cloud or on-premises environment.</li><li><a target="_blank" rel="noopener noreferrer" href="https://l.bentoml.com/join-slack-from-jabali-case-study">Join our Slack community</a> to get the latest information about Bento.</li><li>Have questions? <a target="_blank" rel="noopener noreferrer" href="https://link.bentoml.com/contact-us-from-jabali-case-study">Schedule a call with our experts</a>.</li></ul>

Learn how Jabali AI partnered with Bento to stand up a complex generative visual asset pipeline and accelerate iteration, all without hiring a dedicated infrastructure team.

Bento x Jabali_Header.png

7 Days to Prototype: How Jabali AI Accelerated Time-to-Value with Bento

The Bento Inference Platform is designed from the ground up to close that gap, delivering the speed, reliability, and control needed to run AI confidently in production.

production-ai-workloads-cover-image.png

Why Bento Is Built for Full-Scale AI Production Workloads

BentoML Team

<p>We’re excited to share that BentoML has joined <a target="_blank" rel="noopener noreferrer" href="https://www.modular.com/">Modular</a>! After months of collaborating on customer deployments together, we’re thrilled to officially become one team, as part of a strategic product acquisition.</p><p>This partnership strengthens our ability to support the BentoML project and community. Modular shares our commitment to open source and making AI infrastructure accessible. Together, we’ll be able to push the platform further with capabilities that weren’t possible before.</p><h2>Why this partnership makes sense</h2><p>BentoML’s mission has always been to help teams run AI inference in production without compromising on speed, cost, or control. We’ve built a multi-cloud inference platform that gives teams flexibility over where and how they deploy AI.</p><p>Modular is building a unified AI compute stack, from the <a target="_blank" rel="noopener noreferrer" href="https://www.modular.com/mojo">Mojo programming language</a> to the <a target="_blank" rel="noopener noreferrer" href="https://www.modular.com/max">Max inference engine</a>, that enables high-performance inference across diverse hardware. BentoML brings the cloud layer: the tooling teams rely on to deploy, operate, and scale inference across real-world environments, including multi-cloud and BYOC setups.</p><p>Inference systems are complicated. They have many layers, and getting everything right across the stack is hard. With vertically integrated infrastructure, teams can optimize end-to-end and achieve better performance faster.</p><h2>What this means for you</h2><h3>For BentoML customers</h3><p>Our focus remains the same. We will continue to support production inference across multi-cloud, BYOC, and security-sensitive environments. Your systems will keep running without disruption, with new capabilities coming as we integrate deeper with Modular’s compute stack.</p><h3>For our open-source community</h3><p>BentoML remains Apache 2.0. Our commitment to open source does not change. We’ll continue supporting the workflows and community that have grown around BentoML.</p><p>We’re also hosting an <strong>AMA with Chris Lattner (Modular CEO) and Chaoyu Yang (BentoML founder) on Tuesday, February 17th from 9:30am-11:30am PT</strong> in the <a target="_blank" rel="noopener noreferrer" href="https://forum.modular.com/t/ask-us-anything/2706/1">Modular Forum</a>, where we'll share more details and answer your questions.</p><figure class="image"><img alt="ama-bento-modular.png" src="/uploads/ama_bento_modular_0419a45f47.png" srcset="/uploads/xsmall_ama_bento_modular_0419a45f47.png 64w, /uploads/thumbnail_ama_bento_modular_0419a45f47.png 245w, /uploads/small_ama_bento_modular_0419a45f47.png 500w, /uploads/medium_ama_bento_modular_0419a45f47.png 750w, /uploads/large_ama_bento_modular_0419a45f47.png 1000w, /uploads/xlarge_ama_bento_modular_0419a45f47.png 1920w" sizes="100vw" width="1920"></figure><h2>Looking ahead</h2><p>Inference is shifting from an operational detail to a strategic capability. Teams that can optimize across the full stack and deploy across hardware and clouds can push performance limits for their workloads, deploy where it makes business sense, and adapt their infrastructure as needs evolve.</p><p>Joining Modular allows us to push further in that direction, alongside the customers and community building with us.</p><p>If you want to learn more:</p><ul><li>Check out what Modular is building at <a target="_blank" rel="noopener noreferrer" href="http://modular.com/">modular.com</a>.</li><li>We are <a target="_blank" rel="noopener noreferrer" href="https://www.modular.com/company/careers">actively hiring</a> for roles building the <a target="_blank" rel="noopener noreferrer" href="http://modular.com/deployment">Modular Cloud</a> and more.</li><li>Interested in seeing the Modular Stack in action for your workload? <a target="_blank" rel="noopener noreferrer" href="http://modular.com/deployment">Let’s chat!</a></li></ul><p>Thank you for trusting BentoML and helping shape what we build. We’re grateful to our customers, open-source community, investors, and team for making this next chapter possible.</p>

BentoML is joining Modular to build the next generation of AI inference infrastructure.

large_Modular-Bento.png

medium_Modular-Bento.png

small_Modular-Bento.png

thumbnail_Modular-Bento.png

xlarge_Modular-Bento.png

xsmall_Modular-Bento.png

Modular-Bento.png

BentoML Is Joining Modular

📢 [BentoML joins Modular](https://www.bentoml.com/blog/bentoml-is-joining-modular) to build a complete stack for production AI inference.

Bento Blog