Modal

RSVP to Modal x Mistral's SF demo night 🍏🍊 We're hosting a demo night & happy hour with Mistral AI on Thursday, March 6th. Our goal is to bring together attendees who can learn from one another—whether you're just starting to explore open-source LLMs or have already put them in production, we'd love to see you there. There will be plenty of food, drinks, and swag!

Cloud commitment issues this Valentine's? We have GPUs for you 💌

Congrats to Mistral AI for launching a new, super-fast version of their AI assistant ⚡ We're excited that their researchers used Modal Sandboxes to speed up the development of Le Chat's code interpreter feature. The code interpreter allows users to execute code directly within the chat interface, such as for scientific analysis or running simulations. Sandboxes are a Modal primitive to safely run LLM-generated code. It comes with both secure isolation and all the usual high-performance, fast-scaling benefits of our platform.

🌎 Region selection for Functions and Sandboxes is now available across all plan tiers! Region selection allows you to define specific regions for your Functions to run in. It's useful for: 🔐 Complying with data residency requirements 💰 Reducing egress costs 🚤 Reducing latency for apps that need to run near an external DB We've gotten many requests to make this feature more broadly available, and we're excited that anyone can use it now! For more information on the feature and how it's priced: https://lnkd.in/enDewYBQ

🌈 Apply to demo at Modal x Mistral AI's SF demo night 🌈 Calling all AI builders! Modal is hosting a demo night in SF with Mistral on Thursday, March 6th. We want to showcase the coolest projects that the community is building on Mistral & Modal, whether they’re personal projects or for work. You'll have an audience of 80+ founders, builders, and investors in the community! If selected, you'll also get $1000 in Modal credits to keep on building 🎖️ Submit your demo proposal here: https://lnkd.in/egv_aHys

New blog post by Jonathon Belotti diving into the technical details of our container memory snapshot restore functionality. Container memory restores can reduce cold start latency on user functions by more than 50% 💥

We have been working a lot to make Modal Functions start really fast at Modal. This is one of my favorite ideas: memory snapshots. https://lnkd.in/eRhTfrcN We take a snapshot after your function boots. We then restore your function's memory when your function starts again. This yields 2.5x faster cold boot times. Neat!

Today on the Modal blog we’re sharing how a warmed up container can be saved to disk and later restored for a 2.5x cold start performance boost. Restoring a Stable Diffusion container and getting the model into the GPU takes only 3.56s at p50! How exactly this works has lots of fun OS stuff: CRIU, FUSE, vDSO, CPU fingerprinting. Saving live container processes to disk turns out to be pretty whacky and interesting! A process is a sophisticated OS resource and wasn’t designed to be savable, but these are computer things and so what isn’t possible if you try?

🐋 How to deploy DeepSeek AI models on Modal: https://lnkd.in/eJWYybWF Check out our end-to-end code sample for running DeepSeek-R1. No need to bring your own infra, and you can try this out well within our $30/mo free credits! This example makes use of Unsloth AI's 1.58 bit quantization of DeepSeek-R1, which fits on four L40S GPUs, and runs it with llama.cpp, a C++ inference library. The full version of this 671B parameter model requires a minimum of 8 H200s (>1 TB of VRAM) to run! More inference-time optimizations to come so stay tuned. And if there’s other flavors of DeepSeek models you want to see examples of, let us know ⬇️

We're saying goodbye to our winter interns Qijia (Joy) Liu, Andrew Liu, and Tarushii Goel this week! ☃️ They worked on some awesome features during their internships like sandbox memory snapshots, proxy auth tokens, and automatic workload profiling. 🚀 And looked great doing it! 💚