AI21 Labs

A long context window, without a long *effective* context, is effectively a small context window. According to analysis from NVIDIA 'leaderboard models claim to offer long context windows with lightning latency, but upon thorough testing, their 'effective' context lengths, where the models still produce good results, are half of what’s advertised or less' —50% or less. And yet, "Jamba-Instruct offers an inference context window of 256,000 tokens and can actually use all those 256,000 tokens effectively," as noted by Or Dagan. To read more about how LLM architecture is paving the way towards Agentic AI, check out this article by Daniel Singer. #Agents #ContextWindow #Agentic #Jamba

Everyone's talking about AI as if it's the latest trend, but in reality, it has been around since 1950. For context: the *color television*, the *transistor radio*, the *nonstick pan*, and *nondairy creamer* were also invented in the 50's. It's certainly has come a long way. With a long way to go. AI doesn't start and end with LLMs. LLMs don't start and end with Transformers. However, today most LLMs are built upon them. We built Jamba, a groundbreaking Mamba-Transformer hybrid model, but we've got more coming down pipeline that just might make today's tech seem as outdated as inventions from the 1950s. Check out this timeline to get a glimpse at how far we've come with genAI. Double-tap to zoom in!

Deciding whether to build your own Generative AI solution or buy an off-the-shelf option can be tricky. Yesterday we covered the 7 things to consider when investing in AI. Now, let's spice things up with the *Buy vs. Build* Game! Find your enterprise use case//business goal card, lock in your guess (no cheating!), then swipe to see the Buy vs. Build answer. Want a more thorough guide? Check out our free guide that outlines a step-by-step process used by leading enterprises to implement AI successfully: >>>> https://lnkd.in/d4DAgQ4E

No matter where you are in your AI journey, it’s crucial to be aware of some critical blind spots. Enterprise AI a long-term game, it’s also an evolving field. The checklist below provides a starting point, but for a truly informed decision, you need a comprehensive framework. Download our free guide to discover a step-by-step process to navigate your AI investment journey >>>> https://lnkd.in/d_RFkg_5

🎉 Exciting news - Jamba is now officially supported by vLLM! Due to its novel hybrid SSM-Transformer architecture, Jamba didn't work out-of-the-box with #vLLM, the leading open-source LLM serving framework. Mor Zusman 曹木 and Tomer Asida successfully integrated #Jamba into the framework in an elegant, creative, and clean manner, with improvements that could benefit other future hybrid SSM-Transformer models as well. Some of the features this PR brings to vLLM: 1. A Jamba modeling file to support different variations of the Jamba architecture. 2. Mamba cache management that could benefit other Mamba-based models. 3. Sending more request-related properties to the model forward pass, allowing for greater control over model-specific implementations, such as finished requests tracking needed for speculative decoding. Collaborating with Cade Daniel, Zhuohan Li, and the rest of the vLLM team to get this PR approved last week was a pleasure. Thank you for your support and guidance, and for building this great framework! 👨🔬 Bonus: Since Jamba, with its 256K context window, is the longest-context MoE-based #open-source model, we stumbled upon an issue with MoE models in vLLM that occurs with VERY LONG contexts. It seems like we were not the only ones experiencing this problem. This hard-to-reproduce bug was tricky and sometimes led to runtime errors, and other times, worse—unexplained bad or mixed-up outputs. Avshalom Manevich found an elegant workaround, utilizing the fact that there is no interaction between different tokens in the #MoE block, allowing us to iterate over the long context in chunks. This fix was also merged and is now part of vLLM. 😎 https://lnkd.in/d4irK3BG

Yesterday, we hosted 'Moving LLMs to Production: The Data Challenge' with Pitango and Preql. The event was sold out, and it’s easy to see why, with an outstanding lineup of speakers: 🎤 Dan Padnos, VP of Alignment, and one of the masterminds behind #Jamba 🎤 Eyal Niv, Managing General Partner at Pitango Venture Capital 🎤 Gabi Steele , CEO & Co-Founder of Preql 🎤 Assaf Elovic, Head of R&D at Wix These industry leaders tackled how to maximize transformative impact on the data stack while maneuvering real issues surrounding implementation. A big thank you to everyone who attended and contributed to making this event a tremendous success! 🚀 #DataScience #LLMs #Networking #TechCommunity

So, what does a 256k context window actually look like? (🔉sound on!)

It was a pleasure to present alongside some of my personal favorite people in the #GenAI and #Partneships space at the Microsoft All Founders event Kudos Nadav Tzuker, Meital Shamia, Efrat Garber-Aran, Yair Adato & Uri Lampert for the great sessions delivered💪 And thanks Raz Bachar & Yarden Dor 🇮🇱 for putting such an insightful event AI21 Labs #GenAI #LLMs #founders

FBI-AGI, running on Jamba, gathered incriminating data of many predators harassing children online. Hackers Alex Sima and Apurva Mishra built FBI-AGI at the recent AGI House Jamba Hackathon. Within minutes of launching Emma’s profile, a fictitious 14-year-old girl, the AI agent received tens of messages from online predators with offensive and inappropriate messages. The #agent then captured the chat data and metadata to identify and incriminate these online predators in real-time. The best part? By leveraging Jamba’s massive 256K #contextwindow, ‘Emma’ ran multiple chats simultaneously, gathering proof of multiple predators at a time. And that’s just one of the many impressive projects made at the Jamba #Hackathon. Check out this blogpost with insights from Yaniv Markovski, our Head of Developer Relations, to read more about the projects built on #Jamba. 👉 https://lnkd.in/djCZFSYc 👇 Enjoy this clip capturing just a taste of the energy at the 12-hour hack

Between the *whitepapers*, *ablations*, *graphs*, and *endless acronyms*, the topic of genAI can seem pretty daunting—especially when considering scaling at the enterprise level. It really doesn’t need to be. Check out this easy-to-digest glimpse showcasing our latest advancements in the field. A warm thank you to BNP Paribas CIB for illustrating this Summary Scribe following Ori Goshen's keynote address at The BNP Paribas Exane CEO Conference. #FSI #NeuroSymbolic #LLM #AISystem