Privately owned AI21 Labs, headquartered in the US, was founded in 2017 and employs approximately 230 individuals. The company specializes in software development, providing access to language models that power text generation and comprehension features.
AI21 Labs eliminated approximately 90% of padding-related overhead when training large language models (LLMs) on February 11, applying a model-agnostic approach that utilizes micro-batch-level truncation and padding-aware micro-batching. This method addresses a significant source of wasted compute, particularly for hybrid Transformer-SSM models where traditional sequence packing is less applicable. Previously, on February 5, the company mitigated GPU underutilization and vulnerability to buckling under load in its LLM-as-a-Judge deployments. This involved a two-pronged strategy focused on optimizing single-node performance and scaling multi-node deployment, creating a method applicable to high-throughput inference deployments facing variable loads. Earlier, on January 29, AI21 Labs debugged a rare issue in its Jamba model where it would generate gibberish once out of every thousand prompts, tracing the bug to vLLM's scheduler interaction with different model architectures. Also on January 29, the company demonstrated a simple multi-scale approach to RAG retrieval that improved performance by 1-37% across benchmarks without retraining models. This was achieved by indexing the same corpus at multiple chunk sizes and aggregating results with Reciprocal Rank Fusion. In late January, specifically on January 22, AI21 Labs introduced "dynamic difficulty snoozing," a strategy designed to cut compute waste without breaking training stability when running GRPO on verifiable rewards in online Reinforcement Learning.
Subscribe for full access to AI21 Labs's products in full detail 
