Is BAGEL The Future of Multimodal Understanding and Generation?

Developing AI systems that can seamlessly understand and generate content across various modalities — such as text, images, and video — with reasoning capabilities approaching human cognition has been a central goal in the field.

While proprietary models have long showcased this integrated intelligence, their underlying mechanisms remain private. BAGELhttps://bagel-ai.org (Scalable Generative Cognitive Model), an innovative open-source foundational model released in late May of 2025, is now stepping into this crucial space. As it scales, the model uncovers emergent multimodal abilities that push the field forward. With a robust architecture boasting 7 billion active parameters (and 14 billion in total), BAGEL performs better than open-source unified models across multimodal generation and understanding benchmarks. Its training on trillions of tokens derived from extensive, interleaved text, image, video, and web data has enabled advanced multimodal reasoning, including capabilities like free-form image manipulation, predicting future frames in a sequence, 3D manipulation, and world navigation.

So of course, we had to take it for a spin.

Here's what we'll cover:

• The core architecture behind BAGEL, including its Mixture-of-Transformer experts and dual vision modules for understanding and generating images

• How a multi-stage pipeline (alignment, large-scale pretraining, continued training, and supervised fine-tuning) shapes multimodal reasoning

• Key benchmark results that show where BAGEL excels against other open-source and some proprietary models

• Practical takeaways from hands-on testing, highlighting both impressive image outputs and current demo limitations

• How to tap into BAGEL’s open-source checkpoints, code, and public demo for your own experiments

Let's begin!

Architecture: 4 key components#

BAGEL follows the design principle of maximizing the model’s capabilities without introducing artificial limits. It uses a Mixture of Transformer experts (MoT) architecture, which means it has specialized parts for different tasks, but they all work together closely. Unlike earlier methods that often created choke points, BAGEL’s design allows information to flow freely, enabling extensive interaction between understanding and generation processes. This open design facilitates efficient scaling of training and data, allowing the model’s full potential to develop without being held back by its structure.

1. The unified backbone#

BAGEL’s foundation is built upon a powerful, language-focused Qwen2.5 LLM. This model is designed to process information effectively, incorporating advanced techniques to ensure stability and efficiency.

2. How BAGEL sees and creates visuals#

Visual information is processed in two distinct ways to support both understanding and generation tasks:

• For visual understanding: BAGEL employs a ViT encoder. This component acts as an advanced image reader, converting raw image pixels into digital tokens that the model can interpret. It can flexibly handle images at their native aspect ratios up to 980x980. A two-layer MLP connector helps these visual tokens align with the main language model’s internal data format.

• For visual generation: BAGEL utilizes a pretrained VAE model from FLUX. This VAE converts images between pixel space and a compressed latent space. This latent representation is then further processed to match the hidden dimension of the main language model. Importantly, the VAE model’s parameters remain fixed during BAGEL’s training, providing a stable tool for visual creation.

3. Interleaving modalities with attention#

All types of tokens, including text, understanding image tokens (ViT), and generation image tokens (VAE), are combined and interleaved according to the input’s modality structure. Before being integrated, both ViT and VAE tokens receive 2D positional encoding. For diffusion-based generation, a timestep embedding is directly added to the initial states of VAE tokens for a cleaner architecture. BAGEL employs the rectified flow method to generate images from visual tokens, aligning with leading techniques in visual generation.

4. Fueling BAGEL’s intelligence#

BAGEL’s advanced capabilities are built on a meticulously structured, multi-stage training process, leveraging an exceptionally rich and diverse dataset. This includes trillions of tokens from interleaved text, image, video, and web data, carefully filtered and augmented for complex multimodal reasoning.

BAGEL’s training prioritizes sampling generation examples more often than understanding examples. Its data corpus spans text, image-text pairs, and crucial interleaved data from videos and web, which are specially prepared to support complex in-context reasoning, world modeling, and even future frame prediction.

Performance: how did it fare?#

BAGEL’s performance across various benchmarks demonstrates its significant capabilities. It often surpasses specialized and other unified open-source models and even competes with some proprietary systems.

1. Multimodal understanding performance#

BAGEL shows strong performance in understanding tasks across diverse public benchmarks, outperforming existing unified and often specialized understanding models. These benchmarks (MME-P, MMBench, MMMU, MM-Vet) comprehensively evaluate a model’s abilities in multimodal understanding, ranging from basic perception and all-around performance to expert-level reasoning across various disciplines, and integrated capability verification.

2. Image generation performance#

BAGEL delivers competitive and often superior results in text-to-image generation, surpassing both specialized image generation models and other unified approaches. The benchmarks (single object, two object, counting, colors, position, color attribute, overall) are common metrics used in image generation benchmarks like GenEval. They assess a model’s ability to accurately generate images with specific object counts, colors, positions, and attributes based on textual prompts, along with an overall quality score.

Accessibility: a step in the right direction#

BAGEL’s open-source nature is an important contribution, aiming to democratize advanced AI capabilities by making its foundational model publicly available. This commitment includes sharing its code and releasing its trained checkpoints, which allow developers, researchers, and tech professionals globally to inspect, utilize, and build upon a sophisticated multimodal model without proprietary barriers. This accessibility enables users to reproduce results, fine-tune for specific applications, and innovate in new directions, fostering a vibrant ecosystem in multimodal AI.

A public project page with a demo is available, allowing direct interaction with the model to test its understanding and generation abilities firsthand, proving invaluable for quick evaluation, inspiration, and learning across the community. Here is what the demo looks like.

But how good is it really?#

BAGEL is advertised as being meticulously pretrained on extensive, interleaved video and web data, which equips it with the ability to produce high-fidelity, photorealistic images, dynamic video frames, or complex interleaved image-text content. With such impressive claims, it's time to put BAGEL’s capabilities to the test and examine its real-world performance.

Observation: The image captures the prompt’s mystical tone and color palette, with glowing crystals on a velvet-lined shelf. However, it partially fails in label accuracy — “LYNX” is misspelled as “LYYXX” and “NOVA” is missing or unclear.

This is how the authors demonstrate the model’s capabilities in the paper.

The bottom line#

While BAGEL is a new and powerful model, considerable room remains for improvement. The demo did not consistently function during testing, and the text descriptions exhibited inaccuracies or lacked clarity. Despite these challenges, the commitment to open-source development is a major positive, offering great potential for collaborative enhancement and wider accessibility within the AI community.

Curious to learn more about image generation models? You can start by exploring these exciting courses: