Baidu's 'Unlimited OCR' Processes Dozens of Document Pages in One Pass Using Human-Like Forgetting

Baidu researchers have developed an optical character recognition (OCR) model capable of processing dozens of document pages in a single inference pass while maintaining constant memory usage and consistent speed. The system, called Unlimited OCR, achieves this through a novel attention mechanism inspired by how humans copy text by hand, representing a significant advance in document AI. For the latest developments in AI research and technology, visit AI Buzz Wire.

Overcoming the KV Cache Bottleneck

Current end-to-end OCR systems that use language models as their decoders face a fundamental architectural limitation. As a model processes text, it stores information about previously processed tokens in a buffer called the KV cache, allowing it to reference earlier content during generation. This buffer grows with every new line of text, steadily increasing memory consumption and slowing generation speed.

In practice, existing systems work around this bottleneck by processing documents page by page in a loop, resetting the cache after each page is completed. This approach works but introduces inefficiencies and prevents the model from maintaining context across page boundaries. No current OCR model handles more than approximately ten pages in a single pass, the Baidu researchers note in their technical report.

Unlimited OCR eliminates this constraint entirely. By redesigning the attention mechanism that governs how the model accesses its cache, the system keeps memory use constant regardless of how many pages it processes.

How Reference Sliding Window Attention Works

The key innovation is what the Baidu team calls Reference Sliding Window Attention (R-SWA). The mechanism is built on a simple but powerful insight drawn from a human analogy: when a person copies text from a book, they do not continuously re-read everything they have already written. Instead, they keep their attention focused on the source material, the last few characters they transcribed, and the next character to write down. Older passages naturally fade from active memory through a kind of soft forgetting.

R-SWA implements this principle by treating different types of tokens differently. Each generated token retains full attention to all reference tokens — the visual image tokens that encode the document and the processing prompt. But when it comes to previously generated output text, the model only looks back at the most recent 128 tokens.

This design keeps the KV cache at a fixed size equal to the sum of the prefix length and the window size, regardless of how much text has been generated. The cache functions as a queue, with each new token pushing out the oldest one, preventing the unbounded memory growth that plagues standard attention mechanisms.

Protecting Visual Information

A critical design choice in R-SWA is how it handles visual tokens. Standard sliding window attention would subject all tokens to ongoing state changes, gradually blurring image features and degrading recognition accuracy over time. R-SWA exempts visual tokens from these transitions — they are encoded once and remain unchanged throughout the entire decoding process.

This preservation of visual information is essential for OCR accuracy. By keeping the original image representation intact while limiting how far back the model looks in its own output, R-SWA achieves the best of both worlds: stable memory usage and reliable text recognition.

Architecture and Training

Unlimited OCR is built on top of the open-source Deepseek OCR model. Baidu retains its DeepEncoder, which compresses a 1024-by-1024-pixel PDF image down to 256 tokens, and pairs it with a mixture-of-experts (MoE) architecture. The decoder has three billion total parameters, but only approximately 500 million are active during inference, keeping computational costs manageable.

The system offers two resolution modes. Base mode is optimized for multi-page documents, while Gundam mode uses dynamic resolution for single-page processing. Every standard attention layer in the decoder was replaced with R-SWA.

Training was conducted on approximately two million document samples, split nine-to-one between single-page and multi-page data. PaddleOCR handled annotation for single pages, while multi-page data was constructed synthetically by stitching single pages together into documents ranging from two to fifty pages. All training data was packed into sequences of 32,000 tokens, and training ran for 4,000 steps on 8 sets of 16 Nvidia A800 GPUs. The DeepEncoder remained frozen throughout training, with only the language model parameters being updated.

Benchmark Results

Unlimited OCR achieves strong performance across multiple evaluation metrics. On the OmniDocBench v1.5 document benchmark, the model scores 93 percent overall, six percentage points above the Deepseek OCR baseline.

In the critical long-horizon test — where the model processes many pages in a single pass — the error rate stays below 0.11 even beyond 40 pages. The researchers attribute remaining errors not to lost context but to the DeepEncoder's resolution limit in Base mode, where extremely small text becomes difficult to recognize.

The restricted attention window also yields an unexpected benefit. On single-page documents, limiting the window to 128 tokens does not hurt accuracy and actually improves it slightly. The researchers hypothesize that R-SWA forces the model to focus more tightly on the dense OCR task, while full attention tends toward divergence as output length grows.

Speed improvements are equally notable. In Base mode, Unlimited OCR achieves 5,580 tokens per second compared to 4,951 for Deepseek OCR, a 12.7 percent improvement. In theoretical upper-bound comparisons with ideal parallelism, the model leads the baseline by 35 percent at approximately 6,000 output tokens.

Broader Significance

The Unlimited OCR architecture demonstrates a principle with implications beyond document processing. The idea of keeping memory constant through selective attention — inspired by cognitive science rather than pure scaling — could inform the design of more efficient AI systems across a range of applications.

As organizations grapple with the computational costs of deploying large language models, approaches that reduce memory consumption without sacrificing quality are increasingly valuable. Baidu's work suggests that biological metaphors, when translated into mathematical mechanisms, can yield practical engineering breakthroughs.

The research also highlights China's growing influence in foundational AI research. While much attention has focused on Chinese achievements in large language models, work like Unlimited OCR demonstrates that Chinese researchers are also making significant contributions to specialized AI systems with immediate practical applications.

Stay Ahead of AI

For more coverage of AI research, document AI, and technology breakthroughs, visit AI Buzz Wire.

Read more AI news →