Technical information is carried by tables, figures, formulas and layout as well as sentences. Visual document understanding brings these elements together to prepare structured content for enterprise search and AI applications.
Keep technical information connected to its context
Design documents, specifications and test reports often express important conditions through table headers, captions and annotations. Plain-text extraction can lose those connections, making a retrieved passage difficult to interpret. Multimodal document analysis complements text extraction by considering visual structure alongside the text.
The approach
Vision-language models analyze images and text together. Research such as LayoutLMv3 and Donut explores different ways to process document content and layout. These are research references, rather than a claim that every model or method is used in a particular deployment.
- Analyze layout: Identify headings, text regions, tables, figures and their relationships.
- Organize content: Reconstruct reading order and document hierarchy, including content continuing across pages.
- Extract elements: Structure table cells, connect figures to surrounding text and extract formulas, including LaTeX output where appropriate.
- Prepare downstream output: Retain available source references and organize content for retrieval and knowledge applications.
Business value
- More useful retrieval
Keep values with their conditions and figures with their explanations so users can interpret retrieved content.
- Support source checks
Source references help users compare extracted information with the original document.
- Reuse technical material
Prepare both digital documents and scans for downstream search, analysis and knowledge reuse.
Application considerations
- Assess extraction on representative documents, especially scans, drawings and domain-specific notation. Source quality and layout complexity affect the result.
- For consequential uses, validate critical values and relationships against source material. The output structure and source-reference detail should match the target application.
Related products
- AIKNOW Capture
Combines VLMs, OCR and document processing to extract and structure technical content.
- AIKNOW
Connects structured document content with enterprise knowledge retrieval and applications.
References
- Huang, Y. et al. (2022). LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking. arXiv:2204.08387. https://arxiv.org/abs/2204.08387
- Kim, G. et al. (2022). OCR-free Document Understanding Transformer (Donut). ECCV 2022. arXiv:2111.15664. https://arxiv.org/abs/2111.15664