Skip to content

OmniSVG: A Unified Scalable Vector Graphics Generation Model

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

OmniSVG adapts a pre-trained Vision-Language Model (Qwen2.5-VL) for end-to-end multimodal SVG generation by parameterizing SVG commands and coordinates as discrete token sequences, enabling autoregressive generation of complex, colored vector graphics from text and image instructions. It introduces MMSVG-2M, a 2-million-asset annotated SVG dataset spanning icons, illustrations, and anime characters, along with MMSVG-Bench for standardized evaluation. OmniSVG outperforms prior art on both text-to-SVG and image-to-SVG tasks while producing far shorter, more structured outputs.

Problem

Existing SVG generation methods are either optimization-based (computationally prohibitive, produce unstructured outputs with redundant anchor points) or autoregressive LLM-based but limited to monochrome icons or simple structures due to short context windows (prior work capped at ~8.2k tokens) and scarce complex training data. No unified model could handle multimodal instructions (text + image) for generating colored, complex SVGs across diverse styles.

Method

OmniSVG fine-tunes Qwen2.5-VL using a custom SVG tokenizer that converts SVG scripts into a flat discrete token sequence within the VLM's vocabulary space. SVGs are simplified to five atomic path commands (M, L, C, A, Z) plus a Fill (F) command for hex color; 2D coordinates are merged into single tokens via the mapping x Ă— w + y to compress sequence length. Multi-modal instruction tokens (text + image) serve as the prefix; SVG token sequences are the suffix, trained with standard next-token prediction (Eq. 1). A learnable embedding layer lifts SVG tokens into the VLM's embedding space. The approach decouples structural logic from geometry, mitigating "coordinate hallucination" and supporting sequences exceeding 30k tokens.

Key Contributions

  • OmniSVG model family: end-to-end multimodal SVG generator (4B and 8B variants) built on Qwen2.5-VL, supporting text-to-SVG, image-to-SVG, and character-reference SVG generation.
  • SVG parameterization scheme: discrete tokenization of commands and coordinates that reduces average sequence length ~4–5Ă— versus non-parameterized baselines while improving all metrics.
  • MMSVG-2M dataset: 2M annotated SVG assets (1.1M icons, 0.5M illustrations, 0.4M anime characters) with BLIP-2-generated captions and atomic-command normalization via picosvg.
  • MMSVG-Bench: standardized evaluation protocol for text-to-SVG (FID, CLIP score, Aesthetic, HPS) and image-to-SVG (DINO, SSIM, LPIPS, MSE) with GPT-4o-generated prompts to avoid train/test overlap.

Results

  • Text-to-SVG (MMSVG-Icon): OmniSVG-4B achieves FID 137.40 vs. 190.87 (Chat2SVG), 213.28 (IconShop), 250.77 (VectorFusion); uses only 3.8k tokens vs. 322k for DiffVG.
  • Text-to-SVG (MMSVG-Illustration): OmniSVG-L (8B) achieves FID 138.42 vs. 107.93 (IconShop best), 210.03 (Chat2SVG); CLIP 0.231 vs. 0.233 (IconShop best).
  • Image-to-SVG (MMSVG-Icon): OmniSVG-4B achieves DINO 0.993, SSIM 0.950—matching VTracer (0.993/0.966)—while generating 3.8k tokens vs. 52.4k for VTracer; outperforms StarVector (DINO 0.895) and GPT-4o (DINO 0.860).
  • Ablation (parameterization): full parameterization (color + coordinate) achieves FID 145.89 and DINO 0.946 vs. 218.76/0.741 without any parameterization, at 4.8k vs. 18.5k tokens.
  • User study (150 samples, 15 participants): OmniSVG scores Preference 96, Vividity 88, Alignment 98—highest across all methods tested.

Limitations

  • Inference generates tens of thousands of tokens for complex samples, leading to high latency.
  • Image-to-SVG is restricted to vector-style image prompts; natural photographs are not supported.
  • Token sequence length still fundamentally constrains complexity for extremely intricate SVGs, despite improvements over prior methods.

Relevance to Vision-Language Models

OmniSVG demonstrates a concrete pathway for repurposing pretrained VLMs (Qwen2.5-VL) as structured code generators beyond natural language, using discrete domain-specific tokenization to bridge modalities. It directly exploits VLMs' multimodal understanding—grounding both textual descriptions and visual references into geometric outputs—which is relevant to researchers studying VLM generalization, token-space design for structured generation, and instruction-following in non-linguistic domains. The work also highlights how the scale and visual reasoning of VLMs can resolve hallucination problems (coordinate drift) that plague pure LLM-based approaches to structured output generation. For VLM researchers, the SVG tokenizer design and the MMSVG-Bench evaluation framework offer transferable methodology for other structured visual generation tasks.