Download
FastVLM is an efficiency-focused vision-language modeling stack that introduces FastViTHD, a hybrid vision encoder engineered to emit fewer visual tokens and slash encoding time, especially for high-resolution images. Instead of elaborate pruning stages, the design trades off resolution and token count through input scaling, simplifying the pipeline while maintaining strong accuracy. Reported results highlight dramatic speedups in time-to-first-token and competitive quality versus contemporary open VLMs, including comparisons across small and larger variants. The repository documents model variants, showcases head-to-head numbers against known baselines, and explains how the encoder integrates with common LLM backbones. Apple’s research brief frames FastVLM as targeting real-time or latency-sensitive scenarios, where lowering visual token pressure is critical to interactive UX. In short, it’s a practical recipe to make VLMs fast without exotic token-selection heuristics.
Features
- FastViTHD hybrid vision encoder with fewer visual tokens
- Significant reductions in encoding latency and TTFT
- Resolution–token trade-off via simple input scaling
- Compatibility with standard LLM backbones in VLM stacks
- Reported outperforming baselines at much lower cost
- Variants tuned for both small and larger model regimes
Project Samples
