VisionAgent: An Agentic Approach for Complex Visual Reasoning - LandingAI

Current vision-language models still struggle with visual reasoning, as shown by their inconsistent answers to a soda-can counting puzzle.
VisionAgent is proposed as an agentic AI framework for solving complex visual tasks by breaking them into smaller subtasks.
The system uses agentic tool selection to pick specialized visual tools for each subtask, such as object detection.
Outputs from earlier subtasks are fed into later steps, enabling task composition from detections to higher-level reasoning.
VisionAgent uses visual design patterns, including arranging detected items in a grid to reason about missing objects.
An event stream data structure stores user and agent messages, dispatches tool actions, and records observations.
The planner agent can run Python code in a notebook-like environment to support reasoning and computation.
A separate tool-choice agent and judge agent compare candidate tools and select the most plausible result.

Your notes

Save this item to your library to add private notes.