AwaRes — Look Where It Matters

Abstract

Vision-language models (VLMs) typically process images at native high resolution, forcing a trade-off between accuracy and computational efficiency: high-resolution inputs capture fine details but incur significant computational costs, while low-resolution inputs promote efficiency but potentially miss critical visual information like small text.

We present AwaRes, a spatial-on-demand framework that resolves this accuracy–efficiency trade-off by operating on a low-resolution global view and using tool-calling to retrieve only the high-resolution segments needed for a given query. We construct supervised data automatically: a judge compares low- vs. high-resolution answers to label whether cropping is needed, and an oracle grounding model localizes the evidence for the correct answer, which we map to a discrete crop set to form multi-turn tool-use trajectories. We train our framework with cold-start SFT followed by multi-turn GRPO with a composite reward combining semantic answer correctness with explicit crop-cost penalties.

AwaRes provides a practical, deployment-friendly path to high-detail VLM reasoning under tight compute budgets.

Overview

AwaRes processes a low-resolution global view by default. When additional detail is required, it invokes a structured tool-call to request only specific high-resolution sub-regions — then answers conditioned on both views.

Figure 1: Left: Given a low-resolution image, AwaRes uses tool-calling to request only the needed crops. Right: Accuracy vs. retained visual tokens — AwaRes achieves Qwen2.5-VL accuracy (80.3%) using only 36% of the visual tokens.

Method

AwaRes learns a coupled-decision policy (CDP) that jointly decides whether additional resolution is needed and where to acquire it, by selecting a subset of crops from a predefined candidate set.

Resolution Sufficiency Labeling

An LLM judge compares low-res vs. high-res outputs against ground truth to determine when cropping is needed.

Crop Target Construction

An oracle grounding model localizes the visual evidence, producing bounding boxes mapped to a discrete crop set via IoU.

Cold-Start SFT

Supervised fine-tuning on multi-turn tool-use trajectories teaches the model the tool protocol and yields a reference policy.

Multi-Turn GRPO

RL with a composite reward balancing correctness against crop-cost penalties refines tool usage for efficiency.

Data Curation Pipeline

Figure 2: Each sample is processed at two resolutions. An LLM judge determines sufficiency; insufficient cases are routed to an oracle for crop localization, producing multi-turn tool-use trajectories.

Crop Annotation Examples

The oracle grounding model localizes answer regions, which are mapped to discrete crops for training.

Example 1: "What website is being advertised on the plane?" — The oracle localizes the text region; the selected crop reveals "Jetstar.com".

Example 2: "What is the Grand Total for Net Block As of 31.3.2012?" — The oracle targets the relevant table cell; the crop enables reading "52708.86".

Main Results

AwaRes matches full-resolution performance while using only 36% of visual tokens, outperforming all efficient baselines across six benchmarks.

Model	Chart	RTR	Doc	RTR	OCR	RTR	POPE	RTR	Real	RTR	V*	RTR	Avg	RTR
Qwen2.5-VL-7B	79.80	1.00	94.00	1.00	81.10	1.00	87.87	1.00	68.80	1.00	71.20	1.00	80.46	1.00
Qwen2.5-VL-7B-LR	65.00	0.25	91.00	0.25	70.70	0.25	84.41	0.25	66.00	0.25	63.20	0.25	73.39	0.25
Holo-V (70%)	69.32	0.70	76.40	0.70	72.40	0.70	87.37	0.70	68.89	0.70	69.11	0.70	73.92	0.70
SparseVLM (70%)	75.80	0.70	87.20	0.70	79.30	0.70	85.40	0.70	68.50	0.70	54.45	0.70	75.11	0.70
VisionZIP (70%)	76.72	0.70	90.75	0.70	72.70	0.70	87.86	0.70	64.84	0.70	65.97	0.70	76.47	0.70
VisionThink	79.90	1.15	90.35	0.32	80.10	0.83	86.70	0.34	66.60	0.55	71.73	0.49	79.23	0.61
AwaRes (Ours)	80.64	0.32	94.43	0.28	81.30	0.42	85.73	0.27	68.50	0.43	71.20	0.42	80.30	0.36

Table 1: RTR = fraction of visual tokens retained (lower is better). AwaRes matches full-resolution at 36% compute.

Performance vs. Wall Clock Time

Figure 3: AwaRes achieves sub-second latency across benchmarks via short tool calls, while VisionThink's reasoning traces increase decoding time (4.3s vs. 0.6s on ChartQA).

From Over-Calling to Looking Where It Matters

Figure 4: Crop selection evolution: Oracle GT → SFT → GRPO. SFT over-uses the crop tool (LR drops 80.8%→46.3%); GRPO corrects this via tool-use cost, recovering to 72.2% low-res decisions.

Visual Results

Examples of AwaRes adaptively selecting crops across charts, documents, and natural images. Each shows the question, tool call, low-res input, and retrieved high-res crop.

Chart: Zooming into center trend-line data points to compute the median value (14.5) of the red graph.

Chart: Cropping the bottom to compare median values between China and Angola trend lines.

Natural image: Zooming into the bottom of a wine bottle to read the label — "Amarone".

Natural scene: Cropping the right side of a garden to identify the black color of a broom handle.

Scene understanding: Cropping the bottom of a rainy street to count 2 pedestrians within 20 meters.

Object counting: Cropping the bottom of a parking lot to identify and count 2 traffic cones.

Citation

If you find AwaRes useful in your research, please consider citing:

@misc{shabtay2026lookmattershighresolutioncrops,
      title={Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs}, 
      author={Nimrod Shabtay and Moshe Kimhi and Artem Spector and Sivan Haray and Ehud Rivlin 
      and Chaim Baskin and Raja Giryes and Eli Schwartz},
      year={2026},
      eprint={2603.16932},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2603.16932},
}