vLLM is vulnerable to DoS in Idefics3 vision models via image payload with ambiguous dimensions

Summary

Users can crash the vLLM engine serving multimodal models that use the Idefics3 vision model implementation by sending a specially crafted 1x1 pixel image. This causes a tensor dimension mismatch that results in an unhandled runtime error, leading to complete server termination.

Details

The vulnerability is triggered when the image processor encounters a 1x1 pixel image with shape (1, 1, 3) in HWC (Height, Width, Channel) format. Due to the ambiguous dimensions, the processor incorrectly assumes the image is in CHW (Channel, Height, Width) format with shape (3, H, W). This misinterpretation causes an incorrect calculation of the number of image patches, resulting in a fatal tensor split operation failure.

Crash location: vllm/model_executor/models/idefics3.py line 672:

def _process_image_input(self, image_input: ImageInputs) -> torch.Tensor | list[torch.Tensor]:
    # ...
    num_patches = image_input["num_patches"]
    return [e.flatten(0, 1) for e in image_features.split(num_patches.tolist())]

The split() call fails because the computed num_patches value (17) does not match the actual tensor dimension (9):

RuntimeError: split_with_sizes expects split_sizes to sum exactly to 9 
(input tensor's size at dimension 0), but got split_sizes=[17]

This unhandled exception terminates the EngineCore process, crashing the server.

Affected Models

Any model using the Idefics3 architecture. The vulnerability was tested with HuggingFaceTB/SmolVLM-Instruct.

Impact

Denial of service by crashing the engine

Mitigation

Validating the input:

def _validate_image_dimensions(self, image_shape):
    h, w = image_shape[:2] if len(image_shape) == 3 else image_shape
    if h < MIN_IMAGE_SIZE or w < MIN_IMAGE_SIZE:
        raise ValueError(f"Image dimensions too small: {h}x{w}")

Managing the exception:

try:
    return [e.flatten(0, 1) for e in image_features.split(num_patches.tolist())]
except RuntimeError as e:
    logger.error(f"Image processing failed: {e}")
    raise InvalidImageError("Failed to process image features") from e

Fixes

• https://github.com/vllm-project/vllm/pull/29881