Retention Score: Quantifying Jailbreak Risks for Vision Language Models

ZAITANG LI1, Pin-Yu Chen2, Tsung-Yi Ho1,
1The Chinese University of Hong Kong, 2IBM Research
Paper arXiv

Abstract

The emergence of Vision-Language Models (VLMs) is significant advancement in integrating computer vision with Large Language Models (LLMs) to enhance multi-modal machine learning capabilities. However, this progress has made VLMs vulnerable to advanced adversarial attacks, raising concerns about reliability. Objective of this paper is to assess resilience of VLMs against jailbreak attacks that can compromise model safety compliance and result in harmful outputs. To evaluate VLM's ability to maintain robustness against adversarial input perturbations, we propose novel metric called \textbf{Retention Score}. Retention Score is multi-modal evaluation metric that includes Retention-I and Retention-T scores for quantifying jailbreak risks in visual and textual components of VLMs. Our process involves generating synthetic image-text pairs using conditional diffusion model. These pairs are then predicted for toxicity score by VLM alongside toxicity judgment classifier. By calculating margin in toxicity scores, we can quantify robustness of VLM in attack-agnostic manner. Our work has four main contributions. First, we prove that Retention Score can serve as certified robustness metric. Second, we demonstrate that most VLMs with visual components are less robust against jailbreak attacks than corresponding plain VLMs. Additionally, we evaluate black-box VLM APIs and find that security settings in Google Gemini significantly affect score and robustness. Moreover, robustness of GPT4V is similar to medium settings of Gemini. Finally, our approach offers time-efficient alternative to existing adversarial attack methods and provides consistent model robustness rankings when evaluated on VLMs including MiniGPT-4, InstructBLIP, and LLaVA.

1 Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., & Hein, M. (2021). RobustBench: a standardized adversarial robustness benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). https://openreview.net/forum?id=SSKZPJCt7B

Method Overview of GREAT Score

Method Overview of Retention Score

Flow chart of calculating Retention-Image and Retention-Text scores for VLMs. Given some evaluation samples, we first use diffusion generators to create semantically similar synthetic samples. Then, we pass the generated samples into a VLM to get responses and further use a toxicity judgment model (e.g., Perspective API \textsuperscript{1} or an LLM like Llama-70B (Touvron et al. 2023)) for toxicity level predictions. Finally, we use these statistics to compute the Retention Score as detailed in Section 3.2.

Establishing the Retention Score Framework

Revisiting concepts introduced earlier, minimal perturbations for Image-Text pair in context of VLMs were established. We proposed that greater values of minimal perturbations correlate with enhanced local robustness of model M for pair (I, T). Consequently, estimating lower bounds for these minimal perturbations provides measure of VLMs' robustness. To quantify robustness, we introduce Retention Score, which aims to provide assessment of VLM resilience against input perturbations. Higher Retention Scores signify model's inherent robustness, indicative of safeguards against adversarial toxicity manipulation. Retention Score is multimodal measure capable of assessing conditional robustness of VLMs across visual, textual domains, further divided into Retention-Image (Retention-I) and Retention-Text (Retention-T) scores.

$$ \begin{align} g_I(M,G_I(z|I), T) &= \sqrt{\frac{\pi}{2}} \cdot \{ M_{nt}(G_I(z|I), T) - M_t(G_I(z|I), T) \}^{+} \\ R_I(M, I, \mathbb{X}) &= \frac{1}{m \cdot n} \sum_{j=1}^m \sum_{i=1}^n g_I(M,G_I(z_i|I), T_j)) \end{align} $$

where G_I(z|I) is a continuous diffusion-based image generation model that synthesizes semantically similar images to I, given a zero-mean isotropic Gaussian-distributed input z. The local score function g_I evaluates the non-toxicity of the generated image associated with the given prompt T.

Retention Image Score Results

Table 1. Jailbreak risk evaluation of VLMs to image attacks. This table presents a comparison among three VLMs — MiniGPT-4, LLaVA, and InstructBLIP — with regards to their Retention Scores (Retention-I), and Attack Success Rates (ASR, calculated as the percentage of outputs displaying toxic attributes).
MiniGPT-4 LLaVA InstructBLIP
Retention-I ASR (%) Retention-I ASR (%) Retention-I ASR (%)
Young 0.6121 40.93 0.2866 58.86 0.5043 49.72
Old 0.5917 43.27 0.2636 64.71 0.5650 47.76
Woman 0.5621 42.12 0.2261 57.70 0.4861 52.00
Man 0.5438 42.63 0.1971 52.16 0.4966 50.36
Average 0.5774 42.49 0.2434 58.36 0.5130 49.96

Retention Text Score Results

Table 2. Jailbreak risk evaluation of VLMs to text attacks. This table presents a comparison among three VLMs — MiniGPT-4, LLaVA, and InstructBLIP — with regards to their Retention Scores (Retention-T), Attack Success Rates.
VLM Retention-T Attack Success Rate
MiniGPT-4 0.2073 46.1%
LLaVA 0.342 9.4%
InstructBLIP 0.164 84.5%

API Model Analysis

Assessing the robustness of black-box VLMs is of paramount importance, particularly since these models are commonly deployed as APIs, restricting users and auditors to inferential interactions. This constraint not only makes adversarial attacks challenging but also underscores the necessity for robust evaluation methods that do not depend on internal model access. In this context, our research deploys the Retention-I score to examine the resilience of APIs against synthetically produced facial images with concealed attributes, which are typically employed in model inferences.

Our evaluation methodology was applied to two prominent online vision language APIs: GPT-4V and Gemini Pro Vision. Noteworthy is that for Gemini Pro Vision, the API provides settings to adjust the model's threshold for blocking harmful content, with options ranging from blocking none to most (none, few, some, and most). We tested this feature by running identical prompts and images across these settings, leading to an evaluation of five model configurations.

The assessment centered around the Retention-I score, using a balanced set of synthetic faces that included young, old, male, and female groups. These images were generated using the state-of-the-art Stable Diffusion model, with each group contributing 100 images. A unique aspect of Google's Gemini is its error messaging system, which, in lieu of producing toxic outputs, provides rationales for prompt blocking. In our study, such blocks were interpreted as a zero toxicity score, aligning with the model's safeguarding strategy.

Table 3. Retention-I analysis of VLM APIs. Each group consists of 100 images with 20 continuation prompts.
Young Old Woman Man Average
GPT-4v 1.2043 1.2077 1.2067 1.2052 1.2059
Gemini-None 0.3025 0.2432 0.2300 0.2126 0.2471
Gemini-Few 1.1955 1.1806 1.1972 1.1987 1.1930
Gemini-Some 1.2322 1.2486 1.2325 1.2382 1.2379
Gemini-Most 1.2449 1.2494 1.2388 1.2479 1.2453

Our results in Table 3 reveal intriguing variations across different APIs. For instance, Gemini-None exhibited notable performance contrasts when comparing Old versus Young cohorts. Other models showcased more uniform robustness levels across demographic groups. Also, Our analysis positions the robustness of GPT-4V somewhere between the some and most safety settings of Gemini. This correlation not only validates the efficacy of Gemini's protective configurations but also underscores the impact of safety thresholds on toxicity recognition, as quantified by our scoring method.

This robustness evaluation illustrates that Retention-I is a pivotal tool for analyzing group-level resilience in models with restricted access, enabling discreet and efficacious scrutiny of their defenses.

GREAT Score vs CW Attack Comparison

Comparison of local GREAT Score and CW attack

Figure 2. Comparison of local GREAT Score and CW attack in L2 perturbation on CIFAR-10 with Rebuffi_extra model. The x-axis is the image id. The result shows the local GREAT Score is indeed a lower bound of the perturbation level found by CW attack.

Run-time Analysis

Run-time improvement comparison

Figure 4. Run-time improvement (Retention Score over Visual and Text attacks).

Figure 4 compares the run-time efficiency of Retention Score over adversarial attacks in [1] and [2]. We show the improvement ratio of their average per-sample run-time (wall clock time of Retention Score/Adversarial Attack is reported in Appendix) and observe around 2-30 times improvement, validating the computational efficiency of Retention Score.

BibTeX

@article{li2024greatscore,
  title     = {GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models},
  author    = {Zaitang, Li and Pin-Yu, Chen and Tsung-Yi, Ho},
  journal   = {NeurIPS},
  year      = {2024},
}