Simple Iterative Method for Generating Targeted Universal Adversarial Perturbations

Abstract

Deep neural networks (DNNs) are vulnerable to adversarial attacks. In particular, a single perturbation known as the universal adversarial perturbation (UAP) can foil most classification tasks conducted by DNNs. Thus, different methods for generating UAPs are required to fully evaluate the vulnerability of DNNs. A realistic evaluation would be with cases that consider targeted attacks; wherein the generated UAP causes the DNN to classify an input into a specific class. However, the development of UAPs for targeted attacks has largely fallen behind that of UAPs for non-targeted attacks. Therefore, we propose a simple iterative method to generate UAPs for targeted attacks. Our method combines the simple iterative method for generating non-targeted UAPs and the fast gradient sign method for generating a targeted adversarial perturbation for an input. We applied the proposed method to state-of-the-art DNN models for image classification and proved the existence of almost imperceptible UAPs for targeted attacks; further, we demonstrated that such UAPs can be easily generated.

1. Introduction

Deep neural networks (DNNs) are widely used for image classification, a task in which an input image is assigned a class from a fixed set of classes. For example, DNN-based image classification has applications in medical science (e.g., medical image-based diagnosis [1]) and self-driving technology (e.g., detecting and classifying traffic signs [2]).

However, DNNs are known to be vulnerable to adversarial examples [3], which are input images that cause misclassifications by DNNs and are generally generated by adding specific, imperceptible perturbations to the original input images that have been correctly classified using DNNs. Many methods for generating adversarial examples have been proposed [4,5]. For example, the fast gradient sign method (FGSM) [3] generates adversarial examples with a linear approximation; specifically, it performs a one step gradient update along the direction of the sign of the gradient of the loss function at each pixel. The basic iterative method [6] is an iterative version of the FGSM; in particular, it repeats the one step gradient update (i.e., FGSM) while the size of the adversarial perturbation is limited by a specified upper bound. DeepFool [7] considers a linear approximation and generates adversarial examples using the fact that the minimal perturbation of an affine classifier is the distance to the separating affine hyperplane. Carlini and Wagner [8] proposed an optimization algorithm for generating adversarial examples; specifically, they considered an optimization problem for finding the smallest adversarial perturbation that can fool the DNNs and solved the problem using gradient descent. The existence of adversarial examples questions the generalization ability of DNNs, reduces model interpretability, and limits the applications of deep learning in safety- and security-critical environments [9].

However, these methods have only considered input-dependent adversarial attacks (i.e., an individual adversarial perturbation is used such that each input image is misclassified). Such adversarial attacks are difficult tasks because they require high computational costs. More realistic adversarial attacks must be further considered. Notably, a single perturbation that can induce DNN failure in most image classification tasks is also generatable as a universal adversarial perturbation (UAP) [10,11]. UAP-based adversarial attacks can be more straightforward to implement by adversaries in real-world environments. As UAPs are image agnostic, adversaries do not need to consider input images when attacking DNNs; as a result, they can result in failed DNN-based image classification at lower costs. The vulnerability in DNNs to adversarial attacks (UAPs, in particular) is a security concern for practical applications of DNNs [10,11,12]. The development of methods for generating UAPs is required to evaluate the reliability and safety of DNNs against adversarial attacks.

A simple iterative method [10], a representative method, for generating UAPs has been proposed. To generate a UAP, this method iteratively updates a perturbation by additively obtaining individual adversarial perturbation for an input image randomly selected from an input image set. Khrulkov and Oseledets [13] proposed a method for generating UAPs based on computing the singular vectors of the Jacobian matrices of the hidden layers of a DNN. Mopuri et al. [14] proposed a network for adversary generation, a generative approach to model the distribution of UAPs, inspired by generative adversarial networks (GANs) [15]. However, these methods are limited to non-targeted attacks that cause misclassification (i.e., a task failure resulting in an input image being assigned an incorrect class).

More realistic cases need to consider targeted attacks, wherein generating a UAP would cause the DNN to classify an input image into a specific class (e.g., into the “diseased” class in medical diagnosis) [16]. The existence of targeted UAPs indicates that adversaries can control DNN-based image classification and may result in more significant security concerns compared with non-targeted attacks.

Several methods for generating UAPs for targeted attacks have been proposed. Hayes and Danezis [17] proposed a method based on a generative network model. Specifically, the method uses the universal adversarial network, trained similarly to the generator network in a GAN [15]. However, the generative network model is difficult to apply to large images (e.g., the ImageNet image dataset [18]) because it requires high computational costs. Brown et al. [19] proposed the targeted adversarial patch approach for targeted universal adversarial attacks. This approach adds an image patch to the input images to generate adversarial examples and seeks the location and transformations (e.g., rotations and scaling) of the patch on the input images that maximize the average confidence score for a target class of the adversarial examples. However, such adversarial patches are perceptible; thus, defenders can easily detect adversarial attacks.

A simpler algorithm for generating almost imperceptible UAPs for targeted attacks is required for practical applications. Thus, herein, we propose a simple iterative method to generate such targeted UAPs by extending the iterative method for generating non-targeted UAPs [10] and show the validity of the proposed method using representative DNN models for image classification.

2. Materials and Methods

2.1. Targeted Universal Adversarial Perturbations

Our algorithm (Algorithm 1) for generating UAPs for targeted attacks is an extension of the simple iterative algorithm for generating UAPs for non-targeted attacks [10]. Similar to the non-targeted UAP algorithm, our algorithm considers a classifier $C(\mathit{x})$ that returns the class or label (with the highest confidence score) for an input image $\mathit{x}$. The algorithm starts with $\mathit{\rho }=\mathbf{0}$ (no perturbation) and iteratively updates the UAP $\mathit{\rho }$ under the constraint that the $L_p$ norm of the perturbation is equal to or less than a small value $\xi$ (i.e., ${\parallel \mathit{\rho }\parallel }_p\le \xi$) by additively obtaining an adversarial perturbation for an input image $\mathit{x}$, which is randomly selected from an input image set $\mathit{X}$ without replacement. These iterative updates continue up till the termination conditions have been satisfied. Unlike the non-targeted UAP algorithm, our algorithm uses the fast gradient sign method for targeted attacks (tFGSM) to generate targeted UAPs, whereas the non-targeted UAP algorithm uses a method (e.g., DeepFool [7]) that generates a non-targeted adversarial example for an input image.

tFGSM generates a targeted adversarial perturbation $\psi (\mathit{x},y)$ that causes an image $\mathit{x}$ to be classified into the target class y using the gradient ${\nabla }_{\mathit{x}}\mathcal{L}(\mathit{x},y)$ of the loss function with respect to pixels [3,20]. The gradient can be computed efficiently using backpropagation (i.e., by computing the chain rule backwards) [3]. Specifically, the forward pass computes values from the inputs (pixels) to the output. The backward pass then performs backpropagation that starts at the end of the DNN with the loss value based on the output and recursively applies the chain rule to compute the gradients all the way to the inputs.

For the $L_{\infty }$ norm, the perturbation is calculated as:

$$\psi (\mathit{x},y)=-ϵ·\mathrm{sign}\left({\nabla }_{\mathit{x}}\mathcal{L}(\mathit{x},y)\right),$$

(1)

where $ϵ$ ($>0$) is the attack strength. For the $L_1$ and $L_2$ norms, the perturbation is obtained as:

$$\psi (\mathit{x},y)=-ϵ\frac{{\nabla }_{\mathit{x}}\mathcal{L}(\mathit{x},y)}{\parallel {\nabla }_{\mathit{x}}{\mathcal{L}(\mathit{x},y)\parallel }_p}.$$

(2)

The adversarial example ${\mathit{x}}_{\mathrm{adv}}$ is obtained as follows:

$${\mathit{x}}_{\mathrm{adv}}=\mathit{x}+\psi (\mathit{x},y).$$

(3)

At each iteration step, our algorithm computes a targeted adversarial perturbation $\psi (\mathit{x}+\mathit{\rho },y)$, if the perturbed image $\mathit{x}+\mathit{\rho }$ is not classified into the target class y (i.e., $C(\mathit{x}+\mathit{\rho })\ne y$); however, the non-targeted UAP algorithm obtains a non-targeted adversarial perturbation that satisfies $C(\mathit{x}+\mathit{\rho })\ne C(\mathit{x})$ if $C(\mathit{x}+\mathit{\rho })=C(\mathit{x})$. After generating the adversarial example at this step (i.e., ${\mathit{x}}_{\mathrm{adv}}\leftarrow \mathit{x}+\mathit{\rho }+\psi (\mathit{x}+\mathit{\rho },y))$, the perturbation $\mathit{\rho }$ is updated if ${\mathit{x}}_{\mathrm{adv}}$ is classified into the target class y (i.e., $C({\mathit{x}}_{\mathrm{adv}})=y$), whereas the non-targeted UAP algorithm updates the perturbation $\mathit{\rho }$ if $C(\mathit{x}+\mathit{\rho })\ne C(\mathit{x})$. Note that tFGSM does not ensure that adversarial examples are classified into a target class. When updating $\mathit{\rho }$, a projection function $\mathrm{project}(\mathit{x},p,\xi )$ is used to satisfy the constraint that ${\parallel \mathit{\rho }\parallel }_p\le \xi$ (i.e., $\mathit{\rho }\leftarrow \mathrm{project}({\mathit{x}}_{\mathrm{adv}}-\mathit{x},p,\xi )$. This projection is defined as follows:

$$\mathrm{project}(\mathit{x},p,\xi )=arg\underset{{\mathit{x}}^{\prime }}{min}\parallel \mathit{x}-{\mathit{x}}^{\prime }{\parallel }_2\mathrm{s}.\mathrm{t}.{\parallel {\mathit{x}}^{\prime }\parallel }_p\le \xi$$

(4)

This update procedure terminates when the targeted attack success rate $r_{ts}$ for input images (i.e., the proportion of input images classified into the target class; ${|\mathit{X}|}^{-1}{\sum }_{\mathit{x}\in \mathit{X}}\mathbb{I}\left(C(\mathit{x}+\mathit{\rho })=y\right)$) equals 100% (i.e., all input images are classified into the target class due to the UAP $\mathit{\rho }$) or the number of iterations reaches the maximum $i_{max}$.

The pseudocode of our algorithm is shown in Algorithm 1. Our algorithm was implemented using Keras (Version 2.2.4; keras.io) and the Adversarial Robustness 360 Toolbox [20] (Version 1.0; github.com/IBM/adversarial-robustness-toolbox).

Algorithm 1 Computation of a targeted UAP.

Input: Set $\mathit{X}$ of input images, target class y, classifier $C(·)$, cap $\xi$ on the $L_p$ norm of the perturbation, norm type p (1, 2, or ∞), maximum number $i_{max}$ of iterations. Output: Targeted UAP vector $\mathit{\rho }$.  1: $\mathit{\rho }\leftarrow \mathbf{0}$, $r_{st}\leftarrow 0$, $i\leftarrow 0$  2: while $r_{st}<1$ and $i<i_{max}$ do  3:  for $\mathit{x}\in \mathit{X}$ in random order do  4:   if $C(\mathit{x}+\mathit{\rho })\ne y$ then  5:    ${\mathit{x}}_{\mathrm{adv}}\leftarrow \mathit{x}+\mathit{\rho }+\psi (\mathit{x}+\mathit{\rho },y)$  6:    if $C({\mathit{x}}_{\mathrm{adv}})=y$ then  7:     $\mathit{\rho }\leftarrow \mathrm{project}({\mathit{x}}_{\mathrm{adv}}-\mathit{x},p,\xi )$  8:    end if  9:   end if  10:  end for  11:  $r_{st}\leftarrow {|\mathit{X}|}^{-1}{\sum }_{\mathit{x}\in \mathit{X}}\mathbb{I}\left(C(\mathit{x}+\mathit{\rho })=y\right)$  12:  $i\leftarrow i+1$  13: end while

2.2. Deep Neural Network Models and Image Datasets

To evaluate targeted UAPs, we used two DNN models that were trained to classify the CIFAR-10 image dataset (www.cs.toronto.edu/~kriz/cifar.html). The CIFAR-10 dataset includes 60,000 RGB color images with the size of $32\times 32$ pixels classified into 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. Six-thousand images are available in each class. The dataset comprises 50,000 training images (5000 images per class) and 10,000 test images (1000 images per class). In particular, we used the VGG-20 [21] and ResNet-20 models [22] for the CIFAR-10 dataset obtained from a GitHub repository (github.com/GuanqiaoDing/CNN-CIFAR10); their test accuracies were 91.1% and 91.3%, respectively.

Moreover, we also considered three DNN models trained to classify the ImageNet image dataset [18] (www.image-net.org). The ImageNet dataset is comprised of RGB color images with the size of $224\times 224$ pixels classified into 1000 classes. In particular, we used the VGG-16 [21], VGG-19 [21], and ResNet-50 models [22] for the ImageNet dataset available in Keras (Version 2.2.4; keras.io), and their test accuracies were 71.6%, 71.5%, and 74.6%, respectively.

2.3. Generating Targeted Adversarial Perturbations and Evaluating Their Performance

Targeted UAPs were generated using an input image set obtained from the datasets. The parameter p was set to two. We generated targeted UAPs with various magnitudes by adjusting the parameters $ϵ$ and $\xi$. The magnitude of a UAP was measured using a normalized $L_2$ norm of the perturbation; in particular, we used the ratio $\zeta$ of the $L_2$ norm of the UAP to the average $L_2$ norm of an image in a dataset. The average $L_2$ norms of an image were 7381 and 50,135 in the CIFAR-10 and ImageNet datasets, respectively.

TO compare the performance of the targeted UAPs generated by our method with random controls, we also generated random vectors (random UAPs) sampled uniformly from the sphere of a given radius [20].

The performance of the UAPs was evaluated using the targeted attack success rate $r_{ts}$. In particular, we considered the success rates $r_{ts}$ for input images. In addition to this, we also computed the success rates $r_{ts}$ for test images to experimentally evaluate the performance of UAPs for unknown images. A test image set was obtained from the dataset and did not overlap with the input image set.

3. Results and Discussion

3.1. Case of the CIFAR-10 Models

For the CIFAR-10 models, we used 10,000 input images to generate the targeted UAPs. The input image set was obtained by randomly selecting 1000 images per class from the training images of the CIFAR-10 dataset. All 10,000 test images of the dataset were used as test images for evaluating the UAP performance. We considered the targeted attack for each class. The parameters $ϵ$ and $i_{max}$ were set to 0.006 and 10, respectively.

For the targeted attacks for each class, the targeted attack success rates $r_{ts}$ for both the input image set and the UAP test image set rapidly increased with the perturbation rate, despite a low $\zeta$ (2–6%). In particular, the success rate was $>80\%$ for $\zeta =5\%$ (Figure 1); moreover, it reached $\sim 100\%$ for $\zeta >10\%$. The success rates of the targeted UAP with the same $\zeta$ for both the input image set and the UAP test image set were almost similar. This result indicates no overfitting of the generated UAPs to the input images.

The targeted UAPs with $\zeta =5\%$ were almost imperceptible (Figure 2). Moreover, the UAPs seemed to represent the object shapes of each target class. The success rates of the targeted UAPs were significantly higher than those of random UAPs. These tendencies were observed both in the VGG-20 model and in the ResNet-20 model. These results indicate that non-random patterns are required for targeted attacks and that the proposed method applies to various DNN architectures.

3.2. Case of ImageNet Models

For the ImageNet models, we used the validation dataset used in the ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012; www.image-net.org/challenges/LSVRC/2012/) to generate the targeted UAPs. The dataset is comprised of 50,000 images (50 images per class). We used 40,000 images as the input images. The input image set was obtained by randomly selecting 40 images per class. The rest (10,000 images; 10 images per class) was used as test images for evaluating the UAPs. The parameters $ϵ$ and $i_{max}$ were set to 0.5 and five, respectively.

In this study, we considered targeted attacks for three classes (golf ball, broccoli, and stone wall) that were randomly selected from 1000 classes in a previous study [17], to avoid redundancy.

We generated targeted UAPs with $\zeta =6\%$ ($\xi =3000$) and $\zeta =8\%$ ($\xi =4000$). The target attack success rates $r_{ts}$ were between ~30% and ~75% and between ~60% and ~90% when $\zeta =6\%$ and $\zeta =8\%$, respectively (

Table 1. Targeted attack success rates $r_{ts}$ of targeted UAPs against the DNN models for each target class. $r_{ts}$ for input images and test images are shown.

Target Class	Model	$\mathit{\zeta }=6\%$		$\mathit{\zeta }=8\%$
		Input	Test	Input	Test
Golf ball	VGG-16	58.0%	57.6%	81.6%	80.6%
	VGG-19	55.3%	55.2%	81.3%	80.1%
	ResNet-50	66.8%	66.5%	90.3%	89.8%
Broccoli	VGG-16	29.3%	29.0%	59.7%	59.5%
	VGG-19	31.2%	30.5%	59.7%	59.4%
	ResNet-50	46.4%	46.6%	74.6%	73.9%
Stone wall	VGG-16	47.1%	46.7%	75.0%	74.5%
	VGG-19	48.4%	48.1%	73.9%	72.9%
	ResNet-50	74.7%	74.4%	92.0%	91.3%

). The success rates for the input images were almost similar to those for the test images, indicating no overfitting of the generated UAPs to the input image set. The success rates of the targeted UAPs were significantly higher than those of random UAPs, which were less than 1% in all cases. This indicates that non-random patterns are required for targeted attacks.

A higher perturbation magnitude $\zeta$ led to a higher targeted attack success rate $r_{ts}$. The success rates $r_{ts}$ depend on the image classes. For example, the targeted attacks for the class “golf ball” were more easily achieved than those for the class “broccoli”. The success rates $r_{ts}$ also depended on the DNN architectures; in particular, the ResNet-50 model was more easy to fool than the VGG models. Overall, however, we confirmed that the proposed method was applicable to various DNN architectures.

The targeted UAPs with $\zeta =6\%$ and $\zeta =8\%$ were almost imperceptible (Figure 3); however, they were partly perceptible for whitish images (e.g., trimaran). Moreover, the UAPs seem to reflect object shapes of each target class.

The targeted attack success rates in the ImageNet models were relatively lower than those in the CIFAR-10 models. This is because the ImageNet dataset has a larger number of classes than the CIFAR-10 dataset. In short, it is more difficult to exactly classify an input image into a specific target class within a larger number of classes. Moreover, the observed lower success rate may be because the validation dataset of ILSVRC2012 was used when generating targeted UAPs. Higher success rates may be obtained when generating targeted UAPs using training images.

These results indicate that the proposed method has several advantages for generating UAPs. As Chaubey et al. [11] mentioned, the previous methods for generating UAPs are limited to non-targeted attacks, except for a generative model-based method [17]. The generative model-based method requires high computational costs; thus, it does not apply to large image datasets such as the ImageNet dataset. On the other hand, our simple iterative method can compute targeted UAPs for large image datasets. Our method can help reliability assessments and increase safety [4,8] in practical applications of DNNs (e.g., medical image-based diagnosis [1,23] and detecting and classifying traffic signs [2,24]).

4. Conclusions

We propose a simple iterative method to generate targeted UAPs for image classification, although the proposed algorithm is a straightforward extension of the non-targeted UAP algorithm [10]. Using the CIFAR-10 and ImageNet models, we demonstrated that a small (almost imperceptible) UAP generated by our method made the models largely classify test images into a target class. Our results indicated the existence of UAPs for targeted attacks and that such UAPs can be easily generated. Our study enhances our understanding of the vulnerabilities of DNNs to adversarial attacks; moreover, it may help accelerate practical applications of DNNs. The source code of our proposed method for generating targeted UPAs is available from our GitHub repository: github.com/hkthirano/targeted_UAP_CIFAR10. Our method is also available in the Adversarial Robustness Toolbox [20] (Version 1.4 and later; github.com/Trusted-AI/adversarial-robustness-toolbox), a Python library for machine learning security.