The fragility of CNNs

Convolutional Neural Networks (CNNs) have been the big disruptors in the computer vision landscape so far, powering automated systems in everything from medical imaging to self-driving cars and agricultural analysis. However, I like to believe there is a critical vulnerability that might threaten their efficiency if we aren't careful: adversarial attacks. Back in my senior year of high school, I had the opportunity to work on a project related to CNNs and adversarial attacks on them, and in this brief article, I will touch on my experience and research.

To put it very simply, adversarial attacks focus on exploiting the mechanics of a neural network, simulating a sequence of invisible pixel changes to deceive the model's prediction. This is a direct attack on how these systems see the real world; for example, let's say my model is looking at a picture of a dog, and I know if I add adversarial noise to it, it'll classify it as a dolphin with 98.2% confidence. Here is a very famous example from the paper Explaining and Harnessing Adversarial Examples.

This is because of the Fast Gradient Sign Method (FGSM). Basically, before the model makes a prediction, FGSM runs a calculation using the gradients of a loss function to predict what the most damaging perturbation might be; they build a distorted representation of an image and predict how the model will fail given a specific set of alterations. Fun fact: this is actually a relatively old model, but with the rise of artificial intelligence models, this becomes more dangerous than ever.

The first time I learned about adversarial attacks, I wondered why one should use specialized defenses since the visual noise seems so minor to the human eye, and the first time I saw an example, I could not make out whether it was distorted or not. In my study, I used a Kaggle dataset, specifically that of Cassava leaves. The idea of my project was that without a robust defense model, an automated farming system might misclassify the Cassava Mosaic Virus as a healthy leaf hundreds or even thousands of times due to slight sensor noise or deliberate attacks. But with a strong defense mechanism (such as Defensive Distillation), the system can perform most of its filtering accurately.

My research utilized 2 distinct defense components to work against these attacks; they are Adversarial Training and Defensive Distillation.

Firstly, Adversarial Training takes raw adversarial examples generated by FGSM and purposely mixes them into the training dataset. Next, Defensive Distillation utilizes a complex teacher model that predicts softened probability distributions. This frequently uses a temperature scaling function to produce more gradual probabilities. Lastly, a simpler student model uses these internal softened labels to pick the optimal flow of classifications (without fear of failure from sharp, sudden input changes). When evaluating the two in my study, Defensive Distillation actually yielded a higher clean accuracy of 11.5% compared to Adversarial Training's 9.9%.

Equipping automated agricultural systems (and all systems at large) with the ability to resist attacks and smooth out their decision boundaries will unlock massive advancements in global food security and healthcare. While my results showed both models still drop to a 0% robust accuracy against high-strength perturbations, I look forward to the continued progress in hybrid defense techniques and robust modeling.

Anyways, thanks for reading, and have a great day.