Regularization refers to a set of techniques designed to prevent overfitting and improve the generalization performance of the network on unseen data. Overfitting occurs when a model learns the training data too well, capturing noise and details that are specific to that dataset but do not generalize well to new, unseen data. Regularization methods help to control the complexity of the neural network and promote better generalization by adding constraints or penalties to the learning process.

Posts

Towards Robustness of Deep Neural Networks via Networks via Regularization

Towards Robustness of Deep Neural Networks via Networks via Regularization Recent studies have demonstrated the vulnerability of deep neural networks against adversarial examples. In-spired by the observation that adversarial examples often lie outside the natural image data manifold and the intrinsic dimension of image data is much smaller than its pixel space dimension, we propose to embed high-dimensional input images into a low-dimensional space and apply regularization on the embedding space to push the adversarial examples back to the manifold. The proposed framework is called Embedding Regularized Classifier (ER-Classifier), which improves the adversarial robustness of the classifier through embedding regularization. Besides improving classification accuracy against adversarial examples, the framework can be combined with detection methods to detect adversarial examples. Experimental results on several benchmark datasets show that, our proposed framework achieves good performance against strong adversarial at-tack methods.

Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding

Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples. Inspired by the observation that the intrinsic dimension of image data is much smaller than its pixel space dimension and the vulnerability of neural networks grows with the input dimension, we propose to embed high-dimensional input images into a low-dimensional space to perform classification. However, arbitrarily projecting the input images to a low-dimensional space without regularization will not improve the robustness of deep neural networks. Leveraging optimal transport theory, we propose a new framework, Optimal Transport Classifier (OT-Classifier), and derive an objective that minimizes the discrepancy between the distribution of the true label and the distribution of the OT-Classifier output. Experimental results on several benchmark datasets show that, our proposed framework achieves state-of-the-art performance against strong adversarial attack methods.