Project description

Artificial intelligence (AI), especially deep neural networks (DNNs), has been widely used, including in applications with security and privacy concerns. Different from traditional software, the performance and security of DNNs highly depend on the data used to train the model, which by no means will be exhaustively tested. Therefore, this project targets the scenario when the DNN fails to operate as expected after it has been deployed. The failure could be caused by, e.g., lack of training on certain features or purposely embedded backdoor via data poisoning. This project aims to automatically repair the model, i.e., generate updates for the DNN that fix such problems, with the ultimate goal of developing resilient AI systems. An update for the DNN is a set of changes to its learnable parameters, i.e., the weights and biases. It could be for increasing the model prediction accuracy, fixing the backdoor, or improving the adversarial robustness of the model.

In this project, the Ph.D. student would work on the fundamental theory for repairing DNNs, which must be applicable to real-world setup and implement it. Realistically, automated repair techniques do not require re-training the DNN to correct the failures, thus not relying on the training data. As it becomes common to use or purchase DNN models developed by a third party, the training data is not always publicly available. It can also be that customers, who use remote computing power for training the DNN, have privacy motivations for withholding the data. Subject to these security and privacy concerns, the repair automatically fixes DNN failures recorded at its runtime; meanwhile, the changes shall not compromise the model's originally correct behaviors.