OPTIMIZING CONFIDENTIAL DEEP LEARNING FOR REAL-TIME SYSTEMS

Mohammad Fakhruddin Babar

Deep neural networks (DNNs) are increasingly used in time-critical, learning-enabled cyber-physical applications such as autonomous driving and robotics. Despite the growing use of various deep learning models, protecting DNN inference from adversarial threats while preserving model privacy and confidentiality remains a key concern for resource and timing-constrained autonomous cyber-physical systems. One potential solution, primarily used in general-purpose systems, is the execution of the DNN workloads within trusted execution environments (TEEs) available on current off-the-shelf processors. I review the various TEE architectures and techniques employed to achieve secure neural network execution and provide a classification of existing work. Additionally, I discuss the challenges and present a few open issues. However, ensuring temporal guarantees when running DNN inference within these trusted enclaves poses significant challenges in real-time applications due to (a) the large computational and memory demands of DNN models and (b) the overhead introduced by frequent context switches between “normal” and “trusted” execution modes. This thesis introduces new time-aware schemes for dynamic (EDF) and fixed-priority (RM) schedulers to preserve the confidentiality of DNN tasks by running them inside trusted enclaves. I first propose a technique that slices each DNN layer and runs them sequentially in the enclave. However, due to the extra context switch overheads of individual layer slices, I further introduce a novel layer fusion technique. Layer fusion improves real-time guarantees by grouping multiple layers of DNN workload from multiple tasks, thus allowing them to fit and run concurrently within the enclaves while maintaining timing constraints. I implemented and tested my ideas on the Raspberry Pi platform running a DNN-enabled trusted operating system (OP-TEE with DarkNet-TZ) and three DNN architectures (AlexNet-squeezed, Tiny Darknet, YOLOv3-tiny). Compared to the layer-wise partitioning approach, layer fusion can (a) schedule up to 3x more tasksets for EDF and 5x for RM and (b) reduce context switches by up to 11.12x for EDF and by up to 11.06x for RM.

OPTIMIZING CONFIDENTIAL DEEP LEARNING FOR REAL-TIME SYSTEMS

Files and links (1)

Abstract

Metrics

Details