Privacy-Preserving Neural Networks: Practical Exploration of Training Neural Networks in a Secure Multi-Party Computation Environment

This project is a graduation thesis completed by Kuber Shahi from Ashoka University in December 2021. It combines a neural network built from scratch with cryptographic primitives to enable neural network training in a three-party secure computation (3PC) environment without directly sharing raw sensitive data. The core technologies reference the SecureNN and SecureML papers, including a plaintext baseline neural network and a secure computation module, solving the conflict between data privacy and model training, and providing a runnable reference implementation for privacy-preserving machine learning (PPML).

Modern machine learning relies on large amounts of sensitive data (medical records, financial transactions, etc.), and traditional centralized training faces privacy compliance challenges. Privacy-preserving machine learning (PPML) has emerged as a solution, with secure multi-party computation (MPC) as its core technology, allowing participants to collaboratively train models in an encrypted state, only obtaining results without leaking input data.

Plaintext Neural Network

Uses a 2-layer fully connected architecture: input layer with 784 neurons (MNIST 28×28), hidden layer with 256 neurons (ReLU activation), output layer with 10 neurons (Softmax activation). Trains on the MNIST dataset using cross-entropy loss and mini-batch SGD.

Secure Computation Module

Implements core MPC primitives: fixed-point number mapping/inverse mapping (SecureML style), ring truncation, additive secret sharing (ℤₗ and ℤₚ rings), secure matrix multiplication (Beaver triples), and private comparison (supports shared/non-shared modes).

1. Data Sharing: Participants convert data into secret shares and distribute them to three computing parties;
2. Forward Propagation: Use secure matrix multiplication to compute activation values (intermediate results are secret shares);
3. Activation Function: Implement ReLU activation via private comparison;
4. Backward Propagation: Securely compute gradients and update weights;
5. Result Reconstruction: Recombine model parameter shares after training. During the process, a single or a few participants cannot obtain raw data or intermediate information.

Privacy-preserving neural networks can be applied in:

• Healthcare: Multiple hospitals collaborate to train diagnostic models without sharing patient records;
• Financial Services: Jointly train fraud detection models to protect customer transaction privacy;
• Smart Cities: Collaboratively optimize traffic prediction models while protecting traffic data;
• Edge Computing: IoT devices collaborate on learning without uploading raw sensor data.

PPML faces inherent challenges:

• Computational Overhead: MPC operations are orders of magnitude slower than plaintext;
• Communication Complexity: Frequent information exchange leads to network latency;
• Accuracy Loss: Fixed-point numbers and truncation introduce numerical errors;
• Scalability: Protocol complexity increases as the number of participants grows. This project achieves a balance in the three-party setting through protocol optimization.

This project demonstrates the combination of cryptography and ML, solving the conflict between data privacy and model performance. It provides researchers with: a clear mapping between theory and practice, runnable code, detailed document derivations, and verified protocol combinations. As regulations like GDPR become stricter, PPML will become an essential capability for AI deployment, and the project's open-source implementation provides valuable references for academia and industry.