Privacy-preservation for gradient descent methods

ABSTRACT

Gradient descent is a widely used paradigm for solving many optimization problems. Stochastic gradient descent performs a series of iterations to minimize a target function in order to reach a local minimum. In machine learning or data mining, this function corresponds to a decision model that is to be discovered. The gradient descent paradigm underlies many commonly used techniques in data mining and machine learning, such as neural networks, Bayesian networks, genetic algorithms, and simulated annealing. To the best of our knowledge, there has not been any work that extends the notion of privacy preservation or secure multi-party computation to gradient-descent-based techniques. In this paper, we propose a preliminary approach to enable privacy preservation in gradient descent methods in general and demonstrate its feasibility in specific gradient descent methods.

References

1. W. Du and M. Atallah. Privacy-Preserving Cooperative Statistical Analysis. Proceedings of the 17th Annual Computer Security Applications Conference, pp. 103--110, Louisiana, USA, 2001.
2. W. Du, Y. S. Han, and S. Chen. Privacy-preserving Multivariate Statistical Analysis: Linear Regression and Classification. Proceedings of the SIAM International Conference on Data Mining, Florida, USA, 2004.
3. W. Du and Z. Zhan. Building Decision Tree Classfier on Private Data. Workshop on Privacy, Security, and Data Mining, held in conjunction with the IEEE International Conference on Data Mining, pp. 1--8, Maebashi City, Japan, 2002.
4. B. Goethals, S. Laur, H. Lipmaa, and T. Mielik. On Private Scalar Product Computation for Privacy-Preserving Data Mining. Proceedings of the 7th Annual International Conference on Information Security and Cryptology, pp. 104--120, Seoul, Korea, 2004.
5. O. Goldreich. The Foundations of Cryptography. Cambridge University Press, 2004.
6. M. Kantarcioglu and C. Clifton. Privacy-Preserving Distributed Mining of Association Rules on Horizontally Partitioned Data. ACM SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery, pp. 24--31, 2002.
7. T. Mitchell. Machine Learning. McGraw-Hill, 1997.
8. S. Russell, J. Binder, D. Koller, and K. Kanazawa. Local Learning in Probabilistic Networks with Hidden Variables. Proceedings of the 14th International Joint Conference on Artificial Intelligence, Montreal, Canada, 1995.
9. J. Vaidya and C. Clifton. Privacy Preserving Association Rule Mining in Vertically Partitioned Data. Proceedings of the 8th ACM International Conference on Knowledge Discovery and Data Mining, pp. 639--644, Edmonton, Canada, 2002.
10. J. Vaidya and C. Clifton. Privacy Preserving K-means Clustering over Vertically Partitioned Data. Proceedings of the 9th ACM International Conference on Knowledge Discovery and Data Mining, pp. 206--215, Washington D. C., USA, 2003.
11. J. Vaidya and C. Clifton. Privacy Preserving Naive Bayes Classifier for Vertically Partitioned Data. Proceedings of the SIAM International Conference on Data Mining, Florida, USA, 2004.
12. J. Vaidya and C. Clifton. Privacy-Preserving Decision Trees over Vertically Partitioned Data. Proceedings of the 19th Annual IFIP WG 11.3 Working Conference on Data and Applications Security, Storrs, USA, 2005.
13. A. Yao. How to Generate and Exchange Secrtes. Proceedings of the 27th IEEE Symposium on Foundations of Computer Science, pp. 162--167, 1986.
14. H. Yu, X. Jiang, and J. Vaidya. Privacy-Preserving SVM using Nonlinear Kernels on Horizontally Partitioned Data. Proceedings of the 21st Annual ACM Symposium on Applied Computing, Dijon, France, 2006.
15. H. Yu, X. Jiang, and J. Vaidya. Privacy Preserving SVM Classification on Vertically Partitioned Data. Proceedings of the 10th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Singapore, 2006.

Index Terms

• Information systems

  • Information systems applications

    • Data mining

• Security and privacy

  • Database and storage security

• Theory of computation

  • Theory and algorithms for application domains

    • Database theory

      • Theory of database privacy and security