I am a Ph.D. student at the National University of Singapore’s School of Computing working under the supervision of Reza Shokri. My research interests are in the privacy and fairness of machine learning, federated learning and Large Language Models.
Publications
Chang, H., Hassani, H., & Shokri, R. (2024). Watermark Smoothing Attacks against Language Models. ArXiv Preprint ArXiv:2407.14206.
@article{chang2024watermark,
title = {Watermark Smoothing Attacks against Language Models},
author = {Chang, Hongyan and Hassani, Hamed and Shokri, Reza},
journal = {arXiv preprint arXiv:2407.14206},
date = {2024},
doi = {2407.14206}
}
Watermarking is a technique used to embed a hidden signal in the probability distribution of text generated by large language models (LLMs), enabling attribution of the text to the originating model. We introduce smoothing attacks and show that existing watermarking methods are not robust against minor modifications of text. An adversary can use weaker language models to smooth out the distribution perturbations caused by watermarks without significantly compromising the quality of the generated text. The modified text resulting from the smoothing attack remains close to the distribution of text that the original model (without watermark) would have produced. Our attack reveals a fundamental limitation of a wide range of watermarking techniques.
Chang, H., Edwards, B., Paul, A., & Shokri, R. (2024). Efficient Privacy Auditing in Federated Learning. Usenix Security Symposium (USENIX).
@article{chang2024efficient,
title = {Efficient Privacy Auditing in Federated Learning},
author = {Chang, Hongyan and Edwards, Brandon and Paul, Anindya and Shokri, Reza},
journaltitle = {Usenix Security Symposium (USENIX)},
date = {2024},
file = {chang2024efficient.pdf}
}
We design a novel efficient membership inference attack to audit privacy risks in federated learning. Our approach involves computing the slope of specific model performance metrics (eg, model’s output and its loss) across FL rounds to differentiate members from non-members. Since these metrics are automatically computed during the FL process, our solution imposes negligible overhead and can be seamlessly integrated without disrupting training. We validate the effectiveness and superiority of our method over prior work across a wide range of FL settings and real-world datasets.
Ganesh, P., Chang, H., Strobel, M., & Shokri, R. (2023). On The Impact of Machine Learning Randomness on Group Fairness. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT) - Best Paper Award.
@inproceedings{hampton2021black,
title = {On The Impact of Machine Learning Randomness on Group Fairness},
author = {Ganesh, Prakhar and Chang, Hongyan and Strobel, Martin and Shokri, Reza},
booktitle = {Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT) - Best Paper Award},
date = {2023},
urla = {https://dl.acm.org/doi/abs/10.1145/3593013.3594116}
}
Statistical measures for group fairness in machine learning reflect the gap in performance of algorithms across different groups. These measures, however, exhibit a high variance between different training instances, which makes them unreliable for empirical evaluation of fairness. What causes this high variance? We investigate the impact on group fairness of different sources of randomness in training neural networks. We show that the variance in group fairness measures is rooted in the high volatility of the learning process on under-represented groups. Further, we recognize the dominant source of randomness as the stochasticity of data order during training. Based on these findings, we show how one can control group-level accuracy (i.e., model fairness), with high efficiency and negligible impact on the model’s overall performance, by simply changing the data order for a single epoch.
Chang, H., & Shokri, R. (2023). Bias Propagation in Federated Learning. International Conference on Learning Representations (ICLR).
@article{chang2023bias,
title = {Bias Propagation in Federated Learning},
author = {Chang, Hongyan and Shokri, Reza},
journaltitle = {International Conference on Learning Representations (ICLR)},
date = {2023},
urla = {https://openreview.net/pdf?id=V7CYzdruWdm}
}
We show that participating in federated learning can be detrimental to group fairness. In fact, the bias of a few parties against under-represented groups (identified by sensitive attributes such as gender or race) can propagate through the network to all the parties in the network. We analyze and explain bias propagation in federated learning on naturally partitioned real-world datasets. Our analysis reveals that biased parties unintentionally yet stealthily encode their bias in a small number of model parameters, and throughout the training, they steadily increase the dependence of the global model on sensitive attributes. What is important to highlight is that the experienced bias in federated learning is higher than what parties would otherwise encounter in centralized training with a model trained on the union of all their data. This indicates that the bias is due to the algorithm. Our work calls for auditing group fairness in federated learning and designing learning algorithms that are robust to bias propagation.
Chang, H., & Shokri, R. (2021). On the privacy risks of algorithmic fairness. 6th IEEE European Symposium on Security and Privacy (Euro S&P).
@article{chang2020privacy,
title = {On the privacy risks of algorithmic fairness},
author = {Chang, Hongyan and Shokri, Reza},
urla = {https://ieeexplore.ieee.org/abstract/document/9581219},
journaltitle = {6th IEEE European Symposium on Security and Privacy (Euro S&P)},
date = {2021}
}
Algorithmic fairness and privacy are essential pillars of trustworthy machine learning. Fair machine learning aims at minimizing discrimination against protected groups by, for example, imposing a constraint on models to equalize their behavior across different groups. This can subsequently change the influence of training data points on the fair model, in a disproportionate way. We study how this can change the information leakage of the model about its training data. We analyze the privacy risks of group fairness (e.g., equalized odds) through the lens of membership inference attacks: inferring whether a data point is used for training a model. We show that fairness comes at the cost of privacy, and this cost is not distributed equally: the information leakage of fair models increases significantly on the unprivileged subgroups, which are the ones for whom we need fair learning. We show that the more biased the training data is, the higher the privacy cost of achieving fairness for the unprivileged subgroups will be. We provide comprehensive empirical analysis for general machine learning algorithms.
Chang, H., Nguyen, T. D., Murakonda, S. K., Kazemi, E., & Shokri, R. (2020). On adversarial bias and the robustness of fair machine learning. In arXiv.
@unpublished{chang2020adversarial,
title = {On adversarial bias and the robustness of fair machine learning},
author = {Chang, Hongyan and Nguyen, Ta Duy and Murakonda, Sasi Kumar and Kazemi, Ehsan and Shokri, Reza},
journaltitle = {arXiv},
date = {2020},
doi = {2006.08669}
}
Optimizing prediction accuracy can come at the expense of fairness. Towards minimizing discrimination against a group, fair machine learning algorithms strive to equalize the behavior of a model across different groups, by imposing a fairness constraint on models. However, we show that giving the same importance to groups of different sizes and distributions, to counteract the effect of bias in training data, can be in conflict with robustness. We analyze data poisoning attacks against group-based fair machine learning, with the focus on equalized odds. An adversary who can control sampling or labeling for a fraction of training data, can reduce the test accuracy significantly beyond what he can achieve on unconstrained models. Adversarial sampling and adversarial labeling attacks can also worsen the model’s fairness gap on test data, even though the model satisfies the fairness constraint on training data. We analyze the robustness of fair machine learning through an empirical evaluation of attacks on multiple algorithms and benchmark datasets.
Chang, H., Shejwalkar, V., Shokri, R., & Houmansadr, A. (2021). Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer. NFFL at NeurIPS.
@article{chang2019cronus,
author = {Chang, Hongyan and Shejwalkar, Virat and Shokri, Reza and Houmansadr, Amir},
title = {Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer},
journaltitle = {NFFL at NeurIPS},
date = {2021},
doi = {1912.11279}
}
Collaborative (federated) learning enables multiple parties to train a model without sharing their private data, but through repeated sharing of the parameters of their local models. Despite its advantages, this approach has many known privacy and security weaknesses and performance overhead, in addition to being limited only to models with homogeneous architectures. Shared parameters leak a significant amount of information about the local (and supposedly private) datasets. Besides, federated learning is severely vulnerable to poisoning attacks, where some participants can adversarially influence the aggregate parameters. Large models, with high dimensional parameter vectors, are in particular highly susceptible to privacy and security attacks: curse of dimensionality in federated learning. We argue that sharing parameters is the most naive way of information exchange in collaborative learning, as they open all the internal state of the model to inference attacks, and maximize the model’s malleability by stealthy poisoning attacks. We propose Cronus, a robust collaborative machine learning framework. The simple yet effective idea behind designing Cronus is to control, unify, and significantly reduce the dimensions of the exchanged information between parties, through robust knowledge transfer between their black-box local models. We evaluate all existing federated learning algorithms against poisoning attacks, and we show that Cronus is the only secure method, due to its tight robustness guarantee. Treating local models as black-box, reduces the information leakage through models, and enables us using existing privacy-preserving algorithms that mitigate the risk of information leakage through the model’s output (predictions). Cronus also has a significantly lower sample complexity, compared to federated learning, which does not bind its security to the number of participants.
Open Source Library
Privacy Meter @ NUS: A valuable resource for privacy research and deployment. 500+ stars on GitHub.
Leading the development team and spearheading the initial 1.0.1 release.
