数据与模型安全 课件 第8周：数据抽取和模型窃取.pptx

DataExtractionandModelStealing姜育刚，马兴军，吴祖煊

Recap:week7ABriefHistoryofBackdoorLearningBackdoorAttacksBackdoorDefensesFutureResearch

ThisWeekDataExtractionAttackDefenseModelStealingAttackFutureResearch

ThisWeekDataExtractionAttackDefenseModelStealingAttackFutureResearch

DataExtractionAttack通过模型逆向得到训练数据:8001/dss/imageClassify

TerminologyThefollowingtermsdescribethesamething:DataExtractionAttackDataStealingAttackTrainingDataExtractionAttackModelMemorizationAttackModelInversionAttack

SecurityThreatsMysocialsecuritynumberis078-PersonalInfoLeakageSensitiveInfoLeakageThreatstoNationalSecurityIllegalDataTrading…

MemorizationofDNNsEvidence1:DNNlearnsdifferentlevelsofrepresentations

MemorizationofDNNsEvidence2:DNNcanmemorizerandomlabels/pixels真实标签随机标签乱序像素随机像素高斯噪声Zhang,Chiyuan,etal.“Understandingdeeplearningrequiresrethinkinggeneralization.”ICLR2017.

MemorizationofDNNsEvidence3:ThesuccessofGANsanddiffusionmodels/;/

Intendedvs.UnintendedMemorizationIntendedMemorizationTask-relatedStatisticsInputsandLabelsArpitetal.“Acloserlookatmemorizationindeepnetworks.”?ICML,2017.Carlinietal.“Thesecretsharer:Evaluatingandtestingunintendedmemorizationinneuralnetworks.”USENIXSecurity,2019.第一层Filter正常CIFAR-10第一层Filter随机标注CIFAR-10自然语言翻译模型记忆：“我的社保号码是xxxx”UnintendedMemorizationTask-irrelevantbutmemorizedEvenappearonlyafewtimes出现4次就能全记住

现有数据窃取攻击

黑盒窃取主动测试：煤矿里的金丝雀“随机号码为****”“我的社保号码为****”主动注入，然后先兆数据在语言模型中的“曝光度”（Exposure）Carlinietal.“Thesecretsharer:Evaluatingandtestingunintendedmemorizationinneuralnetworks.”USENIXSecurity,2019.意外记忆测试和量化：’先兆’

黑盒窃取针对通用语言模型：逆向出大量的：名字、手机号、邮箱、社保号等大模型比小模型更容易记住这些信息即使只在一个文档里出现也能被记住Carlini,Nicholas,etal.“Extractingtrainingdatafromlargelanguagemodels.”?USENIXSecurity,2021.训练数据萃取攻击TrainingDataExtractionAttack

DefinitionofMemorizationCarlini,Nicholas,etal.“Extractingtrainingdatafromlargelangua