数据与模型安全 课件 第2周：可解释性和普通鲁棒性.pptx

Explainability

CommonRobustness姜育刚，马兴军，吴祖煊

1.WhatisMachineLearning2.MachineLearningParadigms3.LossFunctionsRecap:week14.OptimizationMethods

MachineLearningPipelinesetuptheinputsetuptheoptimisersetupthelossregularizationmakesdecisionregionsmootherlandscape?ofalossfunction,itvariesw.r.t.data,thefunctionitself

MachineLearningPipelinesetuptheinputsetuptheoptimisersetupthelossregularizationmakesdecisionregionsmootherlandscape?ofalossfunction,itvariesw.r.t.data,thefunctionitselfModel？

DeepNeuralNetworks/neural-network-zoo/;/articles/cc-machine-learning-deep-learning-architectures/

Feed-ForwardNeuralNetworksFeed-ForwardNeuralNetworks(FNN)FullyConnectedNeuralNetworks(FCN)MultilayerPerceptron(MLP)ThesimplestneuralnetworkFully-connectedbetweenlayersFordatathathasNOtemporalorspatialorder/

ConvolutionalNeuralNetworksForimagesordatawithspatialorderCanstackupto100layers/Neuronsin3dimensionsNeuronsinoneflatlayer

RecurrentNeuralNetworks/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networksTraditionalRNN

TransformersVaswani,Ashish,etal.Attentionisallyouneed.?Advancesinneuralinformationprocessingsystems?30(2017)Transformer:anewtypeofDNNsbasedonattentionEncoderDecoder

Self-AttentionExplained/illustrated-self-attention-2d627e33b20a

CNNExplainedLearnsdifferentlevelsofrepresentations/AbriefhistoryofCNNs:LeNet,1990sAlexNet,2012ZFNet,2013GoogLeNet,2014VGGNet,2014ResNet,2015InceptionV4,2016ResNeXt,2017ViT,2021AnImageisWorth16x16Words:TransformersforImageRecognitionatScale,ICLR2021

ExplainableAI深度学习可解释性学习机理推理机理泛化机理认知机理鲁棒性学习过程学习结果决策依据推理机制泛化原因泛化条件认知科学认知启发的智能普通鲁棒性对抗鲁棒性我们想要弄清楚下列问题：DNN是怎么学习的、学到了什么、靠什么泛化、在什么情况下行又在什么情况下不行？深度学习是否是真正的智能，与人类智能比谁更高级，它的未来是什么？是否存在大一统的理论，不但能解释而且能提高？

MethodologicalPrinciplesVisualizationAblationContrastModelComponentLayerOperationNeuronSuperclassClassTraining/TestsetSubsetSampleTrainingInferenc