2024人工智能AI技术教程.pdf

2024人工智能AI技术教程

课程讲义名称备注

1课程介绍Overviewandsystem/AIbasics

人工智能系统概述SystemforAI:ahistoricview;Fundamentalsofneuralnetworks;

2

SystemperspectiveofSystemforAIFundamentalsofSystemforAI

深度神经网络计算框架基础BackpropandAD,Tensor,DAG,Executiongraph

3

ComputationframeworksforDNNPapersandsystems:PyTorch,TensorFlow

矩阵运算与计算机体系结构Matrixcomputation,CPU/SIMD,GPGPU,ASIC/TPU

4

ComputerarchitectureforMatrixcomputationPapersandsystems:Blas,TPU

分布式训练算法Dataparallelism,modelparallelism,distributedSGD

5

DistributedtrainingalgorithmsPapersandsystems:

分布式训练系统MPI,parameterservers,all-reduce,RDMA

6

DistributedtrainingsystemsPapersandsystems:Horovod

异构计算集群调度与资源管理系统RunningDNNjoboncluster:container,resourceallocation,scheduling

7

SchedulingandresourcemanagementsystemPapersandsystems:KubeFlow,OpenPAI,Gandiva,HiveD

深度学习推导系统

8Efficiency,latency,throughput,anddeployment

Inferencesystems

课程讲义名称备注

计算图编译优化IR,sub-graphpatternmatch,Matrixmultiplicationandmemory

9optimization

Computationgraphcompilationandoptimization