DeepSeek-V3技术报告（英）范文.pptx

90.280.075.974.773.874.672.666.265.059.151.651.149.042.039.238.835.623.323.320.316.09.3WepresentDeepSeek-V3,astrongMixture-of-Experts(MoE)languagemodelwith671Btotalparameterswith37Bactivatedforeachtoken.Toachieveefficientinferenceandcost-effectivetraining,DeepSeek-V3adoptsMulti-headLatentAttention(MLA)andDeepSeekMoEarchitec-tures,whichwerethoroughlyvalidatedinDeepSeek-V2.Furthermore,DeepSeek-V3pioneersanauxiliary-loss-freestrategyforloadbalancingandsetsamulti-tokenpredictiontrainingobjectiveforstrongerperformance.Wepre-trainDeepSeek-V3on14.8trilliondiverseandhigh-qualitytokens,followedbySupervisedFine-TuningandReinforcementLearningstagestofullyharnessitscapabilities.ComprehensiveevaluationsrevealthatDeepSeek-V3outperformsotheropen-sourcemodelsandachievesperformancecomparabletoleadingclosed-sourcemodels.Despiteitsexcellentperformance,DeepSeek-V3requiresonly2.788MH800GPUhoursforitsfulltraining.Inaddition,itstrainingprocessisremarkablystable.Throughouttheentiretrainingprocess,wedidnotexperienceanyirrecoverablelossspikesorperformanyrollbacks.Themodelcheckpointsareavailableat/deepseek-ai/DeepSeek-V3.Figure1|BenchmarkperformanceofDeepSeek-V3anditscounterparts.DeepSeek-V3TechnicalReportDeepSeek-V3DeepSeek-V2.5Qwen2.5-72B-InstLlama-3.1-405B-InstGPT-4o-0513Claude-3.5-Sonnet-1022DeepSeek-AIresearch@SWE-benchVerified(Resolved)Accuracy/Percentile(%)GPQA-Diamond(Pass@1)AbstractAIME2024(Pass@1)Codeforces(Percentile)MMLU-Pro(EM)MATH500(EM)10049.941.325.324.824.523.823.622.678.373.378.071.650.816.7408060200

Contents1Introduction42Architecture