付俊伟：AIGC浪潮下WebNN的演进与实践- 2024.docx

演讲人：付俊伟

目录

01WebNN出现的背景

01WebNN出现的背景

02WebNN的架构设计

04

04WebNN的性能对比

03如何使用WebNN

https://microsoft.github.io/webnn-developer-preview/

WebNN

WebNNExecutionProviderofONNXRuntimeWebwithGPUaccelerationfromDirectML.RunningonIntel?CoreUltra7processor155HwithintegratedArcGPU.

Acatunderthesnow

Acatunder

thesnow

StableDiffusion

i

iImageDecoder

Text

TextEncoder

Unet

Step

1

ImageGeneration

Unet

Step

4

Unet

Step

2

Unet

Step

3

WebNNOperation

matMul

gather

sigmoid

softmax

DirectML

GEMM

GATHER

LOGISTIC

SOFTMAX

TFLite

BATH_MATMUL

GATHER

ACTIVATION_SIGMOID

ACTIVATION_SOFTMAX

CoreML

matmul

gather_along_axis

sigmoid

softmax

运用场景

ObjectDetectionNoise

ObjectDetection

Noise

Suppression

Image

Classification

NaturalLanguage

BackgroundSegmentation

TensorFlow.jsMediaPipeWebONNXRuntime

TensorFlow.js

MediaPipeWeb

ONNXRuntimeWeb

框架

Transformers.js

WebAPI

WebAssembly

WebAssembly

API

APIextensions

WebGPU

WebNN

Web引擎

WebBrowser

(e.g.,Chrome/Edge)

JavaScriptRuntime(e.g.,Electron/Node.js)

系统

MLAPIs

DirectMLTFLiteCoreMLWindowsStudio

DirectML

TFLite

CoreML

WindowsStudioEffects

OtherMLOSAPIs

硬件

CPUNPU

CPU

NPU

GPU

devicetype:cpu/gpu/npu

powerpreference:high-perf/low-power

MLGraphBuilder

MLGraphBuilder

InputBuffers(CPU/GPU)MLGraphcompilecomputeOutputBuffers(CPU/GPU)build↓

InputBuffers(CPU/GPU)

MLGraph

compile

compute

OutputBuffers(CPU/GPU)

inputconv2dreluaddtmptmpbiasComputationalGraph(Web)

input

conv2d

relu

add

tmp

tmp

bias

filter

output

inputoutputCompiledGraph(Native)

input

output

Fused

Fusedconv2d

WebNNAPI二OtherWebAPI

Callflow

Dataflow

MLContext↓create

MLContext

↓create

WebNN

WebNN为Web带来了神经网络的统一抽象

Apps/

Frameworks

WebApplication

JSMLFrameworks

RendererProcess

MLContextMLGraphBuilder

MLContext

MLGraphBuilder

↓

MLGraph

WebNNMojoClient

Chromiu