w4lle's Notes

人生如逆旅,我亦是行人。

w4lle's avatar w4lle

Flutter UI 渲染浅析(七)Composite

系列文章的第七篇,本篇文章继续分析下Composite合成过程。

其实叫合成不太准备,应该叫做提交合成。

前面的文章分析完了flushLayout,继续分析下 RendererBinding.drawFrame() 剩余部分。

1
2
3
4
5
6
7
8
9
10
11
12
13
// lib/src/rendering/binding.dart
@protected
void drawFrame() {
assert(renderView != null);
pipelineOwner.flushLayout();
pipelineOwner.flushCompositingBits();
pipelineOwner.flushPaint();
if (sendFramesToEngine) {
renderView.compositeFrame(); // this sends the bits to the GPU
pipelineOwner.flushSemantics(); // this also sends the semantics to the OS.
_firstFrameSent = true;
}
}

1、compositeFrame 合成阶段

照例还是分为两部分,标脏&数据处理。

1.1、markNeedsAddToScene 标脏

在上篇文章中,LayerTree构建过程中以及stopRecording时,会触发标脏方法markNeedsAddToScene();

1
2
3
4
5
6
7
8
// lib/src/rendering/layer.dart
void markNeedsAddToScene() {
// Already marked. Short-circuit.
if (_needsAddToScene) {
return;
}
_needsAddToScene = true;
}

该方法用来标脏该Layer状态已经发生改变,需要调用addToScene将其发送到Flutter Engine。

1.2、compositeFrame

drawFrame方法接着调用renderView.compositeFrame方法,看下实现。

其中的RenderView对象作为RenderObject的根节点,其layer类型为TransformLayer。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// lib/src/rendering/view.dart RenderView
void compositeFrame() {
//记录 Compositing
Timeline.startSync('Compositing', arguments: timelineArgumentsIndicatingLandmarkEvent);
try {
final ui.SceneBuilder builder = ui.SceneBuilder();
final ui.Scene scene = layer.buildScene(builder);
if (automaticSystemUiAdjustment)
_updateSystemChrome();
_window.render(scene);
//清理
scene.dispose();
} finally {
//结束记录
Timeline.finishSync();
}
}
  • 构建SceneBuilder对象,它的实现在Flutter Engine
  • 依赖Layer和SceneBuilder对象,通过layer.buildScene得到Scene对象
  • 将Scene对象通过window对象发送给Flutter Engine
1
2
3
4
5
6
7
8
9
10
11
ui.Scene buildScene(ui.SceneBuilder builder) {
// 深度优先遍历,更新子树标脏
updateSubtreeNeedsAddToScene();
// 遍历LayerTree,构建 EngineLayerTree
addToScene(builder);
// 清除标脏
_needsAddToScene = false;
//生成 Scene
final ui.Scene scene = builder.build();
return scene;
}

首先深度优先遍历,更新子树标脏,逻辑是当前节点Layer被标脏为 _needsAddToScene,那么其祖先节点都会被标脏为_needsAddToScene

然后调用addToScene(builder) 遍历LayerTree,构建 EngineLayerTree

2、构建Engine LayerTree & Scene

按照深度优先遍历LayerTree,构建出Flutter Engine 层的EngineLayerTree

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
// lib/src/rendering/layer.dart ContainerLayer
@override
void addToScene(ui.SceneBuilder builder, [ Offset layerOffset = Offset.zero ]) {
addChildrenToScene(builder, layerOffset);
}
void addChildrenToScene(ui.SceneBuilder builder, [ Offset childOffset = Offset.zero ]) {
Layer child = firstChild;
while (child != null) {
if (childOffset == Offset.zero) {
//偏移值没变,尝试复用缓存
child._addToSceneWithRetainedRendering(builder);
} else {
//上传Layer到Flutter Engine,生成EngineLayer,需要具体Layer重写该方法
child.addToScene(builder, childOffset);
}
child = child.nextSibling;
}
}
void _addToSceneWithRetainedRendering(ui.SceneBuilder builder) {
if (!_needsAddToScene && _engineLayer != null) {
// _needsAddToScene为false,并且上次生成的_engineLayer不空
// 复用_engineLayer,不再重新上传
builder.addRetained(_engineLayer);
return;
}
addToScene(builder);
_needsAddToScene = false;
}
  • 如果偏移值没变,尝试复用缓存
  • 上传Layer到Flutter Engine,生成EngineLayer,需要具体Layer重写该方法

还是以ClipRectLayer为例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
@override
void addToScene(ui.SceneBuilder builder, [ Offset layerOffset = Offset.zero ]) {
final Rect shiftedClipRect = layerOffset == Offset.zero ? clipRect : clipRect.shift(layerOffset);
//通过SceneBuilder上传ClipLayer,构建出EngineLayer,并把EngineLayer缓存下来
engineLayer = builder.pushClipRect(
shiftedClipRect,
clipBehavior: clipBehavior,
oldLayer: _engineLayer as ui.ClipRectEngineLayer,
);
...
//继续遍历子树
addChildrenToScene(builder, layerOffset);
// 结束裁剪效果
builder.pop();
}

通过SceneBuilder上传ClipLayer,构建出EngineLayer,并把EngineLayer缓存下来,在下次调用addToScene时,尝试复用EngineLayer,通过 builder.addRetained(_engineLayer) 实现复用。

SceneBuilder、Scene、EngineLayer的实现都在Flutter Engine层。

先看下SceneBuilder的实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// lib/ui/compositing/scene_builder.cc
#define FOR_EACH_BINDING(V) \
V(SceneBuilder, pushOffset) \
V(SceneBuilder, pushTransform) \
V(SceneBuilder, pushClipRect) \
V(SceneBuilder, pushClipRRect) \
V(SceneBuilder, pushClipPath) \
V(SceneBuilder, pushOpacity) \
V(SceneBuilder, pushColorFilter) \
V(SceneBuilder, pushImageFilter) \
V(SceneBuilder, pushBackdropFilter) \
V(SceneBuilder, pushShaderMask) \
V(SceneBuilder, pushPhysicalShape) \
V(SceneBuilder, pop) \
V(SceneBuilder, addPlatformView) \
V(SceneBuilder, addRetained) \
V(SceneBuilder, addPicture) \
V(SceneBuilder, addTexture) \
V(SceneBuilder, addPerformanceOverlay) \
V(SceneBuilder, setRasterizerTracingThreshold) \
V(SceneBuilder, setCheckerboardOffscreenLayers) \
V(SceneBuilder, setCheckerboardRasterCacheImages) \
V(SceneBuilder, build)
//通过ffi绑定Dart方法
FOR_EACH_BINDING(DART_NATIVE_CALLBACK)
SceneBuilder::SceneBuilder() {
// 添加一个根节点
PushLayer(std::make_shared<flutter::ContainerLayer>());
}

通过ffi绑定Dart方法,并且添加一个根节点。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
// lib/ui/compositing/scene_builder.cc
void SceneBuilder::pushClipRect(Dart_Handle layer_handle,
double left,
double right,
double top,
double bottom,
int clipBehavior) {
//构造SkRect对象
SkRect clipRect = SkRect::MakeLTRB(left, top, right, bottom);
//裁剪类型
flutter::Clip clip_behavior = static_cast<flutter::Clip>(clipBehavior);
// 构造ClipRectLayer
auto layer =
std::make_shared<flutter::ClipRectLayer>(clipRect, clip_behavior);
//
PushLayer(layer);
//包装成EngineLayer,返回给Dart缓存
EngineLayer::MakeRetained(layer_handle, layer);
}
void SceneBuilder::PushLayer(std::shared_ptr<ContainerLayer> layer) {
AddLayer(layer);
//vector 队列维护layer层级,用于生成LayerTree
layer_stack_.push_back(std::move(layer));
}
void SceneBuilder::AddLayer(std::shared_ptr<Layer> layer) {
if (!layer_stack_.empty()) {
//拿到最后一个ContainerLayer引用,将layer添加到它的后面,形成LayerTree
layer_stack_.back()->Add(std::move(layer));
}
}
void SceneBuilder::PopLayer() {
if (layer_stack_.size() > 1) {
//当前layer的子树全部添加到LayerTree中,出队列
layer_stack_.pop_back();
}
}
void SceneBuilder::build(Dart_Handle scene_handle) {]
//通过LayerTree,构造Scene对象,返回给Dart
Scene::create(scene_handle, layer_stack_[0], rasterizer_tracing_threshold_,
checkerboard_raster_cache_images_,
checkerboard_offscreen_layers_);
ClearDartWrapper();
}

Dart Framework中的Layer 类型和C++ Engine中的flow::Layer类型是一一对应的,在这里做了从Dart Framework LayerTree到C++ Engine LayerTree 的转换。

通过Scene,将Dart Framework中的LayerTree依次遍历,映射生成C++ Engine Layer,并构建Engine LayerTree,存储在Scene对象中。

其中通过vector队列维护layer层级,来达到Engine LayerTree与Framework LayerTree层级一一对应。

Flow模块是一个基于Skia的简单合成器,运行在Raster(GPU)线程,并向Skia上传绘制指令信息。这里不展开。

3、Window.render

接着调用_window.render(scene),实现在Flutter Engine,通过Window、Engine,最终调用到Animator::Render

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// flutter/shell/common/animator.cc
void Animator::Render(std::unique_ptr<flutter::LayerTree> layer_tree) {
last_layer_tree_size_ = layer_tree->frame_size();
if (layer_tree) {
// 开始时间、目标时间记录在LayerTree
// LayerTree是Scene中的对象,包含Engine LayerTree
layer_tree->RecordBuildTime(last_frame_begin_time_,
last_frame_target_time_);
}
// 当前任务完成,LayerTree结果写入LayerTreePipeline,
bool result = producer_continuation_.Complete(std::move(layer_tree));
//Raster线程,光栅化和合成
delegate_.OnAnimatorDraw(layer_tree_pipeline_, last_frame_target_time_);
}
  • 记录build时间,CPU 过程从VYSNC触发的时间点,即doFrame的触发时间点开始,到Animator::Render结束
  • 当前任务完成,LayerTree结果写入LayerTreePipeline,继续接收ScheduleFrame -> Animator::RequestFrame() 触发开始绘制,具体细节参考《Flutter UI 渲染浅析(二)VSync 注册》
  • 转到Raster线程,进行光栅化和合成操作

LayerTreePipeline是Engine层的渲染管线,用来调度渲染任务。

LayerTreePipeline构建深度为 2 的 Pipeline 对象,可以持有 flutter::LayerTree 对象。

Pipeline 持有两个信号量 empty_available_,用来控制管线任务调度。

整个 Pipeline 管线的流程为:

  • Animator::RequestFrame()方法触发绘制开始_empty -1,开始生成 flutter::LayerTree,运行在 UIThread
  • 当 UIThread 准备好 Engine flutter::LayerTreeavailable_ +1
  • available_ > 0 时,触发 Raster Thread 工作,拿到 flutter:LayerTree 进行光栅化合成
  • 当 RasterThread 处理完成, _empty + 1,下次 Animator::RequestFrame() 可以正常开始处理生成 flutter::LayerTree 工作
  • _empty 为 0 时,管线任务已满,忽略本次 Animator::RequestFrame() 请求,直到下一次 VSync 信号到来

通过两个信号量来管理管线的调度,这种调度机制可以确保 RasterThread 不至于过载(2个任务),同时也可以避免 UIThread 不必要的资源消耗。

所以,也就明白了文章开头提到的为什么叫提交合成,而不叫合成。Composite 只是将Dart Framework LayerTree通过C++ Engine,映射成 Engine LayerTree,提交给Raster Thread处理,而并非真正的合成

然后执行Semantic语义更新,pipelineOwnerflushSemantics(),跟渲染关系不大,这里略过。

到这里 RendererBinding.drawFrame()方法全部执行完成,接下来转到Raster线程,进行光栅化和合成上屏的操作。

Raster线程也是跑在CPU上的,为了避免歧义,官方将其名称从 GPU Thread 更改为了 Raster Thread。

执行完 RendererBinding.drawFrame()方法后,继续回到 WidgetsBinding.drawFrame() 中。详见《Flutter UI 渲染浅析(四)Build》

最后执行WidgetsBinding.drawFrame()方法中的buildOwner.finalizeTree()方法,卸载未激活状态的 Element 节点(未激活状态的节点在一个绘制帧周期内,是有可能被重新激活的——在Element.rebuild阶段,如果没有重新激活,那么就卸载掉)。

4、总结

至此,_persistentCallbacks 回调执行结束,整个绘制流程也就结束了。

本系列文章从Flutter App启动作为入口,了解了7 个主要的 Binding 类及其作用。

从Widget setState()为切入点,分析了Dart Framework和C++ Engine之间的交互过程。

从ScheduleFrame触发绘制注册Vsync信号,到Vsync信号到来回调Dart Framework,进而触发Animate、Build、Layout、Paint一系列绘制操作。

最终在Dart Framework生成含有层级关系以及绘制指令的Layer Tree,将其映射到C++ Engine生成Engine LayerTree。

最后提交给Raster 线程合成上屏。

整个过程由Engine LayerTreePipeline渲染管线调度管控,周而复始地将Widget内容绘制到屏幕上。

本系列文章事无巨细的分析了上述的关键流程及其实现代码,涵盖Dart Framework和C++ Engine。

总结下来,Dart Framework中的Widget、Element、RenderObject、Binding这些东西的存在,都是为了最后生成Layer Tree用,如果没有这些,能不能直接进行绘制呢?

当然是可以的,官方就提供了这样的demo

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
import 'dart:math' as math;
import 'dart:typed_data';
import 'dart:ui' as ui;
ui.Picture paint(ui.Rect paintBounds) {
// First we create a PictureRecorder to record the commands we're going to
// feed in the canvas. The PictureRecorder will eventually produce a Picture,
// which is an immutable record of those commands.
final ui.PictureRecorder recorder = ui.PictureRecorder();
// Next, we create a canvas from the recorder. The canvas is an interface
// which can receive drawing commands. The canvas interface is modeled after
// the SkCanvas interface from Skia. The paintBounds establishes a "cull rect"
// for the canvas, which lets the implementation discard any commands that
// are entirely outside this rectangle.
final ui.Canvas canvas = ui.Canvas(recorder, paintBounds);
final ui.Paint paint = ui.Paint();
canvas.drawPaint(ui.Paint()..color = const ui.Color(0xFFFFFFFF));
final ui.Size size = paintBounds.size;
final ui.Offset mid = size.center(ui.Offset.zero);
final double radius = size.shortestSide / 2.0;
final double devicePixelRatio = ui.window.devicePixelRatio;
final ui.Size logicalSize = ui.window.physicalSize / devicePixelRatio;
// Saves a copy of current transform onto the save stack
canvas.save();
// Note that transforms that occur after this point apply only to the
// yellow-bluish rectangle
// This line will cause the transform to shift entirely outside the paint
// boundaries, which will cause the canvas interface to discard its
// commands. Comment it out to see it on screen.
canvas.translate(-mid.dx / 2.0, logicalSize.height * 2.0);
// Clips the current transform
canvas.clipRect(
ui.Rect.fromLTRB(0, radius + 50, logicalSize.width, logicalSize.height),
clipOp: ui.ClipOp.difference,
);
// Shifts the coordinate space of and rotates the current transform
canvas.translate(mid.dx, mid.dy);
canvas.rotate(math.pi/4);
final ui.Gradient yellowBlue = ui.Gradient.linear(
ui.Offset(-radius, -radius),
const ui.Offset(0.0, 0.0),
<ui.Color>[const ui.Color(0xFFFFFF00), const ui.Color(0xFF0000FF)],
);
// Draws a yellow-bluish rectangle
canvas.drawRect(
ui.Rect.fromLTRB(-radius, -radius, radius, radius),
ui.Paint()..shader = yellowBlue,
);
// Note that transforms that occur after this point apply only to the
// yellow circle
// Scale x and y by 0.5.
final Float64List scaleMatrix = Float64List.fromList(<double>[
0.5, 0.0, 0.0, 0.0,
0.0, 0.5, 0.0, 0.0,
0.0, 0.0, 1.0, 0.0,
0.0, 0.0, 0.0, 1.0,
]);
canvas.transform(scaleMatrix);
// Sets paint to transparent yellow
paint.color = const ui.Color.fromARGB(128, 0, 255, 0);
// Draws a transparent yellow circle
canvas.drawCircle(ui.Offset.zero, radius, paint);
// Restores the transform from before `save` was called
canvas.restore();
// Sets paint to transparent red
paint.color = const ui.Color.fromARGB(128, 255, 0, 0);
// Note that this circle is drawn on top of the previous layer that contains
// the rectangle and smaller circle
canvas.drawCircle(const ui.Offset(150.0, 300.0), radius, paint);
// When we're done issuing painting commands, we end the recording an receive
// a Picture, which is an immutable record of the commands we've issued. You
// can draw a Picture into another canvas or include it as part of a
// composited scene.
return recorder.endRecording();
}
ui.Scene composite(ui.Picture picture, ui.Rect paintBounds) {
final double devicePixelRatio = ui.window.devicePixelRatio;
final Float64List deviceTransform = Float64List(16)
..[0] = devicePixelRatio
..[5] = devicePixelRatio
..[10] = 1.0
..[15] = 1.0;
final ui.SceneBuilder sceneBuilder = ui.SceneBuilder()
..pushTransform(deviceTransform)
..addPicture(ui.Offset.zero, picture)
..pop();
return sceneBuilder.build();
}
void beginFrame(Duration timeStamp) {
final ui.Rect paintBounds = ui.Offset.zero & (ui.window.physicalSize / ui.window.devicePixelRatio);
final ui.Picture picture = paint(paintBounds);
final ui.Scene scene = composite(picture, paintBounds);
ui.window.render(scene);
}
void main() {
ui.window.onBeginFrame = beginFrame;
ui.window.scheduleFrame();
}

上述代码,直接在dart:ui.window对象中注册了onBeginFrame() 回调,抛弃了Dart Framework中Widget、Element、RenderObject、Binding这些东西,直接构建Canvas对象进行绘制,绘制完成后生成Layer Tree,提交给C++ Engine。

代码运行结果可以在这里看到。

Dart Framework的将这些绘制操作进行封装,可以达到更高的开发效率和渲染性能。

通过Canvas&Scene,开发者不需要关系LayerTree如何提交给Engine;

通过LayerTree&Engine LayerTree,在Composite阶段可以进行缓存复用,没变化的Layer层级可以不用多次提交;

通过RenderTree,开发者可以自由扩展控件的布局和绘制;

通过ElementTree,开发者可以不需要关心如何刷新界面,数据可以直接驱动UI刷新,这一层也是实现hot reload的基础;

通过WidgetTree,开发者可以开发更现代化的声明式UI应用。

最后引用本系列文章开篇中的一张图做个总结。

本文链接: http://w4lle.com/2021/02/02/flutter-ui-composite/

版权声明:本文为 w4lle 原创文章,可以随意转载,但必须在明确位置注明出处!
本文链接: http://w4lle.com/2021/02/02/flutter-ui-composite/