This change enables Flutter to generate multiple Scenes to be rendered into separate FlutterViews from a single widget tree. Each Scene is described by a separate render tree, which are all associated with the single widget tree.
This PR implements the framework-side mechanisms to describe the content to be rendered into multiple views. Separate engine-side changes are necessary to provide these views to the framework and to draw the framework-generated Scene into them.
## Summary of changes
The details of this change are described in [flutter.dev/go/multiple-views](https://flutter.dev/go/multiple-views). Below is a high-level summary organized by layers.
### Rendering layer changes
* The `RendererBinding` no longer owns a single `renderView`. In fact, it doesn't OWN any `RenderView`s at all anymore. Instead, it offers an API (`addRenderView`/`removeRenderView`) to add and remove `RenderView`s that then will be MANAGED by the binding. The `RenderView` itself is now owned by a higher-level abstraction (e.g. the `RawView` Element of the widgets layer, see below), who is also in charge of adding it to the binding. When added, the binding will interact with the `RenderView` to produce a frame (e.g. by calling `compositeFrame` on it) and to perform hit tests for incoming pointer events. Multiple `RenderView`s can be added to the binding (typically one per `FlutterView`) to produce multiple Scenes.
* Instead of owning a single `pipelineOwner`, the `RendererBinding` now owns the root of the `PipelineOwner` tree (exposed as `rootPipelineOwner` on the binding). Each `PipelineOwner` in that tree (except for the root) typically manages its own render tree typically rooted in one of the `RenderView`s mentioned in the previous bullet. During frame production, the binding will instruct each `PipelineOwner` of that tree to flush layout, paint, semantics etc. A higher-level abstraction (e.g. the widgets layer, see below) is in charge of adding `PipelineOwner`s to this tree.
* Backwards compatibility: The old `renderView` and `pipelineOwner` properties of the `RendererBinding` are retained, but marked as deprecated. Care has been taken to keep their original behavior for the deprecation period, i.e. if you just call `runApp`, the render tree bootstrapped by this call is rooted in the deprecated `RendererBinding.renderView` and managed by the deprecated `RendererBinding.pipelineOwner`.
### Widgets layer changes
* The `WidgetsBinding` no longer attaches the widget tree to an existing render tree. Instead, it bootstraps a stand-alone widget tree that is not backed by a render tree. For this, `RenderObjectToWidgetAdapter` has been replaced by `RootWidget`.
* Multiple render trees can be bootstrapped and attached to the widget tree with the help of the `View` widget, which internally is backed by a `RawView` widget. Configured with a `FlutterView` to render into, the `RawView` creates a new `PipelineOwner` and a new `RenderView` for the new render tree. It adds the new `RenderView` to the `RendererBinding` and its `PipelineOwner` to the pipeline owner tree.
* The `View` widget can only appear in certain well-defined locations in the widget tree since it bootstraps a new render tree and does not insert a `RenderObject` into an ancestor. However, almost all Elements expect that their children insert `RenderObject`s, otherwise they will not function properly. To produce a good error message when the `View` widget is used in an illegal location, the `debugMustInsertRenderObjectIntoSlot` method has been added to Element, where a child can ask whether a given slot must insert a RenderObject into its ancestor or not. In practice, the `View` widget can be used as a child of the `RootWidget`, inside the `view` slot of the `ViewAnchor` (see below) and inside a `ViewCollection` (see below). In those locations, the `View` widget may be wrapped in other non-RenderObjectWidgets (e.g. InheritedWidgets).
* The new `ViewAnchor` can be used to create a side-view inside a parent `View`. The `child` of the `ViewAnchor` widget renders into the parent `View` as usual, but the `view` slot can take on another `View` widget, which has access to all inherited widgets above the `ViewAnchor`. Metaphorically speaking, the view is anchored to the location of the `ViewAnchor` in the widget tree.
* The new `ViewCollection` widget allows for multiple sibling views as it takes a list of `View`s as children. It can be used in all the places that accept a `View` widget.
## Google3
As of July 5, 2023 this change passed a TAP global presubmit (TGP) in google3: tap/OCL:544707016:BASE:545809771:1688597935864:e43dd651
## Note to reviewers
This change is big (sorry). I suggest focusing the initial review on the changes inside of `packages/flutter` first. The majority of the changes describe above are implemented in (listed in suggested review order):
* `rendering/binding.dart`
* `widgets/binding.dart`
* `widgets/view.dart`
* `widgets/framework.dart`
All other changes included in the PR are basically the fallout of what's implemented in those files. Also note that a lot of the lines added in this PR are documentation and tests.
I am also very happy to walk reviewers through the code in person or via video call, if that is helpful.
I appreciate any feedback.
## Feedback to address before submitting ("TODO")
## FlutterTimeline
Add a new class `FlutterTimeline` that's a drop-in replacement for `Timeline` from `dart:developer`. In addition to forwarding invocations of `startSync`, `finishSync`, `timeSync`, and `instantSync` to `dart:developer`, provides the following extra methods that make is easy to collect timings for code blocks on a frame-by-frame basis:
* `debugCollect()` - aggregates timings since the last reset, or since the app launched.
* `debugReset()` - forgets all data collected since the previous reset, or since the app launched. This allows clearing data from previous frames so timings can be attributed to the current frame.
* `now` - this was enhanced so that it works on the web by calling `window.performance.now` (in `Timeline` this is a noop in Dart web compilers).
* `collectionEnabled` - a field that controls whether `FlutterTimeline` stores timings in memory. By default this is disabled to avoid unexpected overhead (although the class is designed for minimal and predictable overhead). Specific benchmarks can enable collection to report to Skia Perf.
## Semantics benchmarks
Add `BenchMaterial3Semantics` that benchmarks the cost of semantics when constructing a screen full of Material 3 widgets from nothing. It is expected that semantics will have non-trivial cost in this case, but we should strive to keep it much lower than the rendering cost. This is the case already. This benchmark shows that the cost of semantics is <10%.
Add `BenchMaterial3ScrollSemantics` that benchmarks the cost of scrolling a previously constructed screen full of Material 3 widgets. The expectation should be that semantics will have trivial cost, since we're just shifting some widgets around. As of today, the numbers are not great, with semantics taking >50% of frame time, which is what prompted this PR in the first place. As we optimize this, we want to see this number improve.
* Revert "Remove references to BindingBase.window (#122119)"
This reverts commit c7681f00cf.
* Revert "Remove another reference to BindingBase.window (#122341)"
This reverts commit 6ec4445063.
* Revert "Reland (2): Removes single window assumptions from `flutter_test` (#122233)"
This reverts commit eb3d317ea0.
* Revert "Remove single view assumption from TestViewConfiguration (#122352)"
This reverts commit 927289fb4e.
* Revert "Updates `flutter/test/cupertino` to no longer use `TestWindow` (#122325)"
This reverts commit 67e17e45f0.
* Revert "Updates `flutter/test/gestures` to no longer reference `TestWindow` (#122327)"
This reverts commit c2a5111cc0.
* Revert "Updates `flutter/test/rendering` to no longer use `TestWindow` (#122347)"
This reverts commit 28b65e089b.
* Revert "Updates `flutter_localizations/test` to stop using `TestWindow` (#122321)"
This reverts commit 01367d52d7.
* add asset manifest bin loading and asset manifest api
* use new api for image resolution
* remove upfront smc data casting
* fix typecasting issue
* remove unused import
* fix tests
* lints
* lints
* fix import
* revert image resolution changes
* Update image_resolution_test.dart
* Update decode_and_parse_asset_manifest.dart
* make targetDevicePixelRatio optional
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* fix immutable not being imported
* return List in AssetManifest methods, fix annotation import
* simplify onError callback
* make AssetManifest methods abstract instead of throwing UnimplementedError
* simplify AssetVariant.key docstring
* tweak _AssetManifestBin docstring
* make AssetManifest and AssetVariant doc strings more specific
* use List.of instead of List.from for type-safety
* adjust import
* change _AssetManifestBin comment from doc comment to normal comment
* revert to callback function for onError in loadStructuredBinaryData
* add more to the docstring of AssetManifest.listAssets and AssetVariant.key
* add tests for CachingAssetBundle caching behavior
* add simple test to ensure loadStructuredBinaryData correctly calls load
* Update asset_manifest.dart
* update docstring for AssetManifest.getAssetVariants
* rename getAssetVariants, have it include main asset
* rename isMainAsset field of AssetMetadata to main
* (slightly) shorten name of describeAssetAndVariants
* rename describeAssetVariants back to getAssetVariants
* add tests for TestAssetBundle
* nits
* fix typo in docstring
* remove no longer necessary non-null asserts
* update gallery and google_fonts versions
---------
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* add asset manifest bin loading and asset manifest api
* use new api for image resolution
* remove upfront smc data casting
* fix typecasting issue
* remove unused import
* fix tests
* lints
* lints
* fix import
* revert image resolution changes
* Update image_resolution_test.dart
* Update decode_and_parse_asset_manifest.dart
* make targetDevicePixelRatio optional
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* fix immutable not being imported
* return List in AssetManifest methods, fix annotation import
* simplify onError callback
* make AssetManifest methods abstract instead of throwing UnimplementedError
* simplify AssetVariant.key docstring
* tweak _AssetManifestBin docstring
* make AssetManifest and AssetVariant doc strings more specific
* use List.of instead of List.from for type-safety
* adjust import
* change _AssetManifestBin comment from doc comment to normal comment
* revert to callback function for onError in loadStructuredBinaryData
* add more to the docstring of AssetManifest.listAssets and AssetVariant.key
* add tests for CachingAssetBundle caching behavior
* add simple test to ensure loadStructuredBinaryData correctly calls load
* Update asset_manifest.dart
* update docstring for AssetManifest.getAssetVariants
* rename getAssetVariants, have it include main asset
* rename isMainAsset field of AssetMetadata to main
* (slightly) shorten name of describeAssetAndVariants
* rename describeAssetVariants back to getAssetVariants
* add tests for TestAssetBundle
* nits
* fix typo in docstring
* remove no longer necessary non-null asserts
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Add a microbenchmark that measures performance of flutter compute.
This benchmark should show performance improvements when compute is switched to Isolate.exit
* Fix analyzer warnings, increase warmup iterations
* Fix license header
