07772a3d23
## FlutterTimeline Add a new class `FlutterTimeline` that's a drop-in replacement for `Timeline` from `dart:developer`. In addition to forwarding invocations of `startSync`, `finishSync`, `timeSync`, and `instantSync` to `dart:developer`, provides the following extra methods that make is easy to collect timings for code blocks on a frame-by-frame basis: * `debugCollect()` - aggregates timings since the last reset, or since the app launched. * `debugReset()` - forgets all data collected since the previous reset, or since the app launched. This allows clearing data from previous frames so timings can be attributed to the current frame. * `now` - this was enhanced so that it works on the web by calling `window.performance.now` (in `Timeline` this is a noop in Dart web compilers). * `collectionEnabled` - a field that controls whether `FlutterTimeline` stores timings in memory. By default this is disabled to avoid unexpected overhead (although the class is designed for minimal and predictable overhead). Specific benchmarks can enable collection to report to Skia Perf. ## Semantics benchmarks Add `BenchMaterial3Semantics` that benchmarks the cost of semantics when constructing a screen full of Material 3 widgets from nothing. It is expected that semantics will have non-trivial cost in this case, but we should strive to keep it much lower than the rendering cost. This is the case already. This benchmark shows that the cost of semantics is <10%. Add `BenchMaterial3ScrollSemantics` that benchmarks the cost of scrolling a previously constructed screen full of Material 3 widgets. The expectation should be that semantics will have trivial cost, since we're just shifting some widgets around. As of today, the numbers are not great, with semantics taking >50% of frame time, which is what prompted this PR in the first place. As we optimize this, we want to see this number improve. |
||
|---|---|---|
| .. | ||
| android | ||
| assets | ||
| ios | ||
| lib | ||
| macos | ||
| test | ||
| test_driver | ||
| test_memory | ||
| web | ||
| .gitignore | ||
| pubspec.yaml | ||
| README.md | ||
Macrobenchmarks
Performance benchmarks use either flutter drive or the web benchmark harness.
Mobile benchmarks
Cull opacity benchmark
To run the cull opacity benchmark on a device:
flutter drive --profile test_driver/cull_opacity_perf.dart
Results should be in the file build/cull_opacity_perf.timeline_summary.json.
More detailed logs should be in build/cull_opacity_perf.timeline.json.
Cubic bezier benchmark
To run the cubic-bezier benchmark on a device:
flutter drive --profile test_driver/cubic_bezier_perf.dart
Results should be in the file build/cubic_bezier_perf.timeline_summary.json.
More detailed logs should be in build/cubic_bezier_perf.timeline.json.
Backdrop filter benchmark
To run the backdrop filter benchmark on a device: To run a mobile benchmark on a device:
flutter drive --profile -t test_driver/run_app.dart --driver test_driver/[test_name]_test.dart
Results should be in the file build/[test_name].timeline_summary.json.
More detailed logs should be in build/[test_name].timeline.json.
The key [test_name] can be:
animated_placeholder_perfbackdrop_filter_perfcolor_filter_and_fade_perfcubic_bezier_perfcull_opacity_perffading_child_animation_perfimagefiltered_transform_animation_perfmulti_widget_construction_perfpicture_cache_perfpost_backdrop_filter_perfsimple_animation_perftextfield_perffullscreen_textfield_perf
E2E benchmarks
(On-going work)
E2E-based tests are driven independent of the host machine. The following tests are E2E:
cull_opacity_perf.dartmulti_widget_construction_perf
These tests should be run by:
flutter drive --profile -t test/[test_name]_e2e.dart --driver test_driver/e2e_test.dart
Web benchmarks
Web benchmarks are compiled from the same entry point in lib/web_benchmarks.dart.
How to write a web benchmark
Create a new file for your benchmark under lib/src/web. See bench_draw_rect.dart
as an example.
Choose one of the two benchmark types:
- A "raw benchmark" records performance metrics from direct interactions with
dart:uiwith no framework. This kind of benchmark is good for benchmarking low-level engine primitives, such as layer, picture, and semantics performance. - A "widget benchmark" records performance metrics using a widget. This kind of benchmark is good for measuring the performance of widgets, often together with engine work that widget-under-test incurs.
- A "widget build benchmark" records the cost of building a widget from nothing. This is different from the "widget benchmark" because typically the latter only performs incremental UI updates, such as an animation. In contrast, this benchmark pumps an empty frame to clear all previously built widgets and rebuilds them from scratch.
For a raw benchmark extend RawRecorder (tip: you can start by copying
bench_draw_rect.dart).
For a widget benchmark extend WidgetRecorder (tip: you can start by copying
bench_simple_lazy_text_scroll.dart).
For a widget build benchmark extend WidgetBuildRecorder (tip: you can start by copying
bench_build_material_checkbox.dart).
Pick a unique benchmark name and class name and add it to the benchmarks list
in lib/web_benchmarks.dart.
How to run a web benchmark
Web benchmarks can be run using flutter run in debug, profile, and release
modes, using either the HTML or the CanvasKit rendering backend. Note, however,
that running in debug mode will result in worse numbers. Profile mode is useful
for profiling in Chrome DevTools because the numbers are close to release mode
and the profile contains unobfuscated names.
Example:
cd dev/benchmarks/macrobenchmarks
# Runs in profile mode using the HTML renderer
flutter run --web-renderer=html --profile -d web-server lib/web_benchmarks.dart
# Runs in profile mode using the CanvasKit renderer
flutter run --web-renderer=canvaskit --profile -d web-server lib/web_benchmarks.dart
You can also run all benchmarks exactly as the devicelab runs them:
cd dev/devicelab
# Runs using the HTML renderer
../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_html.dart
# Runs using the CanvasKit renderer
../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_canvaskit.dart
Frame policy test
File test/frame_policy.dart and its driving script test_driver/frame_policy_test.dart
are used for testing fullyLive
and benchmarkLive
policies in terms of its effect on WidgetTester.handlePointerEventRecord.