flutter/dev/benchmarks/microbenchmarks/lib/layout/text_intrinsic_bench.dart
John McDole b755641559
Address frame policy benchmark flakes (#155130)
Recently the microbenchmarks were flakey, but from an older bug. Turns out, `LiveTestWidgetsFlutterBindingFramePolicy` is defaulted to `fadePointers` with this fun note:

> This can result in additional frames being pumped beyond those that
the test itself requests, which can cause differences in behavior

Both `text_intrinsic_bench` and `build_bench` use a similar pattern:
* Load stocks app
* Open the menu
* Switch to `benchmark` frame policy

What happens, rarely, is that
`LiveTestWidgetsFlutterBinding.pumpBenchmark()` will call (async) `handleBeginFrame` and `handleDrawFrame`. `handleDrawFrame` juggles a tri-state boolean (null, false, true). This boolean is only reset to `null` when handleDrawFrame is called back to back, say, from an extra frame that was scheduled.

1. Switch tri-state boolean to an enum, its easier to read
2. remove asserts that compile away in benchmarks (`--profile`)
3. use `Error.throwWithStackTrace` to keep stack traces.

I've been running this test on device lab hardware for hundreds of runs and have not hit a failure yet.

Fixes #150542
Fixes #150543 - throw stack!
2024-09-12 23:19:15 +00:00

68 lines
2.2 KiB
Dart

// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';
import 'package:flutter_test/flutter_test.dart';
import '../common.dart';
const Duration kBenchmarkTime = Duration(seconds: 15);
// Use an Align to loosen the constraints.
final Widget intrinsicTextHeight = Directionality(
textDirection: TextDirection.ltr,
child: Align(
child: IntrinsicHeight(
child: Text('A' * 100),
),
),
);
Future<void> execute() async {
assert(false, "Don't run benchmarks in debug mode! Use 'flutter run --release'.");
// We control the framePolicy below to prevent us from scheduling frames in
// the engine, so that the engine does not interfere with our timings.
final LiveTestWidgetsFlutterBinding binding = TestWidgetsFlutterBinding.ensureInitialized() as LiveTestWidgetsFlutterBinding;
final Stopwatch watch = Stopwatch();
int iterations = 0;
await benchmarkWidgets((WidgetTester tester) async {
runApp(intrinsicTextHeight);
// Wait for the UI to stabilize.
await tester.pumpAndSettle(const Duration(seconds: 1));
final TestViewConfiguration big = TestViewConfiguration.fromView(
size: const Size(360.0, 640.0),
view: tester.view,
);
final TestViewConfiguration small = TestViewConfiguration.fromView(
size: const Size(100.0, 640.0),
view: tester.view,
);
final RenderView renderView = WidgetsBinding.instance.renderViews.single;
binding.framePolicy = LiveTestWidgetsFlutterBindingFramePolicy.benchmark;
watch.start();
while (watch.elapsed < kBenchmarkTime) {
renderView.configuration = iterations.isEven ? big : small;
await tester.pumpBenchmark(Duration(milliseconds: iterations * 16));
iterations += 1;
}
watch.stop();
});
final BenchmarkResultPrinter printer = BenchmarkResultPrinter();
printer.addResult(
description: 'Text intrinsic height',
value: watch.elapsedMicroseconds / iterations,
unit: 'µs per iteration',
name: 'text_intrinsic_height_iteration',
);
printer.printToStdout();
}