The cost of bootstapping the initial PSOs can regress cold startup time
for customer money. As an experiment, attempt to defer PSO construction
to skia like.
---------
Co-authored-by: Aaron Clarke <aaclarke@google.com>
Co-authored-by: gaaclarke <30870216+gaaclarke@users.noreply.github.com>
<!-- start_original_pr_link -->
Reverts: flutter/flutter#165261
<!-- end_original_pr_link -->
<!-- start_initiating_author -->
Initiated by: jonahwilliams
<!-- end_initiating_author -->
<!-- start_revert_reason -->
Reason for reverting: bork the tree
<!-- end_revert_reason -->
<!-- start_original_pr_author -->
Original PR Author: jonahwilliams
<!-- end_original_pr_author -->
<!-- start_reviewers -->
Reviewed By: {gaaclarke}
<!-- end_reviewers -->
<!-- start_revert_body -->
This change reverts the following previous change:
The cost of bootstapping the initial PSOs can regress cold startup time
for customer money. As an experiment, attempt to defer PSO construction
to skia like.
<!-- end_revert_body -->
Co-authored-by: auto-submit[bot] <flutter-engprod-team@google.com>
The cost of bootstapping the initial PSOs can regress cold startup time
for customer money. As an experiment, attempt to defer PSO construction
to skia like.
---------
Co-authored-by: Aaron Clarke <aaclarke@google.com>
Co-authored-by: gaaclarke <30870216+gaaclarke@users.noreply.github.com>
This benchmark is a benchmark of the integration test binding which IMO
is not that important given that we continue to rely on flutter_driver
for actual performance metrics.
Fixes https://github.com/flutter/flutter/issues/162615
Can test 2 different ways.
On a mac (or linux machine) with adb on the path (or android sdk set in
ANDROID_HOME) and an emulator running (or physical device attached) that
is api 30 or higher.
```
cd dev/devicelab
dart bin/test_runner.dart test -t android_display_cutout
```
OR
```
dev/integration_tests/display_cutout_rotation
flutter drive integration_test/display_cutout_test.dart
```
Proof the test ran successfully
```
[2025-02-12 08:08:22.069817] [STDOUT] Removing Synthetic notch...
[2025-02-12 08:08:22.071147] [STDOUT] Executing "/b/s/w/ir/cache/android/sdk/platform-tools/adb -s emulator-5554 shell cmd overlay disable com.android.internal.display.cutout.emulation.tall" in "/b/s/w/ir/x/w/rc/tmpk3k3yhhp/flutter sdk/dev/integration_tests/display_cutout_rotation/" with environment {BOT: true, LANG: en_US.UTF-8}
[2025-02-12 08:08:22.862219] [STDOUT] Checking for reboot
[android_defines_test] Process terminated with exit code 0.
Task result:
{
"success": true,
"data": null,
"detailFiles": [],
"benchmarkScoreKeys": [],
"reason": "success"
}
```
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8723125792202374961/+/u/run_android_defines_test/stdout
All checks passed
https://github.com/flutter/flutter/pull/162641/checks?check_run_id=36991537539
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
Someone could always resurrect it via `git`-magic (or GitHub), but let's
not maintain (including Gradle lock files, Dart analysis, etc) unused
code that hasn't run in 12+ months.
/cc @johnmccutchan
As per https://github.com/flutter/flutter/issues/162620, we are going to
run the web benchmarks at `-O2` for a trial period to evaluate the
performance difference, which will give us some data on whether we can
consider changing to `-O2` by default.
This auto-formats all *.dart files in the repository outside of the
`engine` subdirectory and enforces that these files stay formatted with
a presubmit check.
**Reviewers:** Please carefully review all the commits except for the
one titled "formatted". The "formatted" commit was auto-generated by
running `dev/tools/format.sh -a -f`. The other commits were hand-crafted
to prepare the repo for the formatting change. I recommend reviewing the
commits one-by-one via the "Commits" tab and avoiding Github's "Files
changed" tab as it will likely slow down your browser because of the
size of this PR.
---------
Co-authored-by: Kate Lovett <katelovett@google.com>
Co-authored-by: LongCatIsLooong <31859944+LongCatIsLooong@users.noreply.github.com>
When running `dart format` over these lines the `// ignore` ended up on
a line where it wasn't properly ignoring the lint. This adjusts the
placement of `// ignore`s so they will continue to ignore the right
thing even after the code is auto formatted.
I am hoping that if we do this now the large PR that formats the entire
repo will go in smoother without manual intervention.
Fix the `--ab` option in the benchmark harness:
- Make `--local-engine-host` optional. The web engine doesn't need it, so it doesn't build it. But the tool crashes by failing to find it.
- Disable icon tree shaking because `--ab` runs against local engine build, whose Dart kernel version is out of sync with the framework, which crashes the const finder.
**Three things**
Re-lands #154374
New: fix `platform_channels_benchmarks` to print the "done" key. Updated notes for the microbenchmark parser. There are no other users of `microbenchmarks.readJsonResults`.
Re-Re-land: Uninstall microbenchmarks before running them.
Flakes in https://github.com/flutter/flutter/issues/153828 stem from adb saying the app isn't installed, but then failing to install wtih -r. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.
Potential solution https://github.com/flutter/flutter/issues/153828
Re-land Make things go fast
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Reverts: flutter/flutter#154374
Initiated by: jtmcdole
Reason for reverting: A different benchmark was using the microbenchmark parser and timing out.
Original PR Author: jtmcdole
Reviewed By: {zanderso}
This change reverts the following previous change:
Two things:
**Re-land**: Uninstall microbenchmarks before running them.
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.
Potential solution #153828
**Make things go fast**
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Two things:
**Re-land**: Uninstall microbenchmarks before running them.
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.
Potential solution #153828
**Make things go fast**
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Reverts: flutter/flutter#154183
Initiated by: zanderso
Reason for reverting: microbenchmarks timing out on CI.
Original PR Author: jtmcdole
Reviewed By: {andrewkolos}
This change reverts the following previous change:
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it.
Potential corrects #153828
Flakes in #153828 stem from adb saying the app isn't installed, but then
failing to install wtih `-r`. Several other tests uninstall the app
before trying to run it.
Potential corrects #153828
A recent change to the flutter_drone recipe is setting FLUTTER_TEST_OUTPUTS_DIR in the environment (see https://flutter.googlesource.com/recipes/+/b111cd3ef2297e66905cc48c2cfedce0cf1ba46f)
If FLUTTER_TEST_OUTPUTS_DIR is set, then perf_tests will use an output path based on that variable. perf_tests_test needs to use the same logic when looking for the expected output file.
This test expects to run on 32-bit hardware, but the devicelab no longer
has 32-bit Android hardware. In particular, the test contains hand-coded
32-bit arm assembly for reading the stack pointer from Dart code via
`dart:ffi`.
This test was added in response to a framework change that caused stack
frames to be larger than expected while building widgets, leading apps
to crash with StackOverflow exceptions unexpectedly. Since then (>3
years ago) this test has not prevented any similar issue, so I believe
deleting it rather than fixing it is a better use of resources.
As in https://github.com/flutter/flutter/pull/152696, It's not clear
that DDS initialization is necessary for any of these tests.
In particular, from the comments on the tool flag:
```
'It may be necessary to disable this when attaching to an application with '
'an existing DDS instance (e.g., attaching to an application currently '
'connected to by "flutter run"), or when running certain tests.\n'
'Disabling this feature may degrade IDE functionality if a DDS instance is '
'not already connected to the target application.'
```
These tests are unlikely to need IDE functionality. I have initially
modified the Linux_android_emu tests in the ci.yaml so that they run in
presubmit to verify that this change is safe. Many other postsubmit-only
tests that will now be passed `--no-dds` require physical devices, and
if there is an issue in post-submit, this change will need to be
reverted.
Even though we haven't completed the process of removing the Skia backend, its unlikely that we will target changes to the picture/ayer cache system or investigate changes in these values that don't also result in raster time changes.
Remove to reduce noise in Skia Perf.
Part of https://github.com/flutter/flutter/issues/143532
Reverts: flutter/flutter#152049
Initiated by: cbracken
Reason for reverting: iOS builds failing in post-submit
Original PR Author: loic-sharma
Reviewed By: {jmagman}
This change reverts the following previous change:
Changes:
1. Enables Swift Package Manager by default on the main/master channel
2. Fixes tests that fail if Swift Package Manager is enabled
Corresponding docs change: https://github.com/flutter/website/pull/10938
Addresses https://github.com/flutter/flutter/issues/151567
This is a very common usage of ad so we want to make sure it's performant.
From the video, it scrolls quite smoothly, but we want to see the numbers, and keep monitoring it in case of regression.
https://github.com/flutter/flutter/assets/41930132/c7811c15-ac07-4989-a8a9-3c128e08cbe0
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/150230
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Some performance tests run "flutter create" to generate platform-specific project configuration files for the test app. These files may be further patched by the test script to apply settings such as the Impeller backend.
The manifests need to be deleted and recreated on each run so that settings applied by previous runs will not be used by the current run.
Initial tap is missing sometimes; either its never delivered or it is
delivered before gesture controller is hooked up.
1: Update MemoryTest to have option `requiresTapToStart` guarding the
new paths
2: Update the two perf tests that appear to be flaky to output when
TAPPED is received
3: Update the MemoryTest to keep tapping while waiting for TAPPED
Tested on devicelab:
* setting iterations=1
* removing the timeout before READY
* running tests in a while loop
Before this change, you could get the test to hang often. After this
change you'll see "tapping device... [x]" where x is the counter.
Fixes https://github.com/flutter/flutter/issues/150096
Reverts: flutter/flutter#150287
Initiated by: jtmcdole
Reason for reverting: other memperf tests don't wait for or send a TAPPED; so they fail.
Original PR Author: jtmcdole
Reviewed By: {gaaclarke}
This change reverts the following previous change:
Initial tap is missing sometimes; either its never delivered or it is delivered before gesture controller is hooked up.
1: Update the two perf tests to output when TAPPED is received
2: Update the MemoryTest to keep tapping while waiting for TAPPED
Tested on devicelab:
* setting iterations=1
* removing the timeout before READY
* running tests in a while loop
Before this change, you could get the test to hang often. After this change you'll see "tapping device... [x]" where x is the counter.
Fixes#150096
Initial tap is missing sometimes; either its never delivered or it is delivered before gesture controller is hooked up.
1: Update the two perf tests to output when TAPPED is received
2: Update the MemoryTest to keep tapping while waiting for TAPPED
Tested on devicelab:
* setting iterations=1
* removing the timeout before READY
* running tests in a while loop
Before this change, you could get the test to hang often. After this change you'll see "tapping device... [x]" where x is the counter.
Fixes#150096
