Flakes in #153828 stem from adb saying the app isn't installed, but then
failing to install wtih `-r`. Several other tests uninstall the app
before trying to run it.
Potential corrects #153828
A recent change to the flutter_drone recipe is setting FLUTTER_TEST_OUTPUTS_DIR in the environment (see https://flutter.googlesource.com/recipes/+/b111cd3ef2297e66905cc48c2cfedce0cf1ba46f)
If FLUTTER_TEST_OUTPUTS_DIR is set, then perf_tests will use an output path based on that variable. perf_tests_test needs to use the same logic when looking for the expected output file.
This test expects to run on 32-bit hardware, but the devicelab no longer
has 32-bit Android hardware. In particular, the test contains hand-coded
32-bit arm assembly for reading the stack pointer from Dart code via
`dart:ffi`.
This test was added in response to a framework change that caused stack
frames to be larger than expected while building widgets, leading apps
to crash with StackOverflow exceptions unexpectedly. Since then (>3
years ago) this test has not prevented any similar issue, so I believe
deleting it rather than fixing it is a better use of resources.
As in https://github.com/flutter/flutter/pull/152696, It's not clear
that DDS initialization is necessary for any of these tests.
In particular, from the comments on the tool flag:
```
'It may be necessary to disable this when attaching to an application with '
'an existing DDS instance (e.g., attaching to an application currently '
'connected to by "flutter run"), or when running certain tests.\n'
'Disabling this feature may degrade IDE functionality if a DDS instance is '
'not already connected to the target application.'
```
These tests are unlikely to need IDE functionality. I have initially
modified the Linux_android_emu tests in the ci.yaml so that they run in
presubmit to verify that this change is safe. Many other postsubmit-only
tests that will now be passed `--no-dds` require physical devices, and
if there is an issue in post-submit, this change will need to be
reverted.
Even though we haven't completed the process of removing the Skia backend, its unlikely that we will target changes to the picture/ayer cache system or investigate changes in these values that don't also result in raster time changes.
Remove to reduce noise in Skia Perf.
Part of https://github.com/flutter/flutter/issues/143532
Reverts: flutter/flutter#152049
Initiated by: cbracken
Reason for reverting: iOS builds failing in post-submit
Original PR Author: loic-sharma
Reviewed By: {jmagman}
This change reverts the following previous change:
Changes:
1. Enables Swift Package Manager by default on the main/master channel
2. Fixes tests that fail if Swift Package Manager is enabled
Corresponding docs change: https://github.com/flutter/website/pull/10938
Addresses https://github.com/flutter/flutter/issues/151567
This is a very common usage of ad so we want to make sure it's performant.
From the video, it scrolls quite smoothly, but we want to see the numbers, and keep monitoring it in case of regression.
https://github.com/flutter/flutter/assets/41930132/c7811c15-ac07-4989-a8a9-3c128e08cbe0
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/150230
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Some performance tests run "flutter create" to generate platform-specific project configuration files for the test app. These files may be further patched by the test script to apply settings such as the Impeller backend.
The manifests need to be deleted and recreated on each run so that settings applied by previous runs will not be used by the current run.
Initial tap is missing sometimes; either its never delivered or it is
delivered before gesture controller is hooked up.
1: Update MemoryTest to have option `requiresTapToStart` guarding the
new paths
2: Update the two perf tests that appear to be flaky to output when
TAPPED is received
3: Update the MemoryTest to keep tapping while waiting for TAPPED
Tested on devicelab:
* setting iterations=1
* removing the timeout before READY
* running tests in a while loop
Before this change, you could get the test to hang often. After this
change you'll see "tapping device... [x]" where x is the counter.
Fixes https://github.com/flutter/flutter/issues/150096
Reverts: flutter/flutter#150287
Initiated by: jtmcdole
Reason for reverting: other memperf tests don't wait for or send a TAPPED; so they fail.
Original PR Author: jtmcdole
Reviewed By: {gaaclarke}
This change reverts the following previous change:
Initial tap is missing sometimes; either its never delivered or it is delivered before gesture controller is hooked up.
1: Update the two perf tests to output when TAPPED is received
2: Update the MemoryTest to keep tapping while waiting for TAPPED
Tested on devicelab:
* setting iterations=1
* removing the timeout before READY
* running tests in a while loop
Before this change, you could get the test to hang often. After this change you'll see "tapping device... [x]" where x is the counter.
Fixes#150096
Initial tap is missing sometimes; either its never delivered or it is delivered before gesture controller is hooked up.
1: Update the two perf tests to output when TAPPED is received
2: Update the MemoryTest to keep tapping while waiting for TAPPED
Tested on devicelab:
* setting iterations=1
* removing the timeout before READY
* running tests in a while loop
Before this change, you could get the test to hang often. After this change you'll see "tapping device... [x]" where x is the counter.
Fixes#150096
This benchmark illustrates the impact that const'ness has on widget build times by building a widget tree with all const-widgets and then building the same widget tree without any const'ness. Local testing with this benchmark running on a Moto G4 shows that const'ness is 13-16% faster.
I'd like to check this benchmark in because the question of how important widget const'ness is comes up every so often. With this benchmark we have up-to-date data to point people to.
Reverts: flutter/flutter#146931
Initiated by: Hixie
Reason for reverting: more failures
Original PR Author: Hixie
Reviewed By: {reidbaker}
This change reverts the following previous change:
Fixes#137555.
This is an updated version of https://github.com/flutter/flutter/pull/146856, which was reverted in https://github.com/flutter/flutter/pull/146927. The first commit is identical to the original PR, and subsequent commits are the fixes to address failures detected in devicelab post-commit.
This pull request aims for improved readability, based on issue #146600.
```dart
// before
Set<Color> _distinctVisibleColors() {
final Set<Color> distinctVisibleColors = <Color>{};
if (top.style != BorderStyle.none) {
distinctVisibleColors.add(top.color);
}
if (right.style != BorderStyle.none) {
distinctVisibleColors.add(right.color);
}
if (bottom.style != BorderStyle.none) {
distinctVisibleColors.add(bottom.color);
}
if (left.style != BorderStyle.none) {
distinctVisibleColors.add(left.color);
}
return distinctVisibleColors;
}
// after
Set<Color> _distinctVisibleColors() {
return <Color>{
if (top.style != BorderStyle.none) top.color,
if (right.style != BorderStyle.none) right.color,
if (bottom.style != BorderStyle.none) bottom.color,
if (left.style != BorderStyle.none) left.color,
};
}
```
Most of the repo should be covered in this PR (aside from `flutter_tools/`, since there was a lot going on in there).
Reverts: flutter/flutter#145224
Initiated by: hellohuanlin
Reason for reverting: breaks the tree
Original PR Author: hellohuanlin
Reviewed By: {gmackall, jmagman}
This change reverts the following previous change:
Reland https://github.com/flutter/flutter/pull/144745, which got reverted due to Android lockfile. Fixed by `dart dev/tools/bin/generate_gradle_lockfiles.dart`
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/143534
Fixes https://github.com/flutter/flutter/issues/143257
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Add the missing `Directionality` widget and `await test.pump()` call: commit: 0fd7712fa7
Without the `pump` 1s, it sometimes schedules multiple `handleBeginFrame`s and `handleDrawFrame`s.
For https://github.com/flutter/flutter/pull/144577. There's no promise that the performance will be great when `IntrinsicHeight/IntrinsicWidth` is used extensively but it's not that uncommon of a widget.
Reverts flutter/flutter#143244
Initiated by: vashworth
Reason for reverting: Increased `flutter_framework_uncompressed_bytes` - see https://github.com/flutter/flutter/issues/144251
Original PR Author: vashworth
Reviewed By: {jmagman}
This change reverts the following previous change:
Original Description:
Replace `FlutterMacOS.framework` cached artifact with `FlutterMacOS.xcframework`. Also, update usage of `FlutterMacOS.framework` to use `FlutterMacOS.xcframework`.
Part of https://github.com/flutter/flutter/issues/126016.
Replace `FlutterMacOS.framework` cached artifact with `FlutterMacOS.xcframework`. Also, update usage of `FlutterMacOS.framework` to use `FlutterMacOS.xcframework`.
Part of https://github.com/flutter/flutter/issues/126016.
This pull request fixes#143803 by taking advantage of Dart's null-aware operators.
And unlike `switch` expressions ([9 PRs](https://github.com/flutter/flutter/pull/143634) and counting), the Flutter codebase is already fantastic when it comes to null-aware coding. After refactoring the entire repo, all the changes involving `?.` and `??` can fit into a single pull request.
Update: Accidentally use `--O4` instead of `-O4` in `dev/devicelab/lib/tasks/web_benchmarks.dart` update.
Original description:
* Make `flutter build web` have one option that determins the
optimization level: `-O<level>` / `--optimization-level=<level>` =>
Defaulting to -O4 => Will apply to both dart2js and dart2wasm
* Deprecate `--dart2js-optimization=O<level>`
* Disentagle concept of optimization from concept of static symbols =>
Add a `--strip-wasm` / `--no-strip-wasm` flag that determins whether
static symbols are kept in the resulting wasm file.
* Remove copy&past'ed code in the tests for wasm build tests
* Cleanup some artifacts code, now that we no longer use `wasm-opt`
inside flutter tools
Reverts flutter/flutter#143517
Initiated by: dnfield
Reason for reverting: broke CI, see https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_benchmarks_skwasm/3446/overview
Original PR Author: mkustermann
Reviewed By: {eyebrowsoffire}
This change reverts the following previous change:
Original Description:
* Make `flutter build web` have one option that determins the optimization level: `-O<level>` / `--optimization-level=<level>` => Defaulting to -O4 => Will apply to both dart2js and dart2wasm
* Deprecate `--dart2js-optimization=O<level>`
* Disentagle concept of optimization from concept of static symbols => Add a `--strip-wasm` / `--no-strip-wasm` flag that determins whether static symbols are kept in the resulting wasm file.
* Remove copy&past'ed code in the tests for wasm build tests
* Cleanup some artifacts code, now that we no longer use `wasm-opt` inside flutter tools
* Make `flutter build web` have one option that determins the
optimization level: `-O<level>` / `--optimization-level=<level>` =>
Defaulting to -O4 => Will apply to both dart2js and dart2wasm
* Deprecate `--dart2js-optimization=O<level>`
* Disentagle concept of optimization from concept of static symbols =>
Add a `--strip-wasm` / `--no-strip-wasm` flag that determins whether
static symbols are kept in the resulting wasm file.
* Remove copy&past'ed code in the tests for wasm build tests
* Cleanup some artifacts code, now that we no longer use `wasm-opt`
inside flutter tools
This is an attempt at a reland of https://github.com/flutter/flutter/pull/141396
The main changes here that are different than the original PR is fixes to wire up the `flutter test` command properly with the web renderer.
Dual Web Compile has had some issues where `flutter test` is not respecting the `--web-renderer` flag for some reason. I haven't gotten entirely to the bottom of the issue, but for now we need to rever these changes while I investigate. This reverts the following PRs:
https://github.com/flutter/flutter/pull/143128https://github.com/flutter/flutter/pull/141396
While doing this revert, I had a few merge conflicts with https://github.com/flutter/flutter/pull/142760, and I tried to resolve the merge conflicts within the spirit of that PR's change, but @chingjun I might need your input on whether the imports I have modified are okay with regards to the change you were making.
This implements dual compile via the newly available flutter.js bootstrapping APIs for intelligent build fallback.
* Users can now use the `FlutterLoader.load` API from flutter.js
* Flutter tool injects build info into the `index.html` of the user so that the bootstrapper knows which build variants are available to bootstrap
* The semantics of the `--wasm` flag for `flutter build web` have changed:
- Instead of producing a separate `build/web_wasm` directory, the output goes to the `build/web` directory like a normal web build
- Produces a dual build that contains two build variants: dart2wasm+skwasm and dart2js+CanvasKit. The dart2wasm+skwasm will only work on Chrome in a cross-origin isolated context, all other environments will fall back to dart2js+CanvasKit.
- `--wasm` and `--web-renderer` are now mutually exclusive. Since there are multiple build variants with `--wasm`, the web renderer cannot be expressed via a single command-line flag. For now, we are hard coding what build variants are produced with the `--wasm` flag, but I plan on making this more customizable in the future.
* Build targets now can optionally provide a "build key" which can uniquely identify any specific parameterization of that build target. This way, the build target can invalidate itself by changing its build key. This works a bit better than just stuffing everything into the environment defines because (a) it doesn't invalidate the entire build, just the targets which are affected and (b) settings for multiple build variants don't translate well to the flat map of environment defines.
It's now possible to natively compile a flutter app for windows-arm64. Cross-compilation is not yet implemented.
Uses arm64 artifacts now available for Dart/Flutter. Platform detection is based on Abi class, provided by Dart. Depending if Dart is an arm64 or x64 binary, the Abi is set accordingly. Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking PROCESSOR_ARCHITECTURE environment variable, which is the way to detect host architecture on Windows.
This is available only for master channel (on other channels, it fallbacks to windows-x64).
On windows-x64, it produces an x64 app. On windows-arm64, it produces an arm64 app.
Work towards https://github.com/flutter/flutter/issues/142178.
---
This PR makes no _behavioral_ changes to executed code, and instead
focuses on organization and naming:
1. Extended the README to explain the intent of the test, as well as how
to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart`
and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is
being asserted (i.e. `widgetBuilds` and friends)
Reverts flutter/flutter#142062
Initiated by: eliasyishak
This change reverts the following previous change:
Original Description:
This PR makes no _behavioral_ changes to executed code, and instead focuses on organization and naming:
1. Almost[^1] anything named `external_ui` is renamed `external_textures`
1. Extended the README to explain the intent of the test, as well as how to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart` and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is being asserted (i.e. `widgetBuilds` and friends)
Given how complex (and in-flux) this directory is, I'm also requesting either John, Jonah or I review any changes.
[^1]: Except the name of the `.ci.yaml` task, i.e. `name: Linux_pixel_7pro external_ui_integration_test` because I'm apparently not able to change that without creating a new task as `bringup: true` and playing a bit of a dance. Maybe that's worth doing though (in future PRs)?
This PR makes no _behavioral_ changes to executed code, and instead
focuses on organization and naming:
1. Almost[^1] anything named `external_ui` is renamed
`external_textures`
1. Extended the README to explain the intent of the test, as well as how
to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart`
and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is
being asserted (i.e. `widgetBuilds` and friends)
Given how complex (and in-flux) this directory is, I'm also requesting
either John, Jonah or I review any changes.
[^1]: Except the name of the `.ci.yaml` task, i.e. `name:
Linux_pixel_7pro external_ui_integration_test` because I'm apparently
not able to change that without creating a new task as `bringup: true`
and playing a bit of a dance. Maybe that's worth doing though (in future
PRs)?
Reverts flutter/flutter#137618
Initiated by: Jasguerrero
This change reverts the following previous change:
Original Description:
It's now possible to natively compile a flutter app for
windows-arm64. Cross-compilation is not yet implemented.
Uses arm64 artifacts now available for Dart/Flutter.
Platform detection is based on Abi class, provided by Dart. Depending if
Dart is an arm64 or x64 binary, the Abi is set accordingly.
Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking
PROCESSOR_ARCHITECTURE environment variable, which is the way to detect
host architecture on Windows.
This is available only for master channel (on other channels, it
fallbacks to windows-x64).
On windows-x64, it produces an x64 app. On windows-arm64, it produces an
arm64 app.
It's now possible to natively compile a flutter app for
windows-arm64. Cross-compilation is not yet implemented.
Uses arm64 artifacts now available for Dart/Flutter.
Platform detection is based on Abi class, provided by Dart. Depending if
Dart is an arm64 or x64 binary, the Abi is set accordingly.
Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking
PROCESSOR_ARCHITECTURE environment variable, which is the way to detect
host architecture on Windows.
This is available only for master channel (on other channels, it
fallbacks to windows-x64).
On windows-x64, it produces an x64 app. On windows-arm64, it produces an
arm64 app.
Currently podhelper.rb will always point plugin builds at the cached engine artifacts, even when using `--local-engine`. In most cases this is fine, since when the final build actually runs it will be using the engine bundled into the app build, which will be the correct local engine build. When trying to test a local engine build with API additions against a local plugin modified to use those additions to ensure that they are working as expected, however, compilation will fail, because the new APIs won't be present in the plugin build.
This fixes that for macOS, and adds a TODO for iOS (which is more complicated to fix due to the host vs target build distinction).
macOS portion of https://github.com/flutter/flutter/issues/132228
ObjC->Swift plugin migration caused a size regression in the gallery app because the Swift runtime was also pulled in.
The gallery app minimum target version is iOS 11.0, which predates Swift ABI compatibility. Pre iOS 12.2 apps embedded the Swift runtime since there wasn't one available to use in the OS.
Add `FLUTTER_XCODE_IPHONEOS_DEPLOYMENT_TARGET` to the compile perf test environment, which gets translated by the tool to an Xcode build setting:
```
[2023-12-14 15:52:14.797318] [STDOUT] stdout: IPHONEOS_DEPLOYMENT_TARGET = 12.2
```
On my machine on main
```
"release_size_bytes": 43717389,
```
becomes
```
"release_size_bytes": 40679432,
```
Fixes https://github.com/flutter/flutter/issues/139605
I previously made a PR (#136140) that used `switch` expressions to make some parts of the Flutter codebase easier to understand. It was assigned to the framework team, and @christopherfujino let me know that it was too large to effectively review and recommended breaking it up into smaller pull requests.
Here's a PR that only targets files in the `dev/` directory. Hopefully this will be easier to work with!
(solves issue https://github.com/flutter/flutter/issues/136139)
Partial repaint is too effective, and we'd like to be able to measure performance without carefully structuring the benchmarks. For example, right now partial repaint is culling any blurs in the multibackdrop case, which we should be using to track https://github.com/flutter/flutter/issues/132735
Pin the dependencies from dart-lang/native to a specific version during testing (rather than having them auto-upgrade during pub resolution). This will prevent tests using the template to start failing if a bad version is published to pub.
Closes: https://github.com/flutter/flutter/issues/137418
Also bumps dep in flutter_tools.
Adds the metadata key required to enable OpenGLES GPU tracing. This is off by default because the API crashes on some GPU models, but it should be safe on the Pixel 7 (others TBD based on testing results).
Reverts flutter/flutter#136562
Initiated by: vashworth
This change reverts the following previous change:
Original Description:
Some of our tests in CI are triggering the `NSLocalNetworkUsageDescription` dialog when they're not supposed to (https://github.com/flutter/flutter/issues/129836) since it's disabled via flags (`--no-publish-port` for flutter/flutter and `--disable-vm-service-publication` for flutter/engine).
Normally, we inject `NSLocalNetworkUsageDescription` (and other bonjour settings) to the Info.plist during the project build for debug and profile mode since by default they will publish the VM Service port over mDNS.
To help diagnose the issue, though, this PR changes it so that we don't inject `NSLocalNetworkUsageDescription` (and other bonjour settings) when port publication is disabled since it shouldn't be needed. Hopefully, this will give us better error messages or cause the app to crash and end the test early (rather than timeout after 30 minutes).
Some of our tests in CI are triggering the `NSLocalNetworkUsageDescription` dialog when they're not supposed to (https://github.com/flutter/flutter/issues/129836) since it's disabled via flags (`--no-publish-port` for flutter/flutter and `--disable-vm-service-publication` for flutter/engine).
Normally, we inject `NSLocalNetworkUsageDescription` (and other bonjour settings) to the Info.plist during the project build for debug and profile mode since by default they will publish the VM Service port over mDNS.
To help diagnose the issue, though, this PR changes it so that we don't inject `NSLocalNetworkUsageDescription` (and other bonjour settings) when port publication is disabled since it shouldn't be needed. Hopefully, this will give us better error messages or cause the app to crash and end the test early (rather than timeout after 30 minutes).
These values are generated since https://github.com/flutter/flutter/pull/135279, but I didn't know to add the new keys to this list to get them to upload.
Failed to do so in #135645, I believe the mistake there was putting them in `_kCommonScoreKeys`, which is also used in "E2E" tests, that don't get full trace data, only high level `FrameTiming` packets.
Part of https://github.com/flutter/flutter/issues/129150
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/121420
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
