We're adding macrobenchmark support to DDC (AMD modules and our new
hot-reload-capable module system) to be aware of any current/future
performance regressions. I'm not terribly aware of the metrics
collection pipeline; please let me know if we need to do any more work
to see these tests run/report numbers to the proper dashboards.
We currently benchmark path tessellation on Android and iOS, but only
for filled paths. We will now run those same benchmarks also with
stroking in a new set of benchmarks.
The cost of bootstapping the initial PSOs can regress cold startup time
for customer money. As an experiment, attempt to defer PSO construction
to skia like.
---------
Co-authored-by: Aaron Clarke <aaclarke@google.com>
Co-authored-by: gaaclarke <30870216+gaaclarke@users.noreply.github.com>
<!-- start_original_pr_link -->
Reverts: flutter/flutter#165261
<!-- end_original_pr_link -->
<!-- start_initiating_author -->
Initiated by: jonahwilliams
<!-- end_initiating_author -->
<!-- start_revert_reason -->
Reason for reverting: bork the tree
<!-- end_revert_reason -->
<!-- start_original_pr_author -->
Original PR Author: jonahwilliams
<!-- end_original_pr_author -->
<!-- start_reviewers -->
Reviewed By: {gaaclarke}
<!-- end_reviewers -->
<!-- start_revert_body -->
This change reverts the following previous change:
The cost of bootstapping the initial PSOs can regress cold startup time
for customer money. As an experiment, attempt to defer PSO construction
to skia like.
<!-- end_revert_body -->
Co-authored-by: auto-submit[bot] <flutter-engprod-team@google.com>
The cost of bootstapping the initial PSOs can regress cold startup time
for customer money. As an experiment, attempt to defer PSO construction
to skia like.
---------
Co-authored-by: Aaron Clarke <aaclarke@google.com>
Co-authored-by: gaaclarke <30870216+gaaclarke@users.noreply.github.com>
This benchmark is a benchmark of the integration test binding which IMO
is not that important given that we continue to rely on flutter_driver
for actual performance metrics.
Fixes https://github.com/flutter/flutter/issues/162615
Can test 2 different ways.
On a mac (or linux machine) with adb on the path (or android sdk set in
ANDROID_HOME) and an emulator running (or physical device attached) that
is api 30 or higher.
```
cd dev/devicelab
dart bin/test_runner.dart test -t android_display_cutout
```
OR
```
dev/integration_tests/display_cutout_rotation
flutter drive integration_test/display_cutout_test.dart
```
Proof the test ran successfully
```
[2025-02-12 08:08:22.069817] [STDOUT] Removing Synthetic notch...
[2025-02-12 08:08:22.071147] [STDOUT] Executing "/b/s/w/ir/cache/android/sdk/platform-tools/adb -s emulator-5554 shell cmd overlay disable com.android.internal.display.cutout.emulation.tall" in "/b/s/w/ir/x/w/rc/tmpk3k3yhhp/flutter sdk/dev/integration_tests/display_cutout_rotation/" with environment {BOT: true, LANG: en_US.UTF-8}
[2025-02-12 08:08:22.862219] [STDOUT] Checking for reboot
[android_defines_test] Process terminated with exit code 0.
Task result:
{
"success": true,
"data": null,
"detailFiles": [],
"benchmarkScoreKeys": [],
"reason": "success"
}
```
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8723125792202374961/+/u/run_android_defines_test/stdout
All checks passed
https://github.com/flutter/flutter/pull/162641/checks?check_run_id=36991537539
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
Towards https://github.com/flutter/flutter/issues/161333.
This adds the initial `.ci.yaml` configuration (marked `bringup: true`),
without changing the existing shard to avoid the problems we ran into
last time when I renamed the shard atomically, as well as piping for
providing context in which the test will run (Vulkan or OpenGLES), which
can't be introspected in the _running_ app.
In a follow-up PR I'll:
- Edit `AndroidManifest.xml` before the suite, setting `ImpellerBackend`
appropriately
- Move the bringup nodes to presubmit
- Delete the existing (legacy) node
It would be nice to have somewhere to iterate and experiment with what we can do to either fix frequent timeout problems we have with `flutter build apk` (across platforms, to be clear, though I've just started with Linux) or get more information on why the crashes/timeouts happen.
Open to other ways to doing this (though preferably _not_ LED).
Allows applying of `include_flutter.groovy` via the `apply from:` syntax, which allows using a host app that is using the Gradle Kotlin DSL (the default these days when creating an Android app in AS).
Explanation: The `include_flutter.groovy` script is currently not able to be called by Kotlin gradle files, because it is [intended to be invoked with the following lines](https://docs.flutter.dev/add-to-app/android/project-setup#depend-on-the-modules-source-code):
```
setBinding(new Binding([gradle: this])) // new
evaluate(new File( // new
settingsDir.parentFile, // new
'flutter_module/.android/include_flutter.groovy' // new
))
```
`setBinding` isn't part of the Kotlin gradle DSL, and there isn't (that I can find) an easy Kotlin equivalent. If this binding isn't set, the reference to `gradle` in `include_flutter.groovy` is wrong, which breaks the script.
This PR modifies `include_flutter.groovy` to also support being invoked through the standard way of invoking a script via the Gradle Groovy/Kotlin DSLs, which is `apply from:` (or it's slightly different Kotlin syntax). The start of the script identifies which of the two approaches is being used by checking if the binding is set, and then initializes some variables differently depending on the case.
If we land this, I believe we should update the example Gradle files for both the `kts` and `groovy` cases to prefer the `apply from` syntax as I think this is the syntax most developers would be more familiar with already seeing in their Gradle files.
5eca44a1c4 moved some ownership of tests from @christopherfujino to @andrewkolos since Chris was no longer leading engineering efforts for the `flutter` tool. Now that @bkonyi is leading, this PR moves ownership of these test suites to him.
@bkonyi, note that this doesn't imply that you will be responsible for fixing any issues in these tests. This just means that the bot will assign flake issues to you for further triage.
Sets up tests that verify we can build a fresh counter app across our Gradle/AGP/Kotlin support range.
Post submit only, because the suite takes ~30 minutes to run, and I expect it to be _somewhat_ rare that we break only one of these versions (and therefore it doesn't get caught by existing presubmits).
This is a very common usage of ad so we want to make sure it's performant.
From the video, it scrolls quite smoothly, but we want to see the numbers, and keep monitoring it in case of regression.
https://github.com/flutter/flutter/assets/41930132/c7811c15-ac07-4989-a8a9-3c128e08cbe0
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/150230
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Reverts: flutter/flutter#150969
Initiated by: goderbauer
Reason for reverting: Failing test in https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8743574743030691569/+/u/run_android_obfuscate_test/stdout
Original PR Author: gmackall
Reviewed By: {christopherfujino, reidbaker}
This change reverts the following previous change:
After the land of https://github.com/flutter/engine/pull/53592, there is some log spam:
```
e: /Users/mackall/.gradle/caches/transforms-3/c1e137371ec1afe9bc9bd7b05823752d/transformed/fragment-1.7.1/jars/classes.jar!/META-INF/fragment_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
e: /Users/mackall/.gradle/caches/transforms-3/d86c7cb1c556fe1655fa56db671c649c/transformed/jetified-activity-1.8.1/jars/classes.jar!/META-INF/activity_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
...
```
I think this is harmless, but still annoying. Upgrading the AGP version fixes it. To be honest, I don't know why (I expected the Kotlin version would do it). But after https://github.com/flutter/flutter/pull/146307, our tests have been running on AGP/Gradle 8.1/8.3 for a while, so it makes sense to upgrade anyways.
In a follow up PR:
1. Also upgrade the tests that were left behind in https://github.com/flutter/flutter/pull/146307, as I think removal of discontinued plugins paved the way here.
After the land of https://github.com/flutter/engine/pull/53592, there is some log spam:
```
e: /Users/mackall/.gradle/caches/transforms-3/c1e137371ec1afe9bc9bd7b05823752d/transformed/fragment-1.7.1/jars/classes.jar!/META-INF/fragment_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
e: /Users/mackall/.gradle/caches/transforms-3/d86c7cb1c556fe1655fa56db671c649c/transformed/jetified-activity-1.8.1/jars/classes.jar!/META-INF/activity_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
...
```
I think this is harmless, but still annoying. Upgrading the AGP version fixes it. To be honest, I don't know why (I expected the Kotlin version would do it). But after https://github.com/flutter/flutter/pull/146307, our tests have been running on AGP/Gradle 8.1/8.3 for a while, so it makes sense to upgrade anyways.
In a follow up PR:
1. Also upgrade the tests that were left behind in https://github.com/flutter/flutter/pull/146307, as I think removal of discontinued plugins paved the way here.
Instead I've added @andrewkolos for tools tests and @jtmcdole for engine
tests. This doesn't mean you have to investigate and solve the issues
filed by the flake bot. Instead, an assignee is just the mechanism used
by the flake bot to ensure that the issues are triaged at the
appropriate team triage meetings.
FYI @christopherfujino
* Adds support for `flutter test --wasm`.
* The test compilation flow is a bit different now, so that it supports compilers other than DDC. Specifically, when we run a set of unit tests, we generate a "switchboard" main function that imports each unit test and runs the main function for a specific one based off of a value set by the JS bootstrapping code. This way, there is one compile step and the same compile output is invoked for each unit test file.
* Also, removes all references to `dart:html` from flutter/flutter.
* Adds CI steps for running the framework unit tests with dart2wasm+skwasm
* These steps are marked as `bringup: true`, so we don't know what kind of failures they will result in. Any failures they have will not block the tree at all yet while we're still in `bringup: true`. Once this PR is merged, I plan on looking at any failures and either fixing them or disabling them so we can get these CI steps running on presubmit.
This fixes https://github.com/flutter/flutter/issues/126692
Reverts: flutter/flutter#145224
Initiated by: hellohuanlin
Reason for reverting: breaks the tree
Original PR Author: hellohuanlin
Reviewed By: {gmackall, jmagman}
This change reverts the following previous change:
Reland https://github.com/flutter/flutter/pull/144745, which got reverted due to Android lockfile. Fixed by `dart dev/tools/bin/generate_gradle_lockfiles.dart`
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/143534
Fixes https://github.com/flutter/flutter/issues/143257
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
