Refactor framework coverage tests in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing framework coverage tests
Part of https://github.com/flutter/flutter/issues/145482
Move from `test.dart` into new `runner_utils.dart` file:
- shard calculation related methods + `shuffleSeed` logic
- env variable flag calculation
- runner wrapper methods `runDartTest` and `_runFromList`
Create web tests suite class containing:
- web long running tests
- runWebHtmlUnitTests
- runWebCanvasKitUnitTests
- runWebSkwasmUnitTests
- the associate helper methods with the above
This pull request aims for improved readability, based on issue #146600.
```dart
// before
Set<Color> _distinctVisibleColors() {
final Set<Color> distinctVisibleColors = <Color>{};
if (top.style != BorderStyle.none) {
distinctVisibleColors.add(top.color);
}
if (right.style != BorderStyle.none) {
distinctVisibleColors.add(right.color);
}
if (bottom.style != BorderStyle.none) {
distinctVisibleColors.add(bottom.color);
}
if (left.style != BorderStyle.none) {
distinctVisibleColors.add(left.color);
}
return distinctVisibleColors;
}
// after
Set<Color> _distinctVisibleColors() {
return <Color>{
if (top.style != BorderStyle.none) top.color,
if (right.style != BorderStyle.none) right.color,
if (bottom.style != BorderStyle.none) bottom.color,
if (left.style != BorderStyle.none) left.color,
};
}
```
Most of the repo should be covered in this PR (aside from `flutter_tools/`, since there was a lot going on in there).
Refactor verify codesigned tests in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing verify codesigned tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor fuchsia_precache suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing fuchsia_precache tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor customer_testing suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing customer_testing tests
Part of https://github.com/flutter/flutter/issues/145482
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Refactor docs testing suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing docs tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor the analyze testing suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing analyze tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor realm_checker suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing realm_checker tests
Part of https://github.com/flutter/flutter/issues/145482
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Starts creating metric files consistently for web, dart and flutter runners. It also starts creating the files in the temp folder instead of the checkout one.
Bug: https://github.com/flutter/flutter/issues/145793
Refactor flutter_plugins suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing flutter_plugins tests
Also update the test_test invocation of getFlutterPackagesVersion
Part of https://github.com/flutter/flutter/issues/145482
Shard tests present output inconsistently using the compact or extended reporter depending on the defaults of flutter test and dart test. To make the output consistent and simplify debugging of failures this PR is enabling the extended reporter for all the tests based on unit/integration test packages.
Bug: https://github.com/flutter/flutter/issues/145616
Refactor skp_generator_tests suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing skp_generator_tests tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor web long running tests in order to reduce testing logic in test.dart, create a suite_runners directory and allow for later implementing package:test onto the existing tests
Part of https://github.com/flutter/flutter/issues/145482
This PR makes the static API of the `BackgroundIsolateBinaryMessenger` consistent between its web and IO implementations.
This way, multi-platform applications will compile for the web when using the `ensureInitialized` method (even though it'll throw at runtime, like the `instance` getter, because isolates are not supported on the web).
### Issues
* Fixes: https://github.com/flutter/flutter/issues/145260
Refactor add_to_app_life_cycle_tests in order to reduce testing logic in test.dart, create a suite_runners directory and allow for later implementing package:test onto add_to_app_life_cycle_tests
Part of https://github.com/flutter/flutter/issues/145482
I did some local debugging but couldn't get an exact root cause on the flakiness of these tests. Running them locally, there are a lot of assertions and exceptions being thrown in console, so I think we need to figure out what's going on there, but we should disable these for now to unblock the tree.
* Adds support for `flutter test --wasm`.
* The test compilation flow is a bit different now, so that it supports compilers other than DDC. Specifically, when we run a set of unit tests, we generate a "switchboard" main function that imports each unit test and runs the main function for a specific one based off of a value set by the JS bootstrapping code. This way, there is one compile step and the same compile output is invoked for each unit test file.
* Also, removes all references to `dart:html` from flutter/flutter.
* Adds CI steps for running the framework unit tests with dart2wasm+skwasm
* These steps are marked as `bringup: true`, so we don't know what kind of failures they will result in. Any failures they have will not block the tree at all yet while we're still in `bringup: true`. Once this PR is merged, I plan on looking at any failures and either fixing them or disabling them so we can get these CI steps running on presubmit.
This fixes https://github.com/flutter/flutter/issues/126692
Reverts: flutter/flutter#144706
Initiated by: gspencergoog
Reason for reverting: This has broken the tree because some tests are still failing post completion. This particular one looks like it might have to do with a gold image not existing.
Original PR Author: goderbauer
Reviewed By: {Piinks}
This change reverts the following previous change:
A test was failing silently because of this (see https://github.com/flutter/flutter/issues/144353 and fixed in https://github.com/flutter/flutter/pull/144709). The failure went undetected for months. Ideally, this should have been a regular non-silent failure. This change makes that so. `package:test` can properly handle reported exceptions outside of test cases. With this change, the test fails as follows:
```
00:03 +82: Smoke test material/color_scheme/dynamic_content_color.0.dart
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test (but after the test had completed):
setState() called after dispose(): _DynamicColorExampleState#1cd37(lifecycle state: defunct, not
mounted)
This error happens if you call setState() on a State object for a widget that no longer appears in
the widget tree (e.g., whose parent widget no longer includes the widget in its build). This error
can occur when code calls setState() from a timer or an animation callback.
The preferred solution is to cancel the timer or stop listening to the animation in the dispose()
callback. Another solution is to check the "mounted" property of this object before calling
setState() to ensure the object is still in the tree.
This error might indicate a memory leak if setState() is being called because another object is
retaining a reference to this State object after it has been removed from the tree. To avoid memory
leaks, consider breaking the reference to this object during dispose().
When the exception was thrown, this was the stack:
#0 State.setState.<anonymous closure> (package:flutter/src/widgets/framework.dart:1167:9)
#1 State.setState (package:flutter/src/widgets/framework.dart:1202:6)
#2 _DynamicColorExampleState._updateImage (package:flutter_api_samples/material/color_scheme/dynamic_content_color.0.dart:191:5)
<asynchronous suspension>
════════════════════════════════════════════════════════════════════════════════════════════════════
00:03 +81 -1: Smoke test material/context_menu/context_menu_controller.0.dart
00:03 +81 -1: Smoke test material/color_scheme/dynamic_content_color.0.dart [E]
Test failed. See exception logs above.
The test description was: Smoke test material/color_scheme/dynamic_content_color.0.dart
This test failed after it had already completed.
Make sure to use a matching library which informs the test runner
of pending async work.
```
A test was failing silently because of this (see
https://github.com/flutter/flutter/issues/144353 and fixed in
https://github.com/flutter/flutter/pull/144709). The failure went
undetected for months. Ideally, this should have been a regular
non-silent failure. This change makes that so. `package:test` can
properly handle reported exceptions outside of test cases. With this
change, the test fails as follows:
```
00:03 +82: Smoke test material/color_scheme/dynamic_content_color.0.dart
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test (but after the test had completed):
setState() called after dispose(): _DynamicColorExampleState#1cd37(lifecycle state: defunct, not
mounted)
This error happens if you call setState() on a State object for a widget that no longer appears in
the widget tree (e.g., whose parent widget no longer includes the widget in its build). This error
can occur when code calls setState() from a timer or an animation callback.
The preferred solution is to cancel the timer or stop listening to the animation in the dispose()
callback. Another solution is to check the "mounted" property of this object before calling
setState() to ensure the object is still in the tree.
This error might indicate a memory leak if setState() is being called because another object is
retaining a reference to this State object after it has been removed from the tree. To avoid memory
leaks, consider breaking the reference to this object during dispose().
When the exception was thrown, this was the stack:
#0 State.setState.<anonymous closure> (package:flutter/src/widgets/framework.dart:1167:9)
#1 State.setState (package:flutter/src/widgets/framework.dart:1202:6)
#2 _DynamicColorExampleState._updateImage (package:flutter_api_samples/material/color_scheme/dynamic_content_color.0.dart:191:5)
<asynchronous suspension>
════════════════════════════════════════════════════════════════════════════════════════════════════
00:03 +81 -1: Smoke test material/context_menu/context_menu_controller.0.dart
00:03 +81 -1: Smoke test material/color_scheme/dynamic_content_color.0.dart [E]
Test failed. See exception logs above.
The test description was: Smoke test material/color_scheme/dynamic_content_color.0.dart
This test failed after it had already completed.
Make sure to use a matching library which informs the test runner
of pending async work.
```
Reverts flutter/flutter#143244
Initiated by: vashworth
Reason for reverting: Increased `flutter_framework_uncompressed_bytes` - see https://github.com/flutter/flutter/issues/144251
Original PR Author: vashworth
Reviewed By: {jmagman}
This change reverts the following previous change:
Original Description:
Replace `FlutterMacOS.framework` cached artifact with `FlutterMacOS.xcframework`. Also, update usage of `FlutterMacOS.framework` to use `FlutterMacOS.xcframework`.
Part of https://github.com/flutter/flutter/issues/126016.
Replace `FlutterMacOS.framework` cached artifact with `FlutterMacOS.xcframework`. Also, update usage of `FlutterMacOS.framework` to use `FlutterMacOS.xcframework`.
Part of https://github.com/flutter/flutter/issues/126016.
On the beta branch:
```
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-profile/extension_safe/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-profile/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios/extension_safe/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-release/extension_safe/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-release/Flutter.xcframework
```
Fixes https://github.com/flutter/flutter/issues/140934
- Unskip `text_style_test` for CanvasKit.
- Remove no longer necessary `kIsWeb` checks in a few tests.
This PR depends on https://github.com/flutter/engine/pull/49786, which rolled into the framework. If the engine PR needs to be reverted, this PR will need to be reverted too.
The migration of customer tests to sharded tests adds a step that checks out the current tip-of-tree of the framework repo, removing local changes. This does not work with monorepo testing, which modifies engine.version, and does not work with local testing of a branch.
The sharded tests should already be running with the correct checkout of the framework repo. If the REVISION environment variable is set, the framework checkout will still be reset to check out that revision.
These commands were migrated from the existing shell script to the sharded tester in
https://github.com/flutter/flutter/pull/138659
Bug: https://github.com/dart-lang/sdk/issues/51042
Fix and unskip the following CanvasKit tests:
- `test/painting/decoration_test.dart`
- `test/rendering/layers_test.dart`
- `test/widgets/app_overrides_test.dart`
https://github.com/flutter/flutter/pull/138253 demonstrated that the tests for the textures example weren't actually running on CI. This changes the testing script to execute the tests for everything inside the `examples` directory.
## Description
This PR adds a `nonce` parameter to flutter.js' `loadEntrypoint` method.
When set, loadEntrypoint will add a `nonce` attribute to the `main.dart.js` script tag, which allows Flutter to run in environments slightly more restricted by CSP; those that don't add `'self'` as a valid source for `script-src`.
----
### CSP directive
After this change, the CSP directive for a Flutter Web index.html can be:
```
script-src 'nonce-YOUR_NONCE_VALUE' 'wasm-unsafe-eval';
font-src https://fonts.gstatic.com;
style-src 'nonce-YOUR_NONCE_VALUE';
```
When CSP is set via a `meta` tag (like in the test accompanying this change), and to use a service worker, the CSP needs an additional directive: [`worker-src 'self';`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/worker-src)
When CSP set via response headers, the CSP that applies to `flutter_service_worker.js` is determined by its response headers. See **Web Workers API > [Content security policy](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers#content_security_policy)** in MDN.)
----
### Initialization
If the CSP is set to disallow `script-src 'self'`, a nonce needs to also be passed to `loadEntrypoint`:
```javascript
_flutter.loader.loadEntrypoint({
nonce: 'SOME_NONCE',
onEntrypointLoaded: (engineInitializer) async {
const appRunner = await engineInitializer.initializeEngine({
nonce: 'SOME_NONCE',
});
appRunner.runApp();
},
});
```
(`nonce` shows twice for now, because the entrypoint loader script doesn't have direct access to the `initializeEngine` call.)
----
## Tests
* Added a smoke test to ensure an app configured as described above starts.
## Issues
* Fixes https://github.com/flutter/flutter/issues/126977
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
*List which issues are fixed by this PR. You must list at least one issue.*
- https://github.com/flutter/flutter/issues/128381
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
