Refactor the framework + test harness test suites in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing tests
The refactor of both suites included in this PR because they both depended on util functions and variables that also needed to be refactored.
Part of https://github.com/flutter/flutter/issues/145482
Move from `test.dart` into new `runner_utils.dart` file:
- shard calculation related methods + `shuffleSeed` logic
- env variable flag calculation
- runner wrapper methods `runDartTest` and `_runFromList`
Create web tests suite class containing:
- web long running tests
- runWebHtmlUnitTests
- runWebCanvasKitUnitTests
- runWebSkwasmUnitTests
- the associate helper methods with the above
Refactor verify codesigned tests in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing verify codesigned tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor flutter_plugins suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing flutter_plugins tests
Also update the test_test invocation of getFlutterPackagesVersion
Part of https://github.com/flutter/flutter/issues/145482
`@visibleForOverriding` + `@protected` unfortunately does not catch the case where a `compute*` method was overridden in a subtype and the overide was called in that same type's implementation.
I did not add a `flutter_ignore` for this because it doesn't seem there will be false positives.
Reverts: flutter/flutter#144706
Initiated by: gspencergoog
Reason for reverting: This has broken the tree because some tests are still failing post completion. This particular one looks like it might have to do with a gold image not existing.
Original PR Author: goderbauer
Reviewed By: {Piinks}
This change reverts the following previous change:
A test was failing silently because of this (see https://github.com/flutter/flutter/issues/144353 and fixed in https://github.com/flutter/flutter/pull/144709). The failure went undetected for months. Ideally, this should have been a regular non-silent failure. This change makes that so. `package:test` can properly handle reported exceptions outside of test cases. With this change, the test fails as follows:
```
00:03 +82: Smoke test material/color_scheme/dynamic_content_color.0.dart
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test (but after the test had completed):
setState() called after dispose(): _DynamicColorExampleState#1cd37(lifecycle state: defunct, not
mounted)
This error happens if you call setState() on a State object for a widget that no longer appears in
the widget tree (e.g., whose parent widget no longer includes the widget in its build). This error
can occur when code calls setState() from a timer or an animation callback.
The preferred solution is to cancel the timer or stop listening to the animation in the dispose()
callback. Another solution is to check the "mounted" property of this object before calling
setState() to ensure the object is still in the tree.
This error might indicate a memory leak if setState() is being called because another object is
retaining a reference to this State object after it has been removed from the tree. To avoid memory
leaks, consider breaking the reference to this object during dispose().
When the exception was thrown, this was the stack:
#0 State.setState.<anonymous closure> (package:flutter/src/widgets/framework.dart:1167:9)
#1 State.setState (package:flutter/src/widgets/framework.dart:1202:6)
#2 _DynamicColorExampleState._updateImage (package:flutter_api_samples/material/color_scheme/dynamic_content_color.0.dart:191:5)
<asynchronous suspension>
════════════════════════════════════════════════════════════════════════════════════════════════════
00:03 +81 -1: Smoke test material/context_menu/context_menu_controller.0.dart
00:03 +81 -1: Smoke test material/color_scheme/dynamic_content_color.0.dart [E]
Test failed. See exception logs above.
The test description was: Smoke test material/color_scheme/dynamic_content_color.0.dart
This test failed after it had already completed.
Make sure to use a matching library which informs the test runner
of pending async work.
```
A test was failing silently because of this (see
https://github.com/flutter/flutter/issues/144353 and fixed in
https://github.com/flutter/flutter/pull/144709). The failure went
undetected for months. Ideally, this should have been a regular
non-silent failure. This change makes that so. `package:test` can
properly handle reported exceptions outside of test cases. With this
change, the test fails as follows:
```
00:03 +82: Smoke test material/color_scheme/dynamic_content_color.0.dart
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test (but after the test had completed):
setState() called after dispose(): _DynamicColorExampleState#1cd37(lifecycle state: defunct, not
mounted)
This error happens if you call setState() on a State object for a widget that no longer appears in
the widget tree (e.g., whose parent widget no longer includes the widget in its build). This error
can occur when code calls setState() from a timer or an animation callback.
The preferred solution is to cancel the timer or stop listening to the animation in the dispose()
callback. Another solution is to check the "mounted" property of this object before calling
setState() to ensure the object is still in the tree.
This error might indicate a memory leak if setState() is being called because another object is
retaining a reference to this State object after it has been removed from the tree. To avoid memory
leaks, consider breaking the reference to this object during dispose().
When the exception was thrown, this was the stack:
#0 State.setState.<anonymous closure> (package:flutter/src/widgets/framework.dart:1167:9)
#1 State.setState (package:flutter/src/widgets/framework.dart:1202:6)
#2 _DynamicColorExampleState._updateImage (package:flutter_api_samples/material/color_scheme/dynamic_content_color.0.dart:191:5)
<asynchronous suspension>
════════════════════════════════════════════════════════════════════════════════════════════════════
00:03 +81 -1: Smoke test material/context_menu/context_menu_controller.0.dart
00:03 +81 -1: Smoke test material/color_scheme/dynamic_content_color.0.dart [E]
Test failed. See exception logs above.
The test description was: Smoke test material/color_scheme/dynamic_content_color.0.dart
This test failed after it had already completed.
Make sure to use a matching library which informs the test runner
of pending async work.
```
On the beta branch:
```
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-profile/extension_safe/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-profile/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios/extension_safe/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-release/extension_safe/Flutter.xcframework
Verifying the code signature of /Users/m/Projects/flutter/bin/cache/artifacts/engine/ios-release/Flutter.xcframework
```
Fixes https://github.com/flutter/flutter/issues/140934
Each error section is numbered, so you can all be sure you're talking about the same one.
A message is printed at the very end telling you how to find the error blocks in the verbose logs.
I did not include the `'// flutter_ignore_for_file: stopwatch (see analyze.dart)'` directive since it's currently not used, and adding that shouldn't be too difficult.
I plan to extend the prepare_package.dart script to upload the flutter preview device ([design doc](https://docs.google.com/document/d/1AzI-_Uk2v1LA2kKKFJ7gVD4xcakXJ6yVZiS5Ek6RHtg/edit#heading=h.byp03plw7mg9)).
However, given that that script is one large >1k line file, I decided to organize it into smaller libraries in this PR. There should be no behavioral change in this PR, this is a cleanup only. I made the following changes:
1. Created a //dev/bots/prepare_package/ directory to contain helper libraries
2. Moved everything but the `main()` function in //dev/bots/prepare_package.dart into one of 4 helper libraries under the new directory from step 1:
a. archive_creator.dart which contains the code that creates archive directory locally on disk
b. archive_publisher.dart which contains the code that uploads the archive to cloud storage
c. common.dart for shared constants and definitions
d. process_runner.dart for an abstraction over running sub-processes
3. Changed all definitions to `File` and `Directory` from `dart:io` to use the testable versions from `package:file`. This allowed me to use the `MemoryFileSystem` in the unit tests, rather than creating real temp file system directories.
## Description
This checks API doc strings for malformed links to examples. It prevents errors in capitalization, spacing, number of asterisks, etc. It won't catch all errors, because it needs to have a minimally indicative string to know that it even is trying to be a link to an example. At a minimum, the line needs to look like (literally, not as a regexp) `///*seecode.*` in order to be seen as a link to an example.
Separately, I'm going to add a check to the snippets tool that checks to make sure that an `{@tool}` block includes either a link to a sample file or a dart code block.
## Tests
- Added a test to make sure it catches some malformed links.
Use the pub cache resolved by pub itself.
To add packages to the flutter.zip download they are packaged as tar.gz and added to the pub-cache on first run by using `pub cache preload`.
In preparation for the merge of flutter/plugins into flutter/packaegs,
update the cross-repo analysis to flutter/packages rather than
flutter/plugins.
The flutter_plugins.version file is left intentionally for now to avoid
issues with the roller; it will be removed when the roller has been
updated to roll the other repository.
* Generate local metadata even when not publishing.
For SLSA compliance we need to separate the fetch, compile and upload
steps of release artifacts. Translating this to the packaging workflows
the fetch step will checkout the prepare_package script at main ToT, the
compile step generate the bundle archives and the recipes will upload
the artifact bundles as part of the upload stage.
This change adds functionality to generate both the release bundle and
the updated metadata file in a way that both files can be uploaded as
part of the upload stage.
Bug: https://github.com/flutter/flutter/issues/115487
* Address comments.
* Update tests.
