* Move web integration tool tests to web.shard
Web integration tool tests depend on DDC changes in SDK. This change
moves them to a separate shard and subshard so CI bot configurations
can run them separately.
In particular, with will allow running those tests on dart CI flutter
HHH web bot instead of a non-web one, allowing early detection and easy
classification of issues caused by SDK changes as VM- or Web related.
* Enabled verbose mode for flaky web_tool_tests
* Split out the test changes to be commited first
This reverts ae0a9cb560 and therefore relands #69629.
Additionally, `flutter update-packages --force-upgrade` has been run to update `pubspec.yaml` to resolve the analyze test failures.
For the ease of code reviews, this only includes minimal code from
MetricPoint, GoogleBenchmarksParser, and their unit tests.
See go/flutter-metrics-center-migration for the overall plan.
Enables build_test for Linux. Currently only flutter_gallery has platform directories for the desktop platforms, so this will run only that build, but this will provide an end-to-end build test for Linux.
Other example/test projects can be brought online for Linux in the future just by adding the linux/ directory to the project.
Currently we only have a test that measures if flutter drive can fail. Unfortunately it has been failing for multiple years based on the fact that the error is due to a missing Gradle config.
Add a smoke success test case, and update both tests to use the test API and the tester-device.
Enables build_test for Windows. Currently only flutter_gallery has platform directories for the desktop platforms, so this will run only that build, but this will provide an end-to-end build test for Windows.
Other example/test projects can be brought online for Windows in the future just by adding the windows/ directory to the project.
Enables build_test for Windows, macOS, and Linux. Currently only
flutter_gallery has platform directories for the desktop platforms, so
this will run only that build, but this will provide an end-to-end build
test for all three desktop platforms.
Once this lands, other example/test projects can be brought online for
desktop platforms in the future just by adding the relevant platform
directories to the project.
Add integration tests to verify that ddc and dart2js can be built and run in sound mode. Updates dart2js compilation to insert a language version comment into the generated entrypoint if necessary.
dart-lang/sdk#42253
Port the deprecated settings devicelab test to tool integration shard. Tests that apps can be built using the deprecated android/settings.gradle file.
Part of #65790
Currently these tests are taking CI time without providing any value, because there is no verification that network connectivity is avoided. This is better covered by a unit test that verifies that --offline is passed to pub.
To allow tests to begin migrating to null safety, sound null safety needs to be disabled. Also removes the experiment flag that is unnecessary since the framework is on the SDK allowlist.
Like Android/iOS, only enable --null-assertions if asked. Previously this was enabled by default for web, but in general this has proved to be too breaking to enable by default.
#61042
There are some occasional difficult to repro flakes on the tool integration tests. Try running only a single isolate to simplify the environment a bit.
Enable null safety asserts for web debug mode. This induces runtime asserts at the boundaries between null safe and non-null safe libraries. Adds integration test that validates assertion error is thrown.
#61042
Use tester (disclaimer, written by me) to unit test the flutter tools general shard. Since these tests are designed to be hermetic, the single compilation/single isolate model offers a tremendous speedup over the default pub run test workflow. On previous linux shard, test execution alone took 10:30, while now it takes only 1:30.
Ensure that the language version of the test/web generated entrypoint matches the language version of the test file to run, or the overall package language version if no annotation is provided.
Also combines experiments into extraGenSnapshot/ExtraFrontEndOptions. Allows providing --no-sound-null-safety to allow out of order migration and running.
Ensure iOS and android builds can be correctly cached. Use the performance-measurement-file to verify that all targets were skipped on the second invocation. This is only run on the flutter_gallery build.
* Fix web test flakiness; enable web golden tests
The fix is three-part:
- Only allow one test to load _and_ test at any point in time.
- Use a fresh Chrome instance for each test file.
- Increase Cirrus resources.
The first two changes only fix the "Unknown error loading" error, but not hanging tests. The resource increase also prevents hanging tests.
Other minor changes:
- Remove test batching (it's no longer necessary)
- Fix the Chrome class, which was using the wrong Completer.
Move Flutter Gallery to dev/integration_tests/ as it is an older copy used only for testing. The current version of the Flutter Gallery now lives in https://github.com/flutter/gallery.
The stocks example app is outdated and deprecated, but we still use it for some benchmark tests. Moving it into the benchmarks directory to indicate its status.
* Update packages.
* Add many more global analyses.
* Catch trailing spaces and trailing newlines in all text files.
Before we were only checking newly added files, but that means we
missed some.
* Port the trailing spaces logic to work on Windows too.
* Correct all the files with trailing spaces and newlines.
* Refactor some of the dev/bots logic into a utils.dart library.
Notably, the "exit" and "print" shims for testing are now usable
from test.dart, analyze.dart, and run_command.dart.
* Add an "exitWithError" function that prints the red lines and
then exits. This is the preferred way to exit from test.dart,
analyze.dart, and run_command.dart.
* More consistency in the output of analyze.dart.
* Refactor analyze.dart to use the _allFiles file enumerating logic
more widely.
* Add some double-checking logic to the _allFiles logic to catch
cases where changes to that logic end up catching fewer files
than expected (helps prevent future false positives).
* Add a check to prevent new binary files from being added to
the repository. Grandfather in the binaries that we've already
added.
* Update all the dependencies (needed because we now import crypto in
dev/bots/analyze.dart).
