This reverts commit 4300226a50.
Reason for revert: The original PR correctly passes down the local
engine flags to subprocesses of the tests. Though this causes issues
because some tests assume a specific target but do not modify the
value for `--local-engine` (which is set to `host_debug`) to reflect
that.
See more discussion here:
https://github.com/flutter/flutter/pull/71721#issuecomment-743218994
Adds a new Luci test shard that runs flutter/plugins tests.
For now only the analysis phase of the flutter/plugins test suite is executed.
Manual invocation on Luci completed successfully: 3210bb2dac/+/annotations
* Move web integration tool tests to web.shard
Web integration tool tests depend on DDC changes in SDK. This change
moves them to a separate shard and subshard so CI bot configurations
can run them separately.
In particular, with will allow running those tests on dart CI flutter
HHH web bot instead of a non-web one, allowing early detection and easy
classification of issues caused by SDK changes as VM- or Web related.
* Enabled verbose mode for flaky web_tool_tests
* Split out the test changes to be commited first
This reverts ae0a9cb560 and therefore relands #69629.
Additionally, `flutter update-packages --force-upgrade` has been run to update `pubspec.yaml` to resolve the analyze test failures.
For the ease of code reviews, this only includes minimal code from
MetricPoint, GoogleBenchmarksParser, and their unit tests.
See go/flutter-metrics-center-migration for the overall plan.
Enables build_test for Linux. Currently only flutter_gallery has platform directories for the desktop platforms, so this will run only that build, but this will provide an end-to-end build test for Linux.
Other example/test projects can be brought online for Linux in the future just by adding the linux/ directory to the project.
Currently we only have a test that measures if flutter drive can fail. Unfortunately it has been failing for multiple years based on the fact that the error is due to a missing Gradle config.
Add a smoke success test case, and update both tests to use the test API and the tester-device.
Enables build_test for Windows. Currently only flutter_gallery has platform directories for the desktop platforms, so this will run only that build, but this will provide an end-to-end build test for Windows.
Other example/test projects can be brought online for Windows in the future just by adding the windows/ directory to the project.
Enables build_test for Windows, macOS, and Linux. Currently only
flutter_gallery has platform directories for the desktop platforms, so
this will run only that build, but this will provide an end-to-end build
test for all three desktop platforms.
Once this lands, other example/test projects can be brought online for
desktop platforms in the future just by adding the relevant platform
directories to the project.
Add integration tests to verify that ddc and dart2js can be built and run in sound mode. Updates dart2js compilation to insert a language version comment into the generated entrypoint if necessary.
dart-lang/sdk#42253
Port the deprecated settings devicelab test to tool integration shard. Tests that apps can be built using the deprecated android/settings.gradle file.
Part of #65790
Currently these tests are taking CI time without providing any value, because there is no verification that network connectivity is avoided. This is better covered by a unit test that verifies that --offline is passed to pub.
To allow tests to begin migrating to null safety, sound null safety needs to be disabled. Also removes the experiment flag that is unnecessary since the framework is on the SDK allowlist.
Like Android/iOS, only enable --null-assertions if asked. Previously this was enabled by default for web, but in general this has proved to be too breaking to enable by default.
#61042
There are some occasional difficult to repro flakes on the tool integration tests. Try running only a single isolate to simplify the environment a bit.
Enable null safety asserts for web debug mode. This induces runtime asserts at the boundaries between null safe and non-null safe libraries. Adds integration test that validates assertion error is thrown.
#61042
Use tester (disclaimer, written by me) to unit test the flutter tools general shard. Since these tests are designed to be hermetic, the single compilation/single isolate model offers a tremendous speedup over the default pub run test workflow. On previous linux shard, test execution alone took 10:30, while now it takes only 1:30.
Ensure that the language version of the test/web generated entrypoint matches the language version of the test file to run, or the overall package language version if no annotation is provided.
Also combines experiments into extraGenSnapshot/ExtraFrontEndOptions. Allows providing --no-sound-null-safety to allow out of order migration and running.
Ensure iOS and android builds can be correctly cached. Use the performance-measurement-file to verify that all targets were skipped on the second invocation. This is only run on the flutter_gallery build.
* Fix web test flakiness; enable web golden tests
The fix is three-part:
- Only allow one test to load _and_ test at any point in time.
- Use a fresh Chrome instance for each test file.
- Increase Cirrus resources.
The first two changes only fix the "Unknown error loading" error, but not hanging tests. The resource increase also prevents hanging tests.
Other minor changes:
- Remove test batching (it's no longer necessary)
- Fix the Chrome class, which was using the wrong Completer.
Move Flutter Gallery to dev/integration_tests/ as it is an older copy used only for testing. The current version of the Flutter Gallery now lives in https://github.com/flutter/gallery.
The stocks example app is outdated and deprecated, but we still use it for some benchmark tests. Moving it into the benchmarks directory to indicate its status.
* Update packages.
* Add many more global analyses.
* Catch trailing spaces and trailing newlines in all text files.
Before we were only checking newly added files, but that means we
missed some.
* Port the trailing spaces logic to work on Windows too.
* Correct all the files with trailing spaces and newlines.
* Refactor some of the dev/bots logic into a utils.dart library.
Notably, the "exit" and "print" shims for testing are now usable
from test.dart, analyze.dart, and run_command.dart.
* Add an "exitWithError" function that prints the red lines and
then exits. This is the preferred way to exit from test.dart,
analyze.dart, and run_command.dart.
* More consistency in the output of analyze.dart.
* Refactor analyze.dart to use the _allFiles file enumerating logic
more widely.
* Add some double-checking logic to the _allFiles logic to catch
cases where changes to that logic end up catching fewer files
than expected (helps prevent future false positives).
* Add a check to prevent new binary files from being added to
the repository. Grandfather in the binaries that we've already
added.
* Update all the dependencies (needed because we now import crypto in
dev/bots/analyze.dart).
* Support Flutter Driver test for Flutter Web application.
* Support Flutter Driver test for Flutter Web application.
* Fix documentation issues.
* Support Flutter Driver test for Flutter Web application.
* Fix documentation.
* Remove unused file from dartdoc check.
* Sync to date.
* Revert change to dartdoc.
* Address comments.
* Apply suggestions from code review
Co-Authored-By: Jonah Williams <jonahwilliams@google.com>
* Update copyrights.
* Update allowed list for browsers.
* Verify command line arguments for Drive command is correctly parsed.
* Make waitUntilFirstFrameRasterized throw unimplementedError for Flutter Web Driver.
* Add comment for why sync WebDriver is used.
* Update documentations.
* Add more unit tests and update documentation.
* Configure test.dart so that web_extension_test will be executed with --platform=chrome.
* Revert unnecessary changes.
* Add new file path for Windows to blacklist.
* Reconstruct the structure of flutter_driver/test/src folder to remove filtering logic in dev/bots/test.dart/
* Fix path to web_extension_test.dart.
* Add instructions for how to use WebFlutterDriver.
* Update getLayerTree to use sendCommand instead of _sendCommand.
* Update pubspec files.
* Add tests to gen_l10n.dart tool
* Separate out LocalizationsGenerator class to improve testability of code
* Add testing dependencies to dev/tools
* Integrate dev/tools testing to flutter CI
* Restructure dev/tools folder for testing
* Fix license headers
* Update project.pbxproj files to say Flutter rather than Chromium
Also, the templates now have an empty organization so that we don't cause people to give their apps a Flutter copyright.
* Update the copyright notice checker to require a standard notice on all files
* Update copyrights on Dart files. (This was a mechanical commit.)
* Fix weird license headers on Dart files that deviate from our conventions; relicense Shrine.
Some were already marked "The Flutter Authors", not clear why. Their
dates have been normalized. Some were missing the blank line after the
license. Some were randomly different in trivial ways for no apparent
reason (e.g. missing the trailing period).
* Clean up the copyrights in non-Dart files. (Manual edits.)
Also, make sure templates don't have copyrights.
* Fix some more ORGANIZATIONNAMEs
