This adds the 'fail-fast' argument to flutter test, since dart test already supports this feature. Tests can now be stopped after first failure.
Fixes#124406
This adds support for adding the `--wasm` flag to `flutter run` and `flutter drive`
* Emits errors if you attempt to use the skwasm renderer without the `--wasm` flag
* Emits errors if you try to use `--wasm` when not using a web device
* Uses the skwasm renderer by default if you pass `--wasm` and no `--web-renderer`
* Adds support for `flutter test --wasm`.
* The test compilation flow is a bit different now, so that it supports compilers other than DDC. Specifically, when we run a set of unit tests, we generate a "switchboard" main function that imports each unit test and runs the main function for a specific one based off of a value set by the JS bootstrapping code. This way, there is one compile step and the same compile output is invoked for each unit test file.
* Also, removes all references to `dart:html` from flutter/flutter.
* Adds CI steps for running the framework unit tests with dart2wasm+skwasm
* These steps are marked as `bringup: true`, so we don't know what kind of failures they will result in. Any failures they have will not block the tree at all yet while we're still in `bringup: true`. Once this PR is merged, I plan on looking at any failures and either fixing them or disabling them so we can get these CI steps running on presubmit.
This fixes https://github.com/flutter/flutter/issues/126692
This PR implements the functionality described above and hides it behind
the `--experimental-faster-testing` flag of `flutter test`.
### The following are some performance measurements from test runs
conducted on GitHub Actions
run 1 logs:
https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1
run 2 logs:
https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2
run 3 logs:
https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3
**length of `flutter test --reporter=expanded test/animation
test/foundation` step**
run 1: 54s
run 2: 52s
run 3: 56s
average: 54s
**length of `flutter test --experimental-faster-testing
--reporter=expanded test/animation test/foundation` step**
run 1: 27s
run 2: 27s
run 3: 29s
average: 27.67s (~48.77% shorter than 54s)
**length of `flutter test --reporter=expanded test/animation
test/foundation test/gestures test/painting test/physics test/rendering
test/scheduler test/semantics test/services` step**
run 1: 260s
run 2: 270s
run 3: 305s
average: 278.33s
**length of `flutter test --experimental-faster-testing
--reporter=expanded test/animation test/foundation test/gestures
test/painting test/physics test/rendering test/scheduler test/semantics
test/services` step**
from a clean build (right after deleting the build folder):
run 1: 215s
run 2: 227s
run 3: 245s
average: 229s (~17.72% shorter than 278.33s)
Note that in reality, `test/material` was not passed to `flutter test`
in the trials below. All of the test files under `test/material` except
for `test/material/icons_test.dart` were listed out individually
**length of `flutter test --reporter=expanded test/material` step**
run 1: 408s
run 2: 421s
run 3: 451s
average: 426.67s
**length of `flutter test --experimental-faster-testing
--reporter=expanded test/material` step**
run 1: 382s
run 2: 373s
run 3: 400s
average: 385s (~9.77% shorter than 426.67s)
---------
Co-authored-by: Dan Field <dnfield@google.com>
This is an attempt at a reland of https://github.com/flutter/flutter/pull/141396
The main changes here that are different than the original PR is fixes to wire up the `flutter test` command properly with the web renderer.
Dual Web Compile has had some issues where `flutter test` is not respecting the `--web-renderer` flag for some reason. I haven't gotten entirely to the bottom of the issue, but for now we need to rever these changes while I investigate. This reverts the following PRs:
https://github.com/flutter/flutter/pull/143128https://github.com/flutter/flutter/pull/141396
While doing this revert, I had a few merge conflicts with https://github.com/flutter/flutter/pull/142760, and I tried to resolve the merge conflicts within the spirit of that PR's change, but @chingjun I might need your input on whether the imports I have modified are okay with regards to the change you were making.
Reland of https://github.com/flutter/flutter/pull/142709.
The revert of the revert is in the first commit, the fix in the commit on top.
The move of the fakes for packages/flutter_tools/test/general.shard/resident_runner_test.dart was erroneous before, as it was trying to use setters instead of a private field. This PR changes the private `_devFS` field in the fake to be a public `fakeDevFS` in line with other fakes.
## Original PR description
Native assets in other build systems are not built with `package:native_assets_builder` invoking `build.dart` scripts. Instead all packages have their own blaze rules. Therefore we'd like to not depend on `package:native_assets_builder` from flutter tools in g3 at all.
This PR aims to move the imports of `native_assets_builder` and `native_assets_cli` into the `isolated/` directory and into the files with a `main` function that are not used in with other build systems.
In order to be able to remove all imports in files used by other build systems, two new interfaces are added `HotRunnerNativeAssetsBuilder` and `TestCompilerNativeAssetsBuilder`. New parameters are then piped all the way through from the entry points:
* bin/fuchsia_tester.dart
* lib/executable.dart
The build_system/targets dir is already excluded in other build systems.
So, after this PR only the two above files and build_system/targets import from `isolated/native_assets/` and only `isolated/native_assets/` import `package:native_assets_cli` and `package:native_assets_builder`.
Context:
* https://github.com/flutter/flutter/issues/142041
Reverts flutter/flutter#142709
Initiated by: vashworth
Reason for reverting: `Mac tool_tests_general` started failing on this commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20tool_tests_general/15552/overview
Original PR Author: dcharkes
Reviewed By: {christopherfujino, chingjun, reidbaker}
This change reverts the following previous change:
Original Description:
Native assets in other build systems are not built with `package:native_assets_builder` invoking `build.dart` scripts. Instead all packages have their own blaze rules. Therefore we'd like to not depend on `package:native_assets_builder` from flutter tools in g3 at all.
This PR aims to move the imports of `native_assets_builder` and `native_assets_cli` into the `isolated/` directory and into the files with a `main` function that are not used in with other build systems.
In order to be able to remove all imports in files used by other build systems, two new interfaces are added `HotRunnerNativeAssetsBuilder` and `TestCompilerNativeAssetsBuilder`. New parameters are then piped all the way through from the entry points:
* bin/fuchsia_tester.dart
* lib/executable.dart
The build_system/targets dir is already excluded in other build systems.
So, after this PR only the two above files and build_system/targets import from `isolated/native_assets/` and only `isolated/native_assets/` import `package:native_assets_cli` and `package:native_assets_builder`.
Context:
* https://github.com/flutter/flutter/issues/142041
Native assets in other build systems are not built with `package:native_assets_builder` invoking `build.dart` scripts. Instead all packages have their own blaze rules. Therefore we'd like to not depend on `package:native_assets_builder` from flutter tools in g3 at all.
This PR aims to move the imports of `native_assets_builder` and `native_assets_cli` into the `isolated/` directory and into the files with a `main` function that are not used in with other build systems.
In order to be able to remove all imports in files used by other build systems, two new interfaces are added `HotRunnerNativeAssetsBuilder` and `TestCompilerNativeAssetsBuilder`. New parameters are then piped all the way through from the entry points:
* bin/fuchsia_tester.dart
* lib/executable.dart
The build_system/targets dir is already excluded in other build systems.
So, after this PR only the two above files and build_system/targets import from `isolated/native_assets/` and only `isolated/native_assets/` import `package:native_assets_cli` and `package:native_assets_builder`.
Context:
* https://github.com/flutter/flutter/issues/142041
PR provides a new option to the `test` command to include coverage info of specified packages.
It helps collecting coverage info in test setups where test code lives in separate packages or for multi-package projects.
At present, only current package is included to the final report.
Usage:
Consider an app with two packages: `app`, `common`.
Some of the tests in `app` use (indirectly) code that is located in `common`. When running with `--coverage` flag, that code is not included in the coverage report by default. To include `common` package in report, we can run:
```sh
flutter test --coverage --coverage-package app --coverage-package common
```
Note that `--coverage-package` accepts regular expression.
Fixes https://github.com/flutter/flutter/issues/79661
Fixes https://github.com/flutter/flutter/issues/101486
Fixes https://github.com/flutter/flutter/issues/93619
This should fix https://github.com/flutter/flutter/issues/126178
When we don't pass a `--concurrency` flag to the test package, it uses a default based on the number of cores that are on the machine. However, the web test platform itself serializes all these requests anyway, which can lead to the test package timing out. This is because from the test package's perspective, it has already started the loading process on a number of suites which are simply waiting for other test suites to compile and run. The ones that wait the longest can run up against the test packages 12 minute timeout for loading a given suite, even though they haven't actually started to try to load.
Instead, we should always pass `--concurrency=1` to the test package so that it doesn't attempt to start loads concurrently in the first place.
* [flutter_tools] Add support for URI formats like ?line=x for "flutter test"
* Remove unnecessary function
* Handle parsing absolute paths on Windows
* Use Windows-style paths when running on Windows
* Fix paths in isFile
* Remove unnecessary clear
* allow passing --file-reporter option to test running refs #69425
* Add trailing comma to help to meet style requirements
* Add space between tests for clarity
---------
Co-authored-by: daniel-v <dvarga@skawa.hu>
* removing default values for [reporter] and [timeout]]
* passing reporter arg to see tests pass
* added test to confirm TestCommand is not passing defaults
* add'l helper message for [reporter] arg
* default behavior for github actions + fixed tests
* removing github conditional for reporter + related test
* removing unused import
