Commit Graph

32 Commits

Author SHA1 Message Date
Derek Xu
dfb5888e8f
Support using lightweight Flutter Engines to run tests (#141726)
This PR implements the functionality described above and hides it behind
the `--experimental-faster-testing` flag of `flutter test`.

### The following are some performance measurements from test runs
conducted on GitHub Actions

run 1 logs:
https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1
run 2 logs:
https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2
run 3 logs:
https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3

**length of `flutter test --reporter=expanded test/animation
test/foundation` step**

run 1: 54s
run 2: 52s
run 3: 56s

average: 54s

**length of `flutter test --experimental-faster-testing
--reporter=expanded test/animation test/foundation` step**

run 1: 27s
run 2: 27s
run 3: 29s

average: 27.67s (~48.77% shorter than 54s)

**length of `flutter test --reporter=expanded test/animation
test/foundation test/gestures test/painting test/physics test/rendering
test/scheduler test/semantics test/services` step**

run 1: 260s
run 2: 270s
run 3: 305s

average: 278.33s


**length of `flutter test --experimental-faster-testing
--reporter=expanded test/animation test/foundation test/gestures
test/painting test/physics test/rendering test/scheduler test/semantics
test/services` step**

from a clean build (right after deleting the build folder):

run 1: 215s
run 2: 227s
run 3: 245s

average: 229s (~17.72% shorter than 278.33s)

Note that in reality, `test/material` was not passed to `flutter test`
in the trials below. All of the test files under `test/material` except
for `test/material/icons_test.dart` were listed out individually

**length of `flutter test --reporter=expanded test/material` step**

run 1: 408s
run 2: 421s
run 3: 451s

average: 426.67s

**length of `flutter test --experimental-faster-testing
--reporter=expanded test/material` step**

run 1: 382s
run 2: 373s
run 3: 400s

average: 385s (~9.77% shorter than 426.67s)

---------

Co-authored-by: Dan Field <dnfield@google.com>
2024-02-22 13:32:29 -05:00
Christopher Fujino
12cf9de68b
Fix mac tool_integration_tests with Xcode 15 (#133217)
Fixes https://github.com/flutter/flutter/issues/132990
2023-08-24 17:03:17 +00:00
Christopher Fujino
cc83f03822
[flutter_tools] Migrate more integration tests to process result matcher (#128737)
Part of https://github.com/flutter/flutter/issues/127135
2023-06-15 13:25:32 -07:00
Christopher Fujino
7f1f765521
[flutter_tools] Use process matcher for multidex test (#127996)
Part of https://github.com/flutter/flutter/issues/127135
Part of https://github.com/flutter/flutter/issues/125115
2023-06-01 22:24:08 +00:00
Ben Konyi
d83c510de9
Reland "Serve DevTools when running flutter test (#123607)" (#124058)
Originally reverted due to a misattributed hot reload benchmark
regression.

This reverts commit f5bce716e1.
2023-04-04 19:44:14 -04:00
Zachary Anderson
f5bce716e1
Revert "Serve DevTools when running flutter test (#123607)" (#123810)
This reverts commit c3de901538.
2023-03-30 14:56:13 -07:00
Ben Konyi
c3de901538
Serve DevTools when running flutter test (#123607)
Also clean up messaging output by `flutter test` when `--start-paused` is provided.
2023-03-30 11:51:08 -04:00
Danny Tuppeny
06952ba254
[flutter_tools] Add support for URI formats like ?line=x for "flutter test" (#119740)
* [flutter_tools] Add support for URI formats like ?line=x for "flutter test"

* Remove unnecessary function

* Handle parsing absolute paths on Windows

* Use Windows-style paths when running on Windows

* Fix paths in isFile

* Remove unnecessary clear
2023-02-27 19:13:00 +00:00
Ben Konyi
47a0674651
Reland "Add --serve-observatory flag to run, attach, and test (#118402)" (#119737)
This reverts commit 254a796bc3.
2023-02-06 09:23:35 -05:00
Zachary Anderson
254a796bc3
Revert "Reland "Add --serve-observatory flag to run, attach, and test (#118402)" (#119529)" (#119729)
This reverts commit 7477d7ac79.
2023-02-01 09:19:48 -08:00
Ben Konyi
7477d7ac79
Reland "Add --serve-observatory flag to run, attach, and test (#118402)" (#119529)
* Reland "Add --serve-observatory flag to run, attach, and test (#118402)"

This reverts commit 86ab01d2bd.

* Fix flaky failures

* Fix VM service disappearing failure
2023-01-30 21:00:18 +00:00
Zachary Anderson
86ab01d2bd
Revert "Add --serve-observatory flag to run, attach, and test (#118402)" (#119302)
This reverts commit 6cd4fa45e2.
2023-01-26 16:43:39 -08:00
Ben Konyi
6cd4fa45e2
Add --serve-observatory flag to run, attach, and test (#118402)
This flag will allow for Observatory to be served by the VM service once it is disabled by default in the Dart SDK.
2023-01-25 12:46:08 -05:00
Michael Goderbauer
b308555ed1
Enable dangling_library_doc_comments and library_annotations lints (#117365) 2022-12-20 16:03:21 -08:00
Elias Yishak
d7454d552a
removing default values for [reporter] and [timeout] in flutter test (#115160)
* removing default values for [reporter] and [timeout]]

* passing reporter arg to see tests pass

* added test to confirm TestCommand is not passing defaults

* add'l helper message for [reporter] arg

* default behavior for github actions + fixed tests

* removing github conditional for reporter + related test

* removing unused import
2022-11-16 18:20:42 +00:00
Jonah Williams
9f28b83c19
[flutter_tools] migrate some integration tests to null safety (#103560) 2022-05-13 09:39:10 -07:00
Alexandre Ardhuin
07f1c20474
add missing trailing commas in list/set/map literals (#102585) 2022-04-27 09:15:35 +02:00
Greg Spencer
52ae102f18
Adds tool warning log level and command line options to fail on warning/error output (#92031) 2021-11-10 16:13:04 -08:00
Christopher Fujino
44bf975115
remove Downloading ... stderr messages (#92211) 2021-10-20 17:13:02 -07:00
Ian Hickson
f25b833f27
Enable avoid_print lint. (#91444) 2021-10-07 16:48:04 -07:00
Mouad Debbar
c2ea78d231
Revert "Enable avoid_print lint. (#91332)" (#91438)
This reverts commit cb378edc9e.
2021-10-07 16:16:17 -04:00
Ian Hickson
cb378edc9e
Enable avoid_print lint. (#91332) 2021-10-07 09:53:03 -07:00
Greg Spencer
738ce43d97
Randomize tests, exclude tests that fail with randomization. (#86793)
* Randomize tests, exclude tests that fail with randomization.

* Disable some more tool tests
2021-07-26 23:40:49 -07:00
Alexandre Ardhuin
34059eec2c
enable lint prefer_interpolation_to_compose_strings (#83407) 2021-06-01 11:14:06 -07:00
Jia Hao
08154b22ce
Revert "Init licenses for test bindings (#81961)" (#82337) 2021-05-13 21:29:03 -07:00
Jia Hao
b3f7ebe069
Init licenses for test bindings (#81961) 2021-05-07 17:54:02 -07:00
Jia Hao
9e3eadb98d
[flutter_tools] Support Integration Tests (#76200) 2021-03-30 19:29:02 -07:00
Jia Hao
9e55af5246
[flutter_tools] Decouple FlutterPlatform from Process (#74236) 2021-02-17 10:06:04 -08:00
Jonah Williams
74bd7b6f6d
[flutter_tools] opt all flutter tool libraries and tests out of null safety. (#74832)
* opt out the flutter tool

* oops EOF

* fix import

* Update tool_backend.dart

* Update daemon_client.dart

* fix more
2021-01-27 15:17:53 -08:00
Jia Hao
22f1700427
[flutter_tools] Make setting of CWD consistent for flutter test (#74622) 2021-01-25 22:34:03 -08:00
Jonah Williams
f4e79e6828
[flutter_tools] add support for dart defines to flutter test (#70791) 2020-11-18 14:52:49 -08:00
Jonah Williams
13bf341584
[flutter_tools] update build rules to depend on subset of package_config contents (#67165)
Split from #66776

Even if pub does not change the packge_config contents, it will still update a timestamp in one of the fields. This causes unnecessary rebuilds. To fix this, generate an additional file when running pub get that only contains the relevant fields and then update the KernelSnapshot rule to depend on it only.
2020-10-07 08:46:11 -07:00