mirror of
https://github.com/flutter/flutter.git
synced 2025-06-03 00:51:18 +00:00
dfb5888e8f
This PR implements the functionality described above and hides it behind the `--experimental-faster-testing` flag of `flutter test`. ### The following are some performance measurements from test runs conducted on GitHub Actions run 1 logs: https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/1 run 2 logs: https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/2 run 3 logs: https://github.com/derekxu16/flutter_test_ci/actions/runs/8008029772/attempts/3 **length of `flutter test --reporter=expanded test/animation test/foundation` step** run 1: 54s run 2: 52s run 3: 56s average: 54s **length of `flutter test --experimental-faster-testing --reporter=expanded test/animation test/foundation` step** run 1: 27s run 2: 27s run 3: 29s average: 27.67s (~48.77% shorter than 54s) **length of `flutter test --reporter=expanded test/animation test/foundation test/gestures test/painting test/physics test/rendering test/scheduler test/semantics test/services` step** run 1: 260s run 2: 270s run 3: 305s average: 278.33s **length of `flutter test --experimental-faster-testing --reporter=expanded test/animation test/foundation test/gestures test/painting test/physics test/rendering test/scheduler test/semantics test/services` step** from a clean build (right after deleting the build folder): run 1: 215s run 2: 227s run 3: 245s average: 229s (~17.72% shorter than 278.33s) Note that in reality, `test/material` was not passed to `flutter test` in the trials below. All of the test files under `test/material` except for `test/material/icons_test.dart` were listed out individually **length of `flutter test --reporter=expanded test/material` step** run 1: 408s run 2: 421s run 3: 451s average: 426.67s **length of `flutter test --experimental-faster-testing --reporter=expanded test/material` step** run 1: 382s run 2: 373s run 3: 400s average: 385s (~9.77% shorter than 426.67s) --------- Co-authored-by: Dan Field <dnfield@google.com> |
||
|---|---|---|
| .. | ||
| debug_adapter | ||
| isolated | ||
| test_data | ||
| analyze_all_templates_test.dart | ||
| analyze_once_test.dart | ||
| analyze_size_test.dart | ||
| analyze_suggestions_integration_test.dart | ||
| android_e2e_api_test.dart | ||
| android_gradle_daemon_cache_test.dart | ||
| android_gradle_deprecated_plugin_apply_test.dart | ||
| android_gradle_flutter_source_path_test.dart | ||
| android_gradle_java_version_test.dart | ||
| android_gradle_outputs_app_link_settings_test.dart | ||
| android_gradle_print_build_variants_test.dart | ||
| android_plugin_compilesdkversion_mismatch_test.dart | ||
| android_plugin_example_app_build_test.dart | ||
| android_plugin_ndkversion_mismatch_test.dart | ||
| android_plugin_new_output_dir_test.dart | ||
| android_plugin_skip_unsupported_test.dart | ||
| background_isolate_test.dart | ||
| bash_entrypoint_test.dart | ||
| batch_entrypoint_test.dart | ||
| break_on_framework_exceptions_test.dart | ||
| build_ios_config_only_test.dart | ||
| build_macos_config_only_test.dart | ||
| build_preview_test.dart | ||
| cache_test.dart | ||
| command_output_test.dart | ||
| coverage_collection_test.dart | ||
| daemon_mode_test.dart | ||
| debugger_stepping_test.dart | ||
| deferred_components_test.dart | ||
| deprecated_gradle_settings_test.dart | ||
| devtools_uri_test.dart | ||
| downgrade_upgrade_integration_test.dart | ||
| exit_code_test.dart | ||
| expression_evaluation_test.dart | ||
| flutter_attach_test.dart | ||
| flutter_build_android_app_project_builddir_test.dart | ||
| flutter_build_apk_verbose_test.dart | ||
| flutter_build_config_only_test.dart | ||
| flutter_build_wasm_test.dart | ||
| flutter_build_windows_test.dart | ||
| flutter_build_with_compilation_error_test.dart | ||
| flutter_gen_test.dart | ||
| flutter_run_test.dart | ||
| flutter_run_with_error_test.dart | ||
| forbidden_imports_test.dart | ||
| gen_l10n_test.dart | ||
| gradle_non_android_plugin_test.dart | ||
| hot_reload_errors_test.dart | ||
| hot_reload_test.dart | ||
| hot_reload_with_asset_test.dart | ||
| lifetime_test.dart | ||
| observatory_port_test.dart | ||
| overall_experience_test.dart | ||
| plist_parser_test.dart | ||
| README.md | ||
| shader_compiler_test.dart | ||
| stateless_stateful_hot_reload_test.dart | ||
| template_manifest_test.dart | ||
| test_driver.dart | ||
| test_test.dart | ||
| test_utils.dart | ||
| timeline_test.dart | ||
| tool_backend_test.dart | ||
| transition_test_utils.dart | ||
| unit_coverage_test.dart | ||
| variable_expansion_windows_test.dart | ||
| variable_expansion_windows.dart | ||
| vmservice_integration_test.dart | ||
| web_plugin_registrant_test.dart | ||
| xcode_backend_test.dart | ||
Integration tests
These tests are not hermetic, and use the actual Flutter SDK. While
they don't require actual devices, they run flutter_tester to test
Dart VM and Flutter integration.
Use this command to run (from the flutter_tools directory):
../../bin/cache/dart-sdk/bin/dart run test test/integration.shard
You need to have downloaded the Dart SDK in your Flutter clone for this
to work. Running ../../bin/flutter will automatically download it.
Coverage exclusion
These tests are expensive to run and do not give meaningful coverage
information for the flutter tool (since they are black-box tests that
run the tool as a subprocess, rather than being unit tests). For this
reason, they are in a separate shard when running on continuous
integration and are not run when calculating coverage.
Adding new test files
When adding a new test file make sure that it ends with _test.dart, or else it will not be run.