flutter/packages/flutter_tools/test/integration.shard
Martin Kustermann 227f6a04f7
Make native asset tests more robust against flutter upgrading its pinned dependencies (#159715)
Native asset tests use `flutter create --no-pub --template=package_ffi`.
The template used for this is checked in. It then adds extra
dependencies to checked-in packages in flutter/flutter (which have
pinned deps) in those generated templates.

It then pins all dependencies in the modified template project. This can
lead to constraint violations when flutter updates pinned dependencies,
because the template uses old constraints (which are turned from `^x` to
`=x`) and the additional dependency on flutter/flutter checked in
package brings in different pinned dependencies.

In a previous PR we already made this more robust by using flutter's
pinned versions over the the versions from the template (that are
changed from `^x` to `=x`).

Though a new upgrade of flutters pinned packages reveals that this isn't
quite sufficient: The template uses `test` at `^X`. The additional
dependency to `link_hook` doesn't depend on `test`. It therefore turns
it into `=X`. BUT `link_hooks` has a non-dev dependency on `test_core`
which is incompatible with `=X`.

=> So we relax this even more by prefering to choose (pinned) versions
of the flutter/flutter `link_hook` dependencies and (new) dev
dependencies over the template dependencies.
=> This will make use use the pinned `test` version from `link_hooks`
instead of from the template.
2024-12-03 12:21:59 +00:00
..
debug_adapter Roll pub packages (#156925) 2024-10-18 20:17:18 +00:00
isolated Make native asset tests more robust against flutter upgrading its pinned dependencies (#159715) 2024-12-03 12:21:59 +00:00
test_data Allow add_format() in flutter gen-l10n DateTime format (#156297) 2024-11-23 18:55:24 +00:00
analyze_all_templates_test.dart [flutter_tools] Use process matcher for multidex test (#127996) 2023-06-01 22:24:08 +00:00
analyze_once_test.dart Roll Flutter Engine from c58d87d62c20 to 31bb9f98472a (8 revisions) (#152555) 2024-07-30 16:42:10 +00:00
analyze_size_test.dart Fix output path for --appSizeBase (#158302) 2024-12-02 19:35:27 +00:00
analyze_suggestions_integration_test.dart Remove unused PubDependenciesProjectValidator. (#157516) 2024-10-24 22:05:50 +00:00
android_e2e_api_test.dart Remove now unused apk-health-tests. Can always re-add in the future. (#159349) 2024-11-23 00:25:46 +00:00
android_gradle_daemon_cache_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
android_gradle_deprecated_plugin_apply_test.dart Remove now unused apk-health-tests. Can always re-add in the future. (#159349) 2024-11-23 00:25:46 +00:00
android_gradle_flutter_source_path_test.dart Try a speculative fix for Gradle OOMs. (#159234) 2024-11-21 17:39:13 +00:00
android_gradle_java_version_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
android_gradle_legacy_flutter_plugins_strings_in_comments_test.dart Remove now unused apk-health-tests. Can always re-add in the future. (#159349) 2024-11-23 00:25:46 +00:00
android_gradle_outputs_app_link_settings_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
android_gradle_print_build_variants_test.dart Remove now unused apk-health-tests. Can always re-add in the future. (#159349) 2024-11-23 00:25:46 +00:00
android_plugin_compilesdkversion_mismatch_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
android_plugin_example_app_build_test.dart Remove now unused apk-health-tests. Can always re-add in the future. (#159349) 2024-11-23 00:25:46 +00:00
android_plugin_ndkversion_mismatch_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
android_plugin_new_output_dir_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
android_plugin_skip_unsupported_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
asset_transformation_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
background_isolate_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
bash_entrypoint_test.dart [flutter_tools] do not try to build tool from dart.sh (#129186) 2023-11-03 00:42:25 +00:00
batch_entrypoint_test.dart [flutter_tools] print out the unzipping method used by update_dart_sdk.ps1 (#133364) 2023-08-31 21:06:47 +00:00
break_on_framework_exceptions_test.dart Terminate flutter test when no longer needed in integration test. (#159117) 2024-11-19 16:16:08 +00:00
build_ios_config_only_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
build_macos_config_only_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
build_preview_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
cache_test.dart Remove redundant arguments passed to redirecting factory constructors (#115497) 2022-11-17 00:47:14 +00:00
command_output_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
coverage_collection_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
daemon_mode_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
debugger_stepping_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
deferred_components_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
deprecated_gradle_settings_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
devtools_uri_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
downgrade_upgrade_integration_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
exit_code_test.dart [flutter_tools] Migrate more integration tests to process result matcher (#128737) 2023-06-15 13:25:32 -07:00
expression_evaluation_test.dart [ tool ] Fix expression evaluation not handling errors correctly (#159151) 2024-11-25 15:04:09 +00:00
flutter_attach_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
flutter_build_android_app_project_builddir_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
flutter_build_apk_verbose_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
flutter_build_config_only_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
flutter_build_wasm_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
flutter_build_windows_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
flutter_build_with_compilation_error_test.dart Remove now unused apk-health-tests. Can always re-add in the future. (#159349) 2024-11-23 00:25:46 +00:00
flutter_gen_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
flutter_plugins_dependencies_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
flutter_run_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
flutter_run_with_error_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
forbidden_imports_test.dart Revert "Allow Flutter golden file tests to be flaky (#114450)" (#114902) 2022-11-08 09:26:55 -08:00
gen_l10n_test.dart Allow add_format() in flutter gen-l10n DateTime format (#156297) 2024-11-23 18:55:24 +00:00
gradle_non_android_plugin_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
hot_reload_errors_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
hot_reload_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
hot_reload_with_asset_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
lifetime_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
observatory_port_test.dart Do a clean shutdown of the flutter_tools processes started by observatory_port_test (#159169) 2024-11-21 19:04:06 +00:00
overall_experience_test.dart [Reland] Introduce double Flex.spacing parameter for Row/Column spacing (#152890) 2024-08-05 23:04:50 +00:00
plist_parser_test.dart Add Swift Package Manager as new opt-in feature for iOS and macOS (#146256) 2024-04-18 21:12:36 +00:00
README.md Added missing code block language in docs (#147481) 2024-05-01 14:44:27 +00:00
shader_compiler_test.dart Avoid depending on files from build_system/targets other than from top level entrypoints in flutter_tools. (#142760) 2024-02-02 18:23:08 +00:00
stateless_stateful_hot_reload_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
swift_package_manager_create_app_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
swift_package_manager_create_plugin_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
swift_package_manager_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
swift_package_manager_utils.dart [Swift Package Manager] Test removing the last Flutter plugin (#153519) 2024-08-16 14:03:13 +00:00
template_manifest_test.dart Simplify null check. (#117026) 2023-01-03 20:51:48 +00:00
test_driver.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
test_test.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
test_utils.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
timeline_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
tool_backend_test.dart Update flutter_tools/bin/*.(dart|sh) to provide, if set, --local-engine-host. (#132336) 2023-08-10 15:25:57 -07:00
transition_test_utils.dart Define and use flutterBin consistently across integration.shard. (#159007) 2024-11-18 19:00:04 +00:00
unit_coverage_test.dart [flutter_tools] migrate some integration tests to null safety (#103560) 2022-05-13 09:39:10 -07:00
variable_expansion_windows_test.dart [flutter_tools] migrate some integration tests to null safety (#103560) 2022-05-13 09:39:10 -07:00
variable_expansion_windows.dart Enable avoid_print lint. (#91444) 2021-10-07 16:48:04 -07:00
vmservice_integration_test.dart Add a tag and assert some state in FlutterTestDriver tests. (#159099) 2024-11-18 23:37:05 +00:00
web_plugin_registrant_test.dart Migrate more integration tests to process result matcher (#130994) 2023-07-20 21:02:12 +00:00
xcode_backend_test.dart [Reland] Skip injecting Bonjour settings when port publication is disabled (#136842) 2023-10-19 17:38:52 +00:00

Integration tests

These tests are not hermetic, and use the actual Flutter SDK. While they don't require actual devices, they run flutter_tester to test Dart VM and Flutter integration.

Use this command to run (from the flutter_tools directory):

../../bin/cache/dart-sdk/bin/dart run test test/integration.shard

You need to have downloaded the Dart SDK in your Flutter clone for this to work. Running ../../bin/flutter will automatically download it.

Coverage exclusion

These tests are expensive to run and do not give meaningful coverage information for the flutter tool (since they are black-box tests that run the tool as a subprocess, rather than being unit tests). For this reason, they are in a separate shard when running on continuous integration and are not run when calculating coverage.

Adding new test files

When adding a new test file make sure that it ends with _test.dart, or else it will not be run.