flutter/packages/flutter_tools/test/integration.shard
Victoria Ashworth e18c5e209c
Improve build time when using SwiftPM (#150052)
When using SwiftPM, we use `flutter assemble` in an Xcode Pre-action to run the `debug_unpack_macos` (or profile/release) target. This target is also later used in a Run Script build phase. Depending on `ARCHS` build setting, the Flutter/FlutterMacOS binary is thinned. In the Run Script build phase, `ARCHS` is filtered to the active arch. However, in the Pre-action it doesn't always filter to the active arch. As a workaround, assume arm64 if the [`NATIVE_ARCH`](https://developer.apple.com/documentation/xcode/build-settings-reference/#NATIVEARCH) is arm, otherwise assume x86_64.

Also, this PR adds a define flag `PreBuildAction`, which gives the Pre-action a [unique configuration of defines](fdb74fd3e7/packages/flutter_tools/lib/src/build_system/build_system.dart (L355-L372)) and therefore a separate filecache from the Run Script build phase filecache. This improves caching so the Run Script build phase action doesn't find missing targets in the filecache. It also uses this flag to skip cleaning up the previous build files.

Lastly, skip the Pre-action if the build command is `clean`.

Note: For iOS, if [`CodesignIdentity`](14df7be3f9/packages/flutter_tools/bin/xcode_backend.dart (L470-L473)) is set, the Pre-action and Run Script build phase will not match because the Pre-action does not send the `EXPANDED_CODE_SIGN_IDENTITY` and therefore will codesign it with [`-`](14df7be3f9/packages/flutter_tools/lib/src/build_system/targets/ios.dart (L695)) instead. This will cause `debug_unpack_macos` to invalidate and rerun every time. A potential solution would be to move [codesigning out of the Run Script build phase](14df7be3f9/packages/flutter_tools/lib/src/build_system/targets/ios.dart (L299)) to the [Thin Binary build phase](14df7be3f9/packages/flutter_tools/bin/xcode_backend.dart (L204-L257)) after it's copied into the TARGET_BUILD_DIR, like we do with [macOS](14df7be3f9/packages/flutter_tools/bin/macos_assemble.sh (L179-L183)).
2024-06-12 21:16:07 +00:00
..
debug_adapter Unpin DDS and roll pub packages (#147925) 2024-05-08 10:35:28 +01:00
isolated [native_assets] Add support for link hooks (#148474) 2024-05-22 16:02:00 +00:00
test_data Reland (2): "Fix how Gradle resolves Android plugin" (#142498) 2024-02-19 18:07:33 +00:00
analyze_all_templates_test.dart [flutter_tools] Use process matcher for multidex test (#127996) 2023-06-01 22:24:08 +00:00
analyze_once_test.dart [flutter_tools] update analyze_once_test.dart to be null-safe (#141790) 2024-01-21 07:54:05 +00:00
analyze_size_test.dart [flutter_tools] Use process matcher for multidex test (#127996) 2023-06-01 22:24:08 +00:00
analyze_suggestions_integration_test.dart [flutter_tools] cache flutter sdk version to disk (#124558) 2023-06-15 00:20:30 +00:00
android_e2e_api_test.dart [flutter_tools] Migrate more integration tests to process result matcher (#128737) 2023-06-15 13:25:32 -07:00
android_gradle_daemon_cache_test.dart Reland (2): "Fix how Gradle resolves Android plugin" (#142498) 2024-02-19 18:07:33 +00:00
android_gradle_deprecated_plugin_apply_test.dart Reland "Warn when Gradle plugins are applied using the legacy apply script method (#140103) 2023-12-19 16:03:23 +00:00
android_gradle_flutter_source_path_test.dart Set default flutter source directory for gradle builds (#142934) 2024-02-08 22:28:39 +00:00
android_gradle_java_version_test.dart [flutter_tools] Migrate more integration tests to process result matcher (#128737) 2023-06-15 13:25:32 -07:00
android_gradle_outputs_app_link_settings_test.dart Update the gradle task to add fallback scheme and host if needed when retrieving deep links. (#146470) 2024-04-11 14:24:51 -07:00
android_gradle_print_build_variants_test.dart Add vmservice for android build options (#123034) 2023-04-18 18:16:09 +00:00
android_plugin_compilesdkversion_mismatch_test.dart Improve Android SDK and NDK mistmatch warning message (#147809) 2024-05-07 15:11:00 +00:00
android_plugin_example_app_build_test.dart Restore log dumps for gradle OOM crashes, and set a value for MaxMetaspaceSize (#143085) 2024-02-07 19:25:39 +00:00
android_plugin_ndkversion_mismatch_test.dart Improve Android SDK and NDK mistmatch warning message (#147809) 2024-05-07 15:11:00 +00:00
android_plugin_new_output_dir_test.dart Change android_plugin_new_output_dir_test.dart test description (#149198) 2024-05-28 23:04:10 +00:00
android_plugin_skip_unsupported_test.dart Reland (2): "Fix how Gradle resolves Android plugin" (#142498) 2024-02-19 18:07:33 +00:00
asset_transformation_test.dart Add integration test for asset transformation feature (#145715) 2024-04-26 21:42:06 +00:00
background_isolate_test.dart
bash_entrypoint_test.dart [flutter_tools] do not try to build tool from dart.sh (#129186) 2023-11-03 00:42:25 +00:00
batch_entrypoint_test.dart [flutter_tools] print out the unzipping method used by update_dart_sdk.ps1 (#133364) 2023-08-31 21:06:47 +00:00
break_on_framework_exceptions_test.dart Eliminate more window singleton usages (#145560) 2024-03-21 20:59:25 +00:00
build_ios_config_only_test.dart [flutter_tools] Migrate more integration tests to process result matcher (#128737) 2023-06-15 13:25:32 -07:00
build_macos_config_only_test.dart Migrate more integration tests to process result matcher (#130994) 2023-07-20 21:02:12 +00:00
build_preview_test.dart [flutter_tools] move build_preview_test from commands/permeable to integration shard (#136912) 2023-10-19 22:03:04 +00:00
cache_test.dart
command_output_test.dart flutter config --list (#135401) 2023-09-27 07:02:13 +00:00
coverage_collection_test.dart
daemon_mode_test.dart
debugger_stepping_test.dart
deferred_components_test.dart Migrate more integration tests to process result matcher (#130994) 2023-07-20 21:02:12 +00:00
deprecated_gradle_settings_test.dart Fix implementation imports outside of lib (#143594) 2024-02-16 22:38:10 +00:00
devtools_uri_test.dart Reland "Fix issue where DevTools would not be immediately available when using --start-paused (#126698)" (#129368) 2023-06-28 00:16:13 +05:30
downgrade_upgrade_integration_test.dart [flutter_tools] Fix flutter upgrade not finding git tags (#133778) 2023-08-31 22:01:10 +00:00
exit_code_test.dart [flutter_tools] Migrate more integration tests to process result matcher (#128737) 2023-06-15 13:25:32 -07:00
expression_evaluation_test.dart Unskip expression evaluation test (#149253) 2024-05-29 23:24:36 +01:00
flutter_attach_test.dart Stop serving Observatory by default (#122419) 2023-03-23 12:52:30 -04:00
flutter_build_android_app_project_builddir_test.dart
flutter_build_apk_verbose_test.dart Update integration tests regexes. (#144847) 2024-03-12 01:31:25 +00:00
flutter_build_config_only_test.dart [flutter_tools] cache flutter sdk version to disk (#124558) 2023-06-15 00:20:30 +00:00
flutter_build_wasm_test.dart [flutter_tools] Enable WebAssembly compilation everywhere, remove feature flag (#145562) 2024-03-26 18:23:49 +00:00
flutter_build_windows_test.dart Enable native compilation for windows-arm64 (#141930) 2024-01-26 00:08:20 +00:00
flutter_build_with_compilation_error_test.dart [flutter_tools] Migrate more integration tests to process result matcher (#128737) 2023-06-15 13:25:32 -07:00
flutter_gen_test.dart
flutter_run_test.dart [flutter_tools] Include mode in app.start event, and forward app.start to DAP clients (#121239) 2023-04-05 05:39:09 +00:00
flutter_run_with_error_test.dart Reland "Remove references to Observatory (#118577)" (#121606) 2023-02-28 11:57:04 -05:00
forbidden_imports_test.dart
gen_l10n_test.dart Gen l10n add named argument option (#138663) 2024-01-05 21:28:08 +00:00
gradle_non_android_plugin_test.dart [flutter_tools] Use process matcher for multidex test (#127996) 2023-06-01 22:24:08 +00:00
hot_reload_errors_test.dart
hot_reload_test.dart add verbose logging to select hot reload/hot restart tests (#147673) 2024-05-02 04:03:32 +00:00
hot_reload_with_asset_test.dart
lifetime_test.dart
observatory_port_test.dart Add spaces after flow control statements (#126320) 2023-05-15 11:07:30 +02:00
overall_experience_test.dart Update integration tests regexes. (#144847) 2024-03-12 01:31:25 +00:00
plist_parser_test.dart Add Swift Package Manager as new opt-in feature for iOS and macOS (#146256) 2024-04-18 21:12:36 +00:00
README.md Added missing code block language in docs (#147481) 2024-05-01 14:44:27 +00:00
shader_compiler_test.dart Avoid depending on files from build_system/targets other than from top level entrypoints in flutter_tools. (#142760) 2024-02-02 18:23:08 +00:00
stateless_stateful_hot_reload_test.dart Remove test that verifies we can switch to stateless (#120390) 2023-02-09 20:58:57 +00:00
swift_package_manager_create_app_test.dart Add create app and plugin templates for Swift Package Manager (#147082) 2024-04-24 22:46:20 +00:00
swift_package_manager_create_plugin_test.dart Add create app and plugin templates for Swift Package Manager (#147082) 2024-04-24 22:46:20 +00:00
swift_package_manager_test.dart Improve build time when using SwiftPM (#150052) 2024-06-12 21:16:07 +00:00
swift_package_manager_utils.dart Improve build time when using SwiftPM (#150052) 2024-06-12 21:16:07 +00:00
template_manifest_test.dart
test_driver.dart add verbose logging to select hot reload/hot restart tests (#147673) 2024-05-02 04:03:32 +00:00
test_test.dart Support using lightweight Flutter Engines to run tests (#141726) 2024-02-22 13:32:29 -05:00
test_utils.dart Update dev/bots/test.dart (and friends) to provide --local-engine-host. (#132354) 2023-08-14 13:21:14 -07:00
timeline_test.dart
tool_backend_test.dart Update flutter_tools/bin/*.(dart|sh) to provide, if set, --local-engine-host. (#132336) 2023-08-10 15:25:57 -07:00
transition_test_utils.dart Implementing switch expressions in flutter_tools/ (#145632) 2024-03-29 22:31:19 +00:00
unit_coverage_test.dart
variable_expansion_windows_test.dart
variable_expansion_windows.dart
vmservice_integration_test.dart add verbose logging to select hot reload/hot restart tests (#147673) 2024-05-02 04:03:32 +00:00
web_plugin_registrant_test.dart Migrate more integration tests to process result matcher (#130994) 2023-07-20 21:02:12 +00:00
xcode_backend_test.dart [Reland] Skip injecting Bonjour settings when port publication is disabled (#136842) 2023-10-19 17:38:52 +00:00

Integration tests

These tests are not hermetic, and use the actual Flutter SDK. While they don't require actual devices, they run flutter_tester to test Dart VM and Flutter integration.

Use this command to run (from the flutter_tools directory):

../../bin/cache/dart-sdk/bin/dart run test test/integration.shard

You need to have downloaded the Dart SDK in your Flutter clone for this to work. Running ../../bin/flutter will automatically download it.

Coverage exclusion

These tests are expensive to run and do not give meaningful coverage information for the flutter tool (since they are black-box tests that run the tool as a subprocess, rather than being unit tests). For this reason, they are in a separate shard when running on continuous integration and are not run when calculating coverage.

Adding new test files

When adding a new test file make sure that it ends with _test.dart, or else it will not be run.