flutter/packages/flutter_tools/test/commands.shard/hermetic
Victoria Ashworth 529a4d2bac
Disable sandboxing for macOS apps and tests in CI (#149618)
macOS 14 added new requirements that un-codesigned sandbox apps must be granted access when changed. Waiting for this UI caused macOS tests to fail on macOS 14 because the test runner forced codesigning off. Additionally, adding codesigning is not sufficient, since it must still be approved before codesigning is enough to pass the check. As a workaround, this PR disables sandboxing for macOS apps/tests in CI.

![Screenshot 2024-05-30 at 2 41 33 PM](https://github.com/flutter/flutter/assets/682784/1bc32620-5edb-420a-866c-5cc529b2ac55)

https://developer.apple.com/documentation/updates/security#June-2023)
> App Sandbox now associates your macOS app with its sandbox container using its code signature. The operating system asks the person using your app to grant permission if it tries to access a sandbox container associated with a different app. For more information, see [Accessing files from the macOS App Sandbox](https://developer.apple.com/documentation/security/app_sandbox/accessing_files_from_the_macos_app_sandbox).

And that link explains why this is happening on a macOS 14 update:

> In macOS 14 and later, the operating system uses your app’s code signature to associate it with its sandbox container. If your app tries to access the sandbox container owned by another app, the system asks the person using your app whether to grant access. If the person denies access and your app is already running, then it can’t read or write the files in the other app’s sandbox container. If the person denies access while your app is launching and trying to enter the other app’s sandbox container, your app fails to launch.
> 
> The operating system also tracks the association between an app’s code signing identity and its sandbox container for helper tools, including launch agents. If a person denies permission for a launch agent to enter its sandbox container and the app fails to start, launchd starts the launch agent again and the operating system re-requests access.

Fixes https://github.com/flutter/flutter/issues/149268.
Fixes framework part of https://github.com/flutter/flutter/issues/149264.
Might fix packages issue: https://github.com/flutter/flutter/issues/149329.

Verified framework tests:
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac%20plugin_test_macos/9/overview
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac%20run_debug_test_macos/2/overview
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac%20tool_integration_tests_4_4/2/overview
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac%20integration_ui_test_test_macos/3/overview
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac%20flavors_test_macos/3/overview
https://ci.chromium.org/ui/p/flutter/builders/staging.shadow/Mac_benchmark%20complex_layout_scroll_perf_macos__timeline_summary/6/overview
2024-06-04 21:51:19 +00:00
..
analyze_continuously_test.dart Enable private field promotion for flutter_tools (#134474) 2023-09-12 20:20:04 +00:00
analyze_suggestion_test.dart Fix formatting (#137613) 2023-10-31 13:27:26 +00:00
analyze_test.dart Make --flutter-repo analyze whole repo (#127990) 2023-06-02 17:17:54 +00:00
android_analyze_test.dart Improves output file path logic in Android analyze (#136981) 2023-11-16 22:59:02 +00:00
assemble_test.dart Migrate command usage values (#139383) 2023-12-07 16:12:03 +00:00
attach_test.dart Support mdns when attaching to proxied devices. (#146021) 2024-04-09 20:00:21 +00:00
build_aar_test.dart provide command to FakeCommand::onRun (#142206) 2024-01-25 07:51:25 +00:00
build_darwin_framework_test.dart Fix embedding FlutterMacOS.framework for macOS add2app via cocoapods (#144248) 2024-03-05 21:59:49 +00:00
build_ios_test.dart Bump dependencies in Flutter (#147546) 2024-04-29 15:53:25 -07:00
build_ipa_test.dart Bump dependencies in Flutter (#147546) 2024-04-29 15:53:25 -07:00
build_linux_test.dart Reland #128236 "Improve build output for all platforms" (#145495) 2024-03-23 12:05:21 +00:00
build_macos_test.dart Disable sandboxing for macOS apps and tests in CI (#149618) 2024-06-04 21:51:19 +00:00
build_test.dart Flutter preview device (#135639) 2023-10-18 00:27:54 +00:00
build_web_test.dart Reland #128236 "Improve build output for all platforms" (#145495) 2024-03-23 12:05:21 +00:00
build_windows_test.dart Bump dependencies in Flutter (#147546) 2024-04-29 15:53:25 -07:00
clean_test.dart Add Swift Package Manager as new opt-in feature for iOS and macOS (#146256) 2024-04-18 21:12:36 +00:00
config_test.dart [Reland] Migration for the sendTiming events for package:unified_analytics (#139299) 2023-12-04 21:39:18 +00:00
create_usage_test.dart Add create app and plugin templates for Swift Package Manager (#147082) 2024-04-24 22:46:20 +00:00
custom_devices_test.dart Remove outdated ignores from tool (#140467) 2023-12-20 22:14:32 +00:00
daemon_test.dart Add a --print-dtd flag to print the DTD address served by DevTools server (#144272) 2024-03-25 13:04:18 -07:00
devices_test.dart Control flow collections: flutter_tools/ (#147450) 2024-05-02 22:19:18 +00:00
doctor_test.dart Remove outdated ignores from tool (#140467) 2023-12-20 22:14:32 +00:00
downgrade_test.dart Give channel descriptions in flutter channel, use branch instead of upstream for channel name (#126936) 2023-05-23 19:59:20 +00:00
drive_test.dart Upload DerivedData logs in CI (#142643) 2024-02-01 21:31:28 +00:00
flutter_web_platform_test.dart flutter test --wasm support (#145347) 2024-03-21 20:08:07 +00:00
generate_localizations_test.dart Enable private field promotion for flutter_tools (#134474) 2023-09-12 20:20:04 +00:00
http_host_validator_test.dart Improve network resources doctor check (#120417) 2023-02-15 00:02:09 +00:00
ide_config_test.dart Null safety migration of packages/flutter_tools/test/commands.shard/hermetic, part 2/3 (#110708) 2022-09-06 10:39:18 -07:00
install_test.dart Remove outdated ignores from tool (#140467) 2023-12-20 22:14:32 +00:00
ios_analyze_test.dart Makes scheme and target optional parameter when getting universal lin… (#134571) 2023-09-15 21:01:05 +00:00
logs_test.dart [flutter_tools] Fix bad state future already completed in flutter logs (#138517) 2023-11-23 19:56:14 +00:00
precache_test.dart
proxied_devices_test.dart Update proxied devices to handle connection interface and diagnostics. (#145061) 2024-03-13 22:24:25 +00:00
proxy_validator_test.dart
pub_test.dart Remove embedding v1 code in framework (#144726) 2024-03-20 20:34:14 +00:00
run_test.dart Disable sandboxing for macOS apps and tests in CI (#149618) 2024-06-04 21:51:19 +00:00
screenshot_command_test.dart Enable flutter screenshot outside Flutter project directory (#138160) 2023-11-17 01:08:13 +00:00
shell_completion_test.dart
symbolize_test.dart Prepare for utf8.encode() to return more precise Uint8List type (#129769) 2023-06-29 22:44:57 +02:00
test_test.dart Support failures-only and silent reporters in flutter test (#148739) 2024-06-03 19:44:39 +00:00
update_packages_test.dart Copy the flutter version JSON file into the simulated Flutter SDK used by update_packages (#143035) 2024-02-06 20:00:08 -08:00
upgrade_test.dart Reland: Update link branches to main (#146882) 2024-04-17 13:16:33 +00:00