Relates to tracker issue:
- https://github.com/flutter/flutter/issues/128251
This PR includes 3 major updates:
- Adding the `commandHasTerminal` parameter for `Event.flutterCommandResult`
- In `packages/flutter_tools/lib/src/runner/flutter_command.dart`
- Adding the new event for `sendException` from package:usage to be `Event.exception` (this event can be used by all dash tools)
- In `packages/flutter_tools/lib/runner.dart`
- Migrating the generic `UsageEvent` which was only used for Apple related workflows for iOS and macOS. I did an initial analysis in this [sheet](https://docs.google.com/spreadsheets/d/11KJLkHXFpECMX7tw-trNkYSr5MHDG15XNGv6TgLjfQs/edit?resourcekey=0-j4qdvsOEEg3wQW79YlY1-g#gid=0) to identify all the call sites
- Found in several files, highlighted in the sheet above
This implements https://github.com/flutter/flutter/issues/132654#issuecomment-1738221257, namely:
Make `Chromium.close` more robust:
* Send `SIGTERM` and wait up to 5 seconds, if the process exited, great! Return from the function.
* If the process has not exited, then send a `SIGKILL`, which is a much firmer way to exit a process. Same as before, wait up to 5 seconds, if the process exited, great! Return from the function.
* If it still hasn't exited then give up trying to exit Chromium, just print a warning to the console and return from the function.
Bonus: a few nullability fixes and extra `-v` logging.
Fixes https://github.com/flutter/flutter/issues/132654
Support for FFI calls with `@Native external` functions through Native assets on Windows. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for Windows.
Mainly follows the design of https://github.com/flutter/flutter/pull/134031.
Specifically for Windows in this PR is the logic for finding the compiler `cl.exe` and environment variables that contain the paths to the Windows headers `vcvars.bat` based on `vswhere.exe`.
So there appears to be a race situation between the flutter CLI and Xcode. In the CLI, we update the `CONFIGURATION_BUILD_DIR` in the Xcode build settings and then tell Xcode to install, launch, and debug the app. When Xcode installs the app, it should use the `CONFIGURATION_BUILD_DIR` to find the bundle. However, it appears that sometimes Xcode hasn't processed the change to the build settings before the install happens, which causes it to not be able to find the bundle.
Fixes https://github.com/flutter/flutter/issues/135442
---
Since it's a timing issue, there's not really a consistent way to test it.
I was able to confirm that it works, though, by using the following steps:
1. Create a flutter project
2. Open the project in Xcode
3. `flutter clean`
4. `flutter run --profile -v`
If I saw a print line `stderr: CONFIGURATION_BUILD_DIR: build/Debug-iphoneos`, that means it first found the old and incorrect `CONFIGURATION_BUILD_DIR` before updating to the the new, so I was able to confirm that it would wait until it updated.
Xcode uses the CONFIGURATION_BUILD_DIR build setting to determine the location of the bundle to build and install. When launching an app via Xcode with the Xcode debug workflow (for iOS 17 physical devices), temporarily set the CONFIGURATION_BUILD_DIR to the location of the bundle so Xcode can find it.
Also, added a Xcode Debug version of the `microbenchmarks_ios` integration test since it uses `flutter run --profile` without using `--use-application-binary`.
Fixes https://github.com/flutter/flutter/issues/134186.
Support for FFI calls with `@Native external` functions through Native assets on MacOS and iOS. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for MacOS and iOS.
Dylibs are bundled by (1) making them fat binaries if multiple architectures are targeted, (2) code signing these, and (3) copying them to the frameworks folder. These steps are done manual rather than via CocoaPods. CocoaPods would have done the same steps, but (a) needs the dylibs to be there before the `xcodebuild` invocation (we could trick it, by having a minimal dylib in the place and replace it during the build process, that works), and (b) can't deal with having no dylibs to be bundled (we'd have to bundle a dummy dylib or include some dummy C code in the build file).
The dylibs are build as a new target inside flutter assemble, as that is the moment we know what build-mode and architecture to target.
The mapping from asset id to dylib-path is passed in to every kernel compilation path. The interesting case is hot-restart where the initial kernel file is compiled by the "inner" flutter assemble, while after hot restart the "outer" flutter run compiled kernel file is pushed to the device. Both kernel files need to contain the mapping. The "inner" flutter assemble gets its mapping from the NativeAssets target which builds the native assets. The "outer" flutter run get its mapping from a dry-run invocation. Since this hot restart can be used for multiple target devices (`flutter run -d all`) it contains the mapping for all known targets.
### Example vs template
The PR includes a new template that uses the new native assets in a package and has an app importing that. Separate discussion in: https://github.com/flutter/flutter/issues/131209.
### Tests
This PR adds new tests to cover the various use cases.
* dev/devicelab/bin/tasks/native_assets_ios.dart
* Runs an example app with native assets in all build modes, doing hot reload and hot restart in debug mode.
* dev/devicelab/bin/tasks/native_assets_ios_simulator.dart
* Runs an example app with native assets, doing hot reload and hot restart.
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
* Runs (incl hot reload/hot restart), builds, builds frameworks for iOS, MacOS and flutter-tester.
* packages/flutter_tools/test/general.shard/build_system/targets/native_assets_test.dart
* Unit tests the new Target in the backend.
* packages/flutter_tools/test/general.shard/ios/native_assets_test.dart
* packages/flutter_tools/test/general.shard/macos/native_assets_test.dart
* Unit tests the native assets being packaged on a iOS/MacOS build.
It also extends various existing tests:
* dev/devicelab/bin/tasks/module_test_ios.dart
* Exercises the add2app scenario.
* packages/flutter_tools/test/general.shard/features_test.dart
* Unit test the new feature flag.
Sometimes `ios-deploy` loses connection to the device after installing, starting debugserver, and launching. This is shown with an error message like:
```
Process 579 exited with status = -1 (0xffffffff) lost connection
```
This happens frequently in our CI system: https://github.com/flutter/flutter/issues/120808
Usually in CI, on retry it'll work and pass - so this is an attempt to retry without failing the test first. It's not guaranteed to fix since we're unable to recreate this error locally.
Starting in Xcode 15, when building macOS, DT_TOOLCHAIN_DIR cannot be used to evaluate LD_RUNPATH_SEARCH_PATHS or LIBRARY_SEARCH_PATHS. `xcodebuild` error message recommend using TOOLCHAIN_DIR instead.
Since Xcode 15 isn't in CI, I tested it in a one-off `led` test:
* [Pre-fix failure](04e485a0b1/+/build.proto)
* [Post-fix success](d454a3e181/+/build.proto)
Fixes https://github.com/flutter/flutter/issues/132755.
Partial work towards https://github.com/flutter/flutter/issues/132245.
I made a minor refactor to test-only code because it was too confusing
to have 2 optional parameters that are technically required together,
but otherwise all other changes *should* be pass throughs. That being
said, I can't say I totally understand the Gradle stuff so I could use a
hand double checking that.
Fixes a couple of issues introduced in new iOS 17 physical device tooling: https://github.com/flutter/flutter/pull/131865.
1) Duplicate messages were being filtered out too aggressively.
For example, if on the counter app, you printed "Increment!" on button click, it would only print once no matter how many times you clicked.
Sometimes more than one log source is used at a time and the original intention was to filter duplicates between two log sources, so it wouldn't print the same message from both logs. However, it would also filter when the same message was added more than once via the same log.
The new solution distinguishes a "primary" and a "fallback" log source and prefers to use the primary source unless it's not working, in which it'll use the fallback. If the fallback is faster than the primary, the primary will exclude the logs received by the fallback in a 1-to-1 fashion to prevent too-aggressive filtering. Once a flutter-message has been received by the primary source, fallback messages will be ignored.
Note: iOS < 17 did not regress.
2) There was a race condition between the shutdown hooks and exiting XcodeDebug that was causing a crash when deleting a file that doesn't exist. This only affects CI - for the new integration tests and when testing with iOS 17 physical devices.
This PR includes the following changes. These changes only apply to iOS 17 physical devices.
| Command | Change Description | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode. | No change. |
| `flutter run` | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app. | No change. |
| `flutter screenshot` | Will return error. | Will return error. |
Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs
Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.
Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
Redo of https://github.com/flutter/flutter/pull/130728 - code is the same as before. That PR was stuck in Google testing and then I messed up the rebase so started over.
----
Starting in Xcode 15, the simulator is no longer included in Xcode and must be downloaded and installed separately. This adds a validation to `flutter doctor` to warn when the needed simulator runtime is missing.
Validation message looks like:
```
[!] Xcode - develop for iOS and macOS (Xcode 15.0)
! iOS 17.0 Simulator not installed; this may be necessary for iOS and macOS development.
To download and install the platform, open Xcode, select Xcode > Settings > Platforms,
and click the GET button for the required platform.
For more information, please visit:
https://developer.apple.com/documentation/xcode/installing-additional-simulator-runtimes
```
It may also show an error like this when something goes wrong when checking for the simulator:
```
[!] Xcode - develop for iOS and macOS (Xcode 15.0)
â Unable to find the iPhone Simulator SDK.
```
Note: I'm unsure of in the future if the SDK and the simulator runtime will need to match the exact version or just the major. For now, it only checks against the major version.
Part 3 of https://github.com/flutter/flutter/issues/129558.
Starting in Xcode 15, the simulator is no longer included in Xcode and must be downloaded and installed separately.
If you try to run flutter and the simulator is missing, you'll get an error like
```
xcodebuild: error: Unable to find a destination matching the provided destination specifier:
{ id:B1234A5C-67B8-901D-B2CB-FE34F56BDE78 }
Ineligible destinations for the "Runner" scheme:
{ platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device, error:iOS 17.0 is not installed. To use with Xcode, first download and install the platform }
```
Print a pretty error to make it easier for developers to know what to do.
Part 2 of https://github.com/flutter/flutter/issues/129558.
Adding debugging for https://github.com/flutter/flutter/issues/129836.
Takes a screenshot when startup test takes too long (10 minutes).
Also, removes some old debugging and add new debugging message.
This PR fixes issue of duplicate entries from `xcdevice list` cause devices to not show in `flutter devices`, `flutter run`, etc.
When a duplicate entry is found, use the entry without errors as the authority. If both have errors, use the one with the higher SDK as the authority.
Fixes https://github.com/flutter/flutter/issues/128719.
This reverts commit a19b3436ee.
We added this logging to try and determine if the reason for Dart VM errors (https://github.com/flutter/flutter/issues/121231) was caused by some issue with the streams.
A recent test proves that is not the case:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_ios%20platform_view_ios__start_up/11046/overview
The test shows the Dart VM url in the device log. However, the test log does NOT show a log for the Dart VM url but does show the stack trace, which all come from the main stream, which means it's not an issue with the secondary streams not receiving the log.
So reverting the debugging we added.
Log all output of `ios-deploy` to try and determine if the issue of https://github.com/flutter/flutter/issues/121231 is with stream or with `ios-deploy`.
Note: This will cause some duplicate logs like example below but only in verbose mode
```
(lldb) 2023-05-19 13:48:19.107935-0500 Runner[2521:390363] [VERBOSE-2:FlutterDarwinContextMetalImpeller.mm(35)] Using the Impeller rendering backend.
(lldb) 2023-05-19 13:48:19.107935-0500 Runner[2521:390363] [VERBOSE-2:FlutterDarwinContextMetalImpeller.mm(35)] Using the Impeller rendering backend.
2023-05-19 13:48:19.156866-0500 Runner[2521:390612] flutter: The Dart VM service is listening on http://127.0.0.1:63508/IsFnhXJykCM=/
VM Service URL on device: http://127.0.0.1:63508/IsFnhXJykCM=/
```
Check what is available in the device's iOS DeviceSupport folder to check if symbols were properly fetched. Also, add some logging to track what status the debugger is in.
Debugging for https://github.com/flutter/flutter/issues/121231.
Adds the ability to rename Runner.xcodeproj and Runner.xcworkspace - fixes https://github.com/flutter/flutter/issues/9767.
To rename a project:
1. Open Runner.xcodeproj in Xcode
2. In the left panel, left click "Show File Inspector"
<img width="441" alt="Screenshot 2023-04-17 at 11 41 07 PM" src="https://user-images.githubusercontent.com/36148254/232692957-8743742d-c3ef-42e5-833f-dff31aeb2b6a.png">
3. In the right panel, the name of the project, "Runner", should be visible under "Identity and Type". Change the name and press enter.
<img width="299" alt="Screenshot 2023-04-17 at 11 40 43 PM" src="https://user-images.githubusercontent.com/36148254/232693315-b6a71165-f5e3-4a0f-8954-2f3eee5b67cf.png">
4. A wizard should pop up. Click Rename.
<img width="573" alt="Screenshot 2023-04-17 at 11 44 01 PM" src="https://user-images.githubusercontent.com/36148254/232693381-bb9cf026-2a75-4844-b42d-ae0036ae9fdd.png">
To rename the workspace:
1. Make sure Xcode is closed.
2. Rename the .xcworkspace to your new name.
If you also renamed the project
3. Reopen the .xcworkspace in Xcode. If the selected project is the old name and in red, update it to match the new project name.
Tests for schemeFor were changed as with Xcode 14, in some cases the scheme will be renamed along with the project. Thus we will get the best match scheme for either the project name, or the default name Runner. However if a flavor is present, the scheme should always match the flavor.
1. Add iOS and macOS migration to mark "last upgraded" Xcode version to 14.3 to prevent `Update to recommended settings` warning.
2. Update iOS and macOS templates to same.
3. Update iOS template to set `BuildIndependentTargetsInParallel` to YES as suggested. I didn't add a migration for this since it seems like a minor optimization and I don't think it's worth a potentially botched/corrupted migration.
4. Run all example/integration test project to see migrator work.
5. Add some missing test projects to the build shard since I noticed they were missing and I had to build those manually outside `SHARD=build_tests`.
Fixes https://github.com/flutter/flutter/issues/125817
See https://github.com/flutter/flutter/pull/90304 for Xcode 13 example.
Rename variables, update comments, etc from `network` to `wireless` to keep it more uniform.
Also, move non-overriden messages related to device selection into the file they're used.
Part 7 in breakdown of https://github.com/flutter/flutter/pull/121262.
