Work towards https://github.com/flutter/flutter/issues/142178.
---
This PR makes no _behavioral_ changes to executed code, and instead
focuses on organization and naming:
1. Extended the README to explain the intent of the test, as well as how
to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart`
and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is
being asserted (i.e. `widgetBuilds` and friends)
Reverts flutter/flutter#142062
Initiated by: eliasyishak
This change reverts the following previous change:
Original Description:
This PR makes no _behavioral_ changes to executed code, and instead focuses on organization and naming:
1. Almost[^1] anything named `external_ui` is renamed `external_textures`
1. Extended the README to explain the intent of the test, as well as how to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart` and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is being asserted (i.e. `widgetBuilds` and friends)
Given how complex (and in-flux) this directory is, I'm also requesting either John, Jonah or I review any changes.
[^1]: Except the name of the `.ci.yaml` task, i.e. `name: Linux_pixel_7pro external_ui_integration_test` because I'm apparently not able to change that without creating a new task as `bringup: true` and playing a bit of a dance. Maybe that's worth doing though (in future PRs)?
This PR makes no _behavioral_ changes to executed code, and instead
focuses on organization and naming:
1. Almost[^1] anything named `external_ui` is renamed
`external_textures`
1. Extended the README to explain the intent of the test, as well as how
to run it
1. Renamed `main.dart` and `main_test.dart` to `frame_rate_main.dart`
and `frame_rate_test.dart` (we'll add more)
1. Did some refactoring of the test to make it more obvious what is
being asserted (i.e. `widgetBuilds` and friends)
Given how complex (and in-flux) this directory is, I'm also requesting
either John, Jonah or I review any changes.
[^1]: Except the name of the `.ci.yaml` task, i.e. `name:
Linux_pixel_7pro external_ui_integration_test` because I'm apparently
not able to change that without creating a new task as `bringup: true`
and playing a bit of a dance. Maybe that's worth doing though (in future
PRs)?
Reverts flutter/flutter#137618
Initiated by: Jasguerrero
This change reverts the following previous change:
Original Description:
It's now possible to natively compile a flutter app for
windows-arm64. Cross-compilation is not yet implemented.
Uses arm64 artifacts now available for Dart/Flutter.
Platform detection is based on Abi class, provided by Dart. Depending if
Dart is an arm64 or x64 binary, the Abi is set accordingly.
Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking
PROCESSOR_ARCHITECTURE environment variable, which is the way to detect
host architecture on Windows.
This is available only for master channel (on other channels, it
fallbacks to windows-x64).
On windows-x64, it produces an x64 app. On windows-arm64, it produces an
arm64 app.
It's now possible to natively compile a flutter app for
windows-arm64. Cross-compilation is not yet implemented.
Uses arm64 artifacts now available for Dart/Flutter.
Platform detection is based on Abi class, provided by Dart. Depending if
Dart is an arm64 or x64 binary, the Abi is set accordingly.
Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking
PROCESSOR_ARCHITECTURE environment variable, which is the way to detect
host architecture on Windows.
This is available only for master channel (on other channels, it
fallbacks to windows-x64).
On windows-x64, it produces an x64 app. On windows-arm64, it produces an
arm64 app.
Currently podhelper.rb will always point plugin builds at the cached engine artifacts, even when using `--local-engine`. In most cases this is fine, since when the final build actually runs it will be using the engine bundled into the app build, which will be the correct local engine build. When trying to test a local engine build with API additions against a local plugin modified to use those additions to ensure that they are working as expected, however, compilation will fail, because the new APIs won't be present in the plugin build.
This fixes that for macOS, and adds a TODO for iOS (which is more complicated to fix due to the host vs target build distinction).
macOS portion of https://github.com/flutter/flutter/issues/132228
ObjC->Swift plugin migration caused a size regression in the gallery app because the Swift runtime was also pulled in.
The gallery app minimum target version is iOS 11.0, which predates Swift ABI compatibility. Pre iOS 12.2 apps embedded the Swift runtime since there wasn't one available to use in the OS.
Add `FLUTTER_XCODE_IPHONEOS_DEPLOYMENT_TARGET` to the compile perf test environment, which gets translated by the tool to an Xcode build setting:
```
[2023-12-14 15:52:14.797318] [STDOUT] stdout: IPHONEOS_DEPLOYMENT_TARGET = 12.2
```
On my machine on main
```
"release_size_bytes": 43717389,
```
becomes
```
"release_size_bytes": 40679432,
```
Fixes https://github.com/flutter/flutter/issues/139605
I previously made a PR (#136140) that used `switch` expressions to make some parts of the Flutter codebase easier to understand. It was assigned to the framework team, and @christopherfujino let me know that it was too large to effectively review and recommended breaking it up into smaller pull requests.
Here's a PR that only targets files in the `dev/` directory. Hopefully this will be easier to work with!
(solves issue https://github.com/flutter/flutter/issues/136139)
Partial repaint is too effective, and we'd like to be able to measure performance without carefully structuring the benchmarks. For example, right now partial repaint is culling any blurs in the multibackdrop case, which we should be using to track https://github.com/flutter/flutter/issues/132735
Pin the dependencies from dart-lang/native to a specific version during testing (rather than having them auto-upgrade during pub resolution). This will prevent tests using the template to start failing if a bad version is published to pub.
Closes: https://github.com/flutter/flutter/issues/137418
Also bumps dep in flutter_tools.
Adds the metadata key required to enable OpenGLES GPU tracing. This is off by default because the API crashes on some GPU models, but it should be safe on the Pixel 7 (others TBD based on testing results).
Reverts flutter/flutter#136562
Initiated by: vashworth
This change reverts the following previous change:
Original Description:
Some of our tests in CI are triggering the `NSLocalNetworkUsageDescription` dialog when they're not supposed to (https://github.com/flutter/flutter/issues/129836) since it's disabled via flags (`--no-publish-port` for flutter/flutter and `--disable-vm-service-publication` for flutter/engine).
Normally, we inject `NSLocalNetworkUsageDescription` (and other bonjour settings) to the Info.plist during the project build for debug and profile mode since by default they will publish the VM Service port over mDNS.
To help diagnose the issue, though, this PR changes it so that we don't inject `NSLocalNetworkUsageDescription` (and other bonjour settings) when port publication is disabled since it shouldn't be needed. Hopefully, this will give us better error messages or cause the app to crash and end the test early (rather than timeout after 30 minutes).
Some of our tests in CI are triggering the `NSLocalNetworkUsageDescription` dialog when they're not supposed to (https://github.com/flutter/flutter/issues/129836) since it's disabled via flags (`--no-publish-port` for flutter/flutter and `--disable-vm-service-publication` for flutter/engine).
Normally, we inject `NSLocalNetworkUsageDescription` (and other bonjour settings) to the Info.plist during the project build for debug and profile mode since by default they will publish the VM Service port over mDNS.
To help diagnose the issue, though, this PR changes it so that we don't inject `NSLocalNetworkUsageDescription` (and other bonjour settings) when port publication is disabled since it shouldn't be needed. Hopefully, this will give us better error messages or cause the app to crash and end the test early (rather than timeout after 30 minutes).
These values are generated since https://github.com/flutter/flutter/pull/135279, but I didn't know to add the new keys to this list to get them to upload.
Failed to do so in #135645, I believe the mistake there was putting them in `_kCommonScoreKeys`, which is also used in "E2E" tests, that don't get full trace data, only high level `FrameTiming` packets.
Part of https://github.com/flutter/flutter/issues/129150
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/121420
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Xcode uses the CONFIGURATION_BUILD_DIR build setting to determine the location of the bundle to build and install. When launching an app via Xcode with the Xcode debug workflow (for iOS 17 physical devices), temporarily set the CONFIGURATION_BUILD_DIR to the location of the bundle so Xcode can find it.
Also, added a Xcode Debug version of the `microbenchmarks_ios` integration test since it uses `flutter run --profile` without using `--use-application-binary`.
Fixes https://github.com/flutter/flutter/issues/134186.
Support for FFI calls with `@Native external` functions through Native assets on MacOS and iOS. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for MacOS and iOS.
Dylibs are bundled by (1) making them fat binaries if multiple architectures are targeted, (2) code signing these, and (3) copying them to the frameworks folder. These steps are done manual rather than via CocoaPods. CocoaPods would have done the same steps, but (a) needs the dylibs to be there before the `xcodebuild` invocation (we could trick it, by having a minimal dylib in the place and replace it during the build process, that works), and (b) can't deal with having no dylibs to be bundled (we'd have to bundle a dummy dylib or include some dummy C code in the build file).
The dylibs are build as a new target inside flutter assemble, as that is the moment we know what build-mode and architecture to target.
The mapping from asset id to dylib-path is passed in to every kernel compilation path. The interesting case is hot-restart where the initial kernel file is compiled by the "inner" flutter assemble, while after hot restart the "outer" flutter run compiled kernel file is pushed to the device. Both kernel files need to contain the mapping. The "inner" flutter assemble gets its mapping from the NativeAssets target which builds the native assets. The "outer" flutter run get its mapping from a dry-run invocation. Since this hot restart can be used for multiple target devices (`flutter run -d all`) it contains the mapping for all known targets.
### Example vs template
The PR includes a new template that uses the new native assets in a package and has an app importing that. Separate discussion in: https://github.com/flutter/flutter/issues/131209.
### Tests
This PR adds new tests to cover the various use cases.
* dev/devicelab/bin/tasks/native_assets_ios.dart
* Runs an example app with native assets in all build modes, doing hot reload and hot restart in debug mode.
* dev/devicelab/bin/tasks/native_assets_ios_simulator.dart
* Runs an example app with native assets, doing hot reload and hot restart.
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
* Runs (incl hot reload/hot restart), builds, builds frameworks for iOS, MacOS and flutter-tester.
* packages/flutter_tools/test/general.shard/build_system/targets/native_assets_test.dart
* Unit tests the new Target in the backend.
* packages/flutter_tools/test/general.shard/ios/native_assets_test.dart
* packages/flutter_tools/test/general.shard/macos/native_assets_test.dart
* Unit tests the native assets being packaged on a iOS/MacOS build.
It also extends various existing tests:
* dev/devicelab/bin/tasks/module_test_ios.dart
* Exercises the add2app scenario.
* packages/flutter_tools/test/general.shard/features_test.dart
* Unit test the new feature flag.
Enable Impeller benchmarks for drawAtlas/drawVertices on iOS/Metal, Android/GLES, and Android/Vulkan.
Enable impeller tessellation benchmarks on iOS/Metal and Android/Vulkan - not GLES as this is measuring backend agnostic performance.
This PR includes the following changes. These changes only apply to iOS 17 physical devices.
| Command | Change Description | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode. | No change. |
| `flutter run` | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app. | No change. |
| `flutter screenshot` | Will return error. | Will return error. |
Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs
Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.
Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
Because the cost of type checks dominate our dart2wasm benchmarks, we've
decided to pass `--omit-type-checks` for now.
This was previously reverted because the skwasm benchmarks were broken
in general for a separate reason, and my getting rid of `bringup: true`
broke the tree. I ended up fixing the benchmarks and getting rid of
`bringup: true` in a separate commit, so this just adds the flag only.
We've decided to use the `--omit-type-checks` flag for our dart2wasm benchmarks. Right now, many of the benchmark results are dominated by type checks and most of what we are actually trying to measure get drowned out in the noise.
Adding debugging for https://github.com/flutter/flutter/issues/129836.
Takes a screenshot when startup test takes too long (10 minutes).
Also, removes some old debugging and add new debugging message.
This enables benchmarks for the Skwasm renderer, compiled with
dart2wasm.
Platform views aren't supported in Skwasm yet, so we are skipping those
benchmarks for now.
By default, the browser fuzzes the timer APIs such that they have a granularity of approximately 100 microseconds (this is due to Spectre mitigation techniques). However, many of the thing we are trying to measure actually have a much finer granularity than 100 microseconds. As a result, many of our benchmarks are extremely noisy and don't provide accurate data.
By serving the initial script files with the `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` HTTP headers, the browser runs the benchmarks in a `crossOriginIsolated` context, which restores the fine granularity of APIs such as `performance.now()` to microsecond precision.
Also, we were considering anything an outlier that was more than one standard deviation away from the mean. In a normal distribution, that means we are only capturing 68% of the data and the rest are considered outliers. This is not ideal. Doing two standard deviations away captures 95% of the data, and the outliers are in the remaining 5%, which seems much more reasonable.
I think the flake is due to setclipboard or semantics update race condition. I migrated the test to use integration test package which relies less on timing
fixes https://github.com/flutter/flutter/issues/124636
* Add Linux unit tests to plugin template
Adds an example native unit test to the plugin template for Linux,
matching the structure we use for our 1P plugin unit tests. Once these
have been added for all platforms+languages, they will be documented on
a new plugin development page to explain their use.
While ideally we would adjust the engine APIs first to allow for testing
the method call handler directly, it's unclear when we will have time
for that work, and for a complex plugin most of the testing wouldn't be
at that layer anyway, so having the structure in place with the
limitations documented is still a significant improvement over having
nothing in the template.
Part of https://github.com/flutter/flutter/issues/82458
* Add creation test
* Add integration tests
* Missing newlines
* test owner
* Typo
* Added integration test for wide gamut support.
* cleaned up
* deleted files that can be generated
* switched back to standard flutter in the shell script
* added devicelab task
* removed analysis options
* analyzer
* Fixed task
* made local_run.sh not executable
* analyzer
* removed the logo asset
* added task to ci
* updated pubspec
* analysis errors fixed
* updated pubspec
* Add Windows unit tests to plugin template
Adds an example native unit test to the plugin template for Windows,
matching the format we use for our 1P plugin example app unit tests.
Once these have been added for all platforms+languages, they will be
documented on a new plugin development page to explain their use.
Since we don't appear to be running our current plugin e2e tests for
Windows, this adds a new configuration to run them. I haven't
`led`-tested this, so it may not work, but this will give a starting
point for getting them running.
Part of https://github.com/flutter/flutter/issues/82458
* Minor fix
* Add test owner
* Fix typo
* Fix test feature flag
