Fixes https://github.com/flutter/flutter/issues/133790
Provides help text for the `--task-args` option of the `test_runner` devicelab command. The current help text is just copypasta from another option's help text
This makes two changes to prepare for incoming changes to skwasm in the web engine:
* We will (at least for now) be depending on the `WebAssembly.Function` constructor in `skwasm`, which is hidden behind the `--experimental-wasm-type-reflection` flag. We need to pass that when running skwasm benchmarks.
* We are going to be upgrading the skwasm build to a newer version of emscripten, which exposes the wasm exports via the `wasmExports` property instead of the `asm` property. Make sure to support either, if passed.
Reverts flutter/flutter#133083
failing on cocoapods:
```
[2023-08-22 16:28:37.783355] [STDOUT] stdout: [ ] Error output from CocoaPods:
[2023-08-22 16:28:37.783379] [STDOUT] stdout: â³
[2023-08-22 16:28:37.783402] [STDOUT] stdout: [ ] [!] The version of CocoaPods used to generate the lockfile (1.12.1) is higher than the version of the current executable (1.11.3). Incompatibility issues may arise.
[2023-08-22 16:28:37.783423] [STDOUT] stdout:
[2023-08-22 16:28:37.783445] [STDOUT] stdout: [!] Automatically assigning platform `iOS` with version `11.0` on target `Runner` because no platform was specified. Please specify a platform for this target in your Podfile. See `https://guides.cocoapods.org/syntax/podfile.html#platform`.
[2023-08-22 16:28:37.783469] [STDOUT] stdout:
[2023-08-22 16:28:37.784059] [STDOUT] stderr: [ ] Error: CocoaPods's specs repository is too out-of-date to satisfy dependencies.
[2023-08-22 16:28:37.784102] [STDOUT] stderr: To update the CocoaPods specs, run:
[2023-08-22 16:28:37.784126] [STDOUT] stderr: pod repo update
[2023-08-22 16:28:37.784147] [STDOUT] stderr:
```
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_ios%20new_gallery_ios__transition_perf/10590/overview
Enable Impeller benchmarks for drawAtlas/drawVertices on iOS/Metal, Android/GLES, and Android/Vulkan.
Enable impeller tessellation benchmarks on iOS/Metal and Android/Vulkan - not GLES as this is measuring backend agnostic performance.
This PR includes the following changes. These changes only apply to iOS 17 physical devices.
| Command | Change Description | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode. | No change. |
| `flutter run` | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app. | No change. |
| `flutter screenshot` | Will return error. | Will return error. |
Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs
Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.
Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
Because the cost of type checks dominate our dart2wasm benchmarks, we've
decided to pass `--omit-type-checks` for now.
This was previously reverted because the skwasm benchmarks were broken
in general for a separate reason, and my getting rid of `bringup: true`
broke the tree. I ended up fixing the benchmarks and getting rid of
`bringup: true` in a separate commit, so this just adds the flag only.
We've decided to use the `--omit-type-checks` flag for our dart2wasm benchmarks. Right now, many of the benchmark results are dominated by type checks and most of what we are actually trying to measure get drowned out in the noise.
Adding debugging for https://github.com/flutter/flutter/issues/129836.
Takes a screenshot when startup test takes too long (10 minutes).
Also, removes some old debugging and add new debugging message.
This enables benchmarks for the Skwasm renderer, compiled with
dart2wasm.
Platform views aren't supported in Skwasm yet, so we are skipping those
benchmarks for now.
By default, the browser fuzzes the timer APIs such that they have a granularity of approximately 100 microseconds (this is due to Spectre mitigation techniques). However, many of the thing we are trying to measure actually have a much finer granularity than 100 microseconds. As a result, many of our benchmarks are extremely noisy and don't provide accurate data.
By serving the initial script files with the `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` HTTP headers, the browser runs the benchmarks in a `crossOriginIsolated` context, which restores the fine granularity of APIs such as `performance.now()` to microsecond precision.
Also, we were considering anything an outlier that was more than one standard deviation away from the mean. In a normal distribution, that means we are only capturing 68% of the data and the rest are considered outliers. This is not ideal. Doing two standard deviations away captures 95% of the data, and the outliers are in the remaining 5%, which seems much more reasonable.
I think the flake is due to setclipboard or semantics update race condition. I migrated the test to use integration test package which relies less on timing
fixes https://github.com/flutter/flutter/issues/124636
* Add Linux unit tests to plugin template
Adds an example native unit test to the plugin template for Linux,
matching the structure we use for our 1P plugin unit tests. Once these
have been added for all platforms+languages, they will be documented on
a new plugin development page to explain their use.
While ideally we would adjust the engine APIs first to allow for testing
the method call handler directly, it's unclear when we will have time
for that work, and for a complex plugin most of the testing wouldn't be
at that layer anyway, so having the structure in place with the
limitations documented is still a significant improvement over having
nothing in the template.
Part of https://github.com/flutter/flutter/issues/82458
* Add creation test
* Add integration tests
* Missing newlines
* test owner
* Typo
* Added integration test for wide gamut support.
* cleaned up
* deleted files that can be generated
* switched back to standard flutter in the shell script
* added devicelab task
* removed analysis options
* analyzer
* Fixed task
* made local_run.sh not executable
* analyzer
* removed the logo asset
* added task to ci
* updated pubspec
* analysis errors fixed
* updated pubspec
* add asset manifest bin loading and asset manifest api
* use new api for image resolution
* remove upfront smc data casting
* fix typecasting issue
* remove unused import
* fix tests
* lints
* lints
* fix import
* revert image resolution changes
* Update image_resolution_test.dart
* Update decode_and_parse_asset_manifest.dart
* make targetDevicePixelRatio optional
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* fix immutable not being imported
* return List in AssetManifest methods, fix annotation import
* simplify onError callback
* make AssetManifest methods abstract instead of throwing UnimplementedError
* simplify AssetVariant.key docstring
* tweak _AssetManifestBin docstring
* make AssetManifest and AssetVariant doc strings more specific
* use List.of instead of List.from for type-safety
* adjust import
* change _AssetManifestBin comment from doc comment to normal comment
* revert to callback function for onError in loadStructuredBinaryData
* add more to the docstring of AssetManifest.listAssets and AssetVariant.key
* add tests for CachingAssetBundle caching behavior
* add simple test to ensure loadStructuredBinaryData correctly calls load
* Update asset_manifest.dart
* update docstring for AssetManifest.getAssetVariants
* rename getAssetVariants, have it include main asset
* rename isMainAsset field of AssetMetadata to main
* (slightly) shorten name of describeAssetAndVariants
* rename describeAssetVariants back to getAssetVariants
* add tests for TestAssetBundle
* nits
* fix typo in docstring
* remove no longer necessary non-null asserts
* update gallery and google_fonts versions
---------
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Add Windows unit tests to plugin template
Adds an example native unit test to the plugin template for Windows,
matching the format we use for our 1P plugin example app unit tests.
Once these have been added for all platforms+languages, they will be
documented on a new plugin development page to explain their use.
Since we don't appear to be running our current plugin e2e tests for
Windows, this adds a new configuration to run them. I haven't
`led`-tested this, so it may not work, but this will give a starting
point for getting them running.
Part of https://github.com/flutter/flutter/issues/82458
* Minor fix
* Add test owner
* Fix typo
* Fix test feature flag
* Testing whether emulator is possible.
* Adding changes to see if emulator can be used from recipe.
* adding emulator support.
* Add the emulator flag for testing.
* Using string for boolean since it cannot be parsed in properties
* Checking to see if these changes are being used.
* Updated bool back to string
* Remove trailing whitespace from file.
* Roll Flutter Engine from 67254d6e4b03 to 8d83b98c55b3
* Roll Dart SDK from 35a9facce191 to e517487c5679 (Dart 3.0) (#38105)
* Bump SDK versions.
* Bump Dart SDK version constraints
* Update shrine package to 2.0.1 (null safe version)
* Fix more tests.
* Include patches from Jason for min android sdk version
* Fix analyzer warning
* wip
* add track entire web build output dir size
* add more fields
* migrate metrics to use bytes rather than kb
* update keys
* use -9 on tar cz
* delete tempDir and tar first before measuring size
* Add Windows startup test
* Add new tests for Windows desktop
* Bring up complex layout tests for Win Desktop
* Bring up flutter_view startup test for Windows
* Add Platform View startup test for Windows
* Fix typo in .ci.yaml
* Make basename variable final
* Link to bug in .ci.yaml
* Add Windows compilation test (hello_world_windows__compile)
* Add devicelab target to .ci.yaml
* Bringup on new target
* Add license to generated files
* Indicate test ownership
* Delete asset from repo
* Add Windows compilation test (hello_world_windows__compile)
* Add devicelab target to .ci.yaml
* Bringup on new target
* Add license to generated files
* Indicate test ownership
* Delete icon
* Use path.basename
Re triggering tests
* Update test owner to desktop team
* Add clipper raster cache benchmarks
* fix ci test error
* Add PhysicalModel widget to test
* Set top margin adaptive screen height
* Remove PhysicalModel
* Starts using the `--source` flag to compile the dart registrant.
* updated general.shard tests
* Fixed the resident compiler flow
* added integration test
* made the integration test self contained
* renamed generated_main to dart_plugin_registrant
* cleaned up for review
* added task runner for ci
* added bringup and TESTOWNERS
* updated failure message
* Update flutter_tools to look for new VM service message
The Dart SDK will soon move away from the current Observatory message:
"Observatory listening on ..."
To a new message that no longer references Observatory:
"Dart VM Service listening on ..."
This change updates all tests with mocks to check for the new message
and also adds support for the new message in ProtocolDiscovery.
See https://github.com/dart-lang/sdk/issues/46756
* Fix some parsing locations
* Fix analysis failures
* Update message
* Remove extra comment
* Update message
* Add globals prefix
