Context: https://github.com/flutter/flutter/issues/131862
This PR injects a "realm" component to the storage base URL when the contents of the file `bin/internal/engine.realm` is non-empty.
As documented in the PR, when the realm is `flutter_archives_v2`, and `bin/internal/engine.version` contains the commit hash for a commit in a `flutter/engine` PR, then the artifacts pulled by the tool will be the artifacts built by the presubmit checks for the PR.
This works for everything but the following two cases:
1. Fuchsia artifacts are not uploaded to CIPD by the Fuchsia presubmit builds.
2. Web artifacts are not uploaded to gstatic by the web engine presubmit builds.
For (1), the flutter/flutter presubmit `fuchsia_precache` is driven by a shell script outside of the repo. It will fail when the `engine.version` and `engine.realm` don't point to a post-submit engine commit.
For (2), the flutter/flutter web presubmit tests that refer to artifacts in gstatic hang when the artifacts aren't found, so this PR skips them.
This PR includes the following changes. These changes only apply to iOS 17 physical devices.
| Command | Change Description | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode. | No change. |
| `flutter run` | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app. | No change. |
| `flutter screenshot` | Will return error. | Will return error. |
Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs
Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.
Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
Because the cost of type checks dominate our dart2wasm benchmarks, we've
decided to pass `--omit-type-checks` for now.
This was previously reverted because the skwasm benchmarks were broken
in general for a separate reason, and my getting rid of `bringup: true`
broke the tree. I ended up fixing the benchmarks and getting rid of
`bringup: true` in a separate commit, so this just adds the flag only.
We've decided to use the `--omit-type-checks` flag for our dart2wasm benchmarks. Right now, many of the benchmark results are dominated by type checks and most of what we are actually trying to measure get drowned out in the noise.
Manual roll is needed because incoming dart sdk requires updated version
vm_snapshot_analysis (>=0.7.4).
5ae09b8b4f...7c83ea3e85
```
7c83ea3e85 Reland "Manual roll Dart SDK from 2d98d9e27dae to 0b07debd5862 (21 revisions) (#43457)" (#43472)
9ef3e8d533 Roll Skia from 5eba922297bb to 93c92f97f5ab (2 revisions) (#43471)
```
Remove implementation of SuitePlatform from the test as well. Remove use
of fake cwd from SuitePlatform as it can't be properly faked.
Adding debugging for https://github.com/flutter/flutter/issues/129836.
Takes a screenshot when startup test takes too long (10 minutes).
Also, removes some old debugging and add new debugging message.
This enables benchmarks for the Skwasm renderer, compiled with
dart2wasm.
Platform views aren't supported in Skwasm yet, so we are skipping those
benchmarks for now.
- Bumps `vm_service` from `11.6.0` to `11.7.1`
- Bumps `web` from `0.1.3-beta` to `0.1.4-beta` and adds it everywhere.
- Moves `js` from `dependencies` to `dev_dependencies`
Changes `Linux_android flutter_engine_group_performance` to uninstall the app that it uses for testing before attempting to install it again to ensure proper cleanup.
Attempt at fixing https://github.com/flutter/flutter/issues/127011.
By default, the browser fuzzes the timer APIs such that they have a granularity of approximately 100 microseconds (this is due to Spectre mitigation techniques). However, many of the thing we are trying to measure actually have a much finer granularity than 100 microseconds. As a result, many of our benchmarks are extremely noisy and don't provide accurate data.
By serving the initial script files with the `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` HTTP headers, the browser runs the benchmarks in a `crossOriginIsolated` context, which restores the fine granularity of APIs such as `performance.now()` to microsecond precision.
Also, we were considering anything an outlier that was more than one standard deviation away from the mean. In a normal distribution, that means we are only capturing 68% of the data and the rest are considered outliers. This is not ideal. Doing two standard deviations away captures 95% of the data, and the outliers are in the remaining 5%, which seems much more reasonable.
I think the flake is due to setclipboard or semantics update race condition. I migrated the test to use integration test package which relies less on timing
fixes https://github.com/flutter/flutter/issues/124636
Removes two plugin tests that aren't actually doing anything useful.
When they were added they were testing v2-embedding plugins in a v1-embedding app, but support for creating v1-embedding apps--and the environment variable that these tests were using to, in theory, trigger them--was removed several years ago in #61203. That means that these tests are in practice exact duplicates of the copies just above without the ennvironment variables.
Fixes https://github.com/flutter/flutter/issues/122200
Adds a macOS implementation of the platform_channel example,
demonstrating method channels and event channels with a battery power
plugin.
Adds platform_channel_sample_test_macos macOS host test to verify the
sample works.
Issue: https://github.com/flutter/flutter/issues/79204
* Add Linux unit tests to plugin template
Adds an example native unit test to the plugin template for Linux,
matching the structure we use for our 1P plugin unit tests. Once these
have been added for all platforms+languages, they will be documented on
a new plugin development page to explain their use.
While ideally we would adjust the engine APIs first to allow for testing
the method call handler directly, it's unclear when we will have time
for that work, and for a complex plugin most of the testing wouldn't be
at that layer anyway, so having the structure in place with the
limitations documented is still a significant improvement over having
nothing in the template.
Part of https://github.com/flutter/flutter/issues/82458
* Add creation test
* Add integration tests
* Missing newlines
* test owner
* Typo
* Added integration test for wide gamut support.
* cleaned up
* deleted files that can be generated
* switched back to standard flutter in the shell script
* added devicelab task
* removed analysis options
* analyzer
* Fixed task
* made local_run.sh not executable
* analyzer
* removed the logo asset
* added task to ci
* updated pubspec
* analysis errors fixed
* updated pubspec
* add asset manifest bin loading and asset manifest api
* use new api for image resolution
* remove upfront smc data casting
* fix typecasting issue
* remove unused import
* fix tests
* lints
* lints
* fix import
* revert image resolution changes
* Update image_resolution_test.dart
* Update decode_and_parse_asset_manifest.dart
* make targetDevicePixelRatio optional
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Update packages/flutter/lib/src/services/asset_manifest.dart
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* fix immutable not being imported
* return List in AssetManifest methods, fix annotation import
* simplify onError callback
* make AssetManifest methods abstract instead of throwing UnimplementedError
* simplify AssetVariant.key docstring
* tweak _AssetManifestBin docstring
* make AssetManifest and AssetVariant doc strings more specific
* use List.of instead of List.from for type-safety
* adjust import
* change _AssetManifestBin comment from doc comment to normal comment
* revert to callback function for onError in loadStructuredBinaryData
* add more to the docstring of AssetManifest.listAssets and AssetVariant.key
* add tests for CachingAssetBundle caching behavior
* add simple test to ensure loadStructuredBinaryData correctly calls load
* Update asset_manifest.dart
* update docstring for AssetManifest.getAssetVariants
* rename getAssetVariants, have it include main asset
* rename isMainAsset field of AssetMetadata to main
* (slightly) shorten name of describeAssetAndVariants
* rename describeAssetVariants back to getAssetVariants
* add tests for TestAssetBundle
* nits
* fix typo in docstring
* remove no longer necessary non-null asserts
* update gallery and google_fonts versions
---------
Co-authored-by: Jonah Williams <jonahwilliams@google.com>
* Add Windows unit tests to plugin template
Adds an example native unit test to the plugin template for Windows,
matching the format we use for our 1P plugin example app unit tests.
Once these have been added for all platforms+languages, they will be
documented on a new plugin development page to explain their use.
Since we don't appear to be running our current plugin e2e tests for
Windows, this adds a new configuration to run them. I haven't
`led`-tested this, so it may not work, but this will give a starting
point for getting them running.
Part of https://github.com/flutter/flutter/issues/82458
* Minor fix
* Add test owner
* Fix typo
* Fix test feature flag
* Add new macos target configured for flavors
* Rename Free App copy-Info.plist to Free App Info.plist
* Remove bogus entitlements
* Remove Generated.xcconfig
* Audit project.pbxproj
* Remove unused configs
* share one info.plist
* Modify scheme so that paid app works
* Codesign automatic
* Pipe flavor as scheme into xcodebuild
* Ignore incoming flavor string
* pipe flavor for flutter run to work
* Add devicelab tests
* Error if host and target device are same for flutter install desktop
* Avoid bang (!) by promoting a local.
Co-authored-by: Jenn Magder <magder@google.com>
* Add supportsInstall property
* Override in test classes
* Add install test on macOS
* Refactor application_package and add tests for package directory
Co-authored-by: a-wallen <stephenwallen@google.com>
Co-authored-by: Jenn Magder <magder@google.com>
* Add new task runner for impeller enabled complex layout test
* Add new target in .ci.yaml
* Claim ownership of complex_layout_scroll_perf_impeller_macos__timeline_summary
* Task runner for non-impeller enabled scroll test
* Add test target for non impeller complex layout scroll
* claim ownership of the last target
* mark as bringup
Co-authored-by: a-wallen <stephenwallen@google.com>
Update test expectations from application/javascript -> text/javascript
`package:mime` now uses `text/javascript` and not `application/javascript`.
See https://github.com/dart-lang/mime/pull/76.
See https://datatracker.ietf.org/doc/html/rfc9239.
> This document defines equivalent processing requirements for the various script media types. The most widely supported media type in use is `text/javascript`; all others are considered historical and obsolete aliases of `text/javascript`.
* Add new task runner for animated_complex_opacity_perf_impeller e2e_summary on macos
* Register new target for macos e2e_summary task
* Claim ownership in TESTOWNERS
* Add task runner for animated complex opacity without impeller
* Register new target in .ci.yaml for non-impeller test
* Claim ownership of non-impeller task
Co-authored-by: a-wallen <stephenwallen@google.com>
* Testing whether emulator is possible.
* Adding changes to see if emulator can be used from recipe.
* adding emulator support.
* Add the emulator flag for testing.
* Using string for boolean since it cannot be parsed in properties
* Checking to see if these changes are being used.
* Updated bool back to string
* Remove trailing whitespace from file.
* Add fix and test
* Add test owner
* Print error for testing
* Make sure locale is the problem
* Test capitalization fix
* Take out Android specific test
* Verify eng fix
* Test fix
* Empty-Commit
* Test all tests
* Up string size for testing
* Remove test
* Undo ci.yaml testing changes
* Test android variant:
* Change to testwidgets
* Add widget tester
* Remove android only attempt
* Revert ci.yaml changes
* [tools]some ui polish for build ipa validation
* do not print out a few success validations
* rename installed type to success for more general usage
* forgot nit after reverting custom validation types and re-use doctor types
* Add an integration test to plugin template example
Dart unit tests don't exercise host-side plugin code at all, so the
example tests in the plugin template currently have very little
meaningful coverage. This adds an integration test to the example app
when creating a plugin, so that there's an example of how to actually
test that a complete round-trip plugin call works.
This is done as a separate template that's currently only used by the
plugin template because I don't know what a good example for a
non-plugin case would be that isn't largely just a duplicate of the
widget tests. However, the integration test pre-includes conditionals
around the parts that are plugin-specific so that it can more easily be
expanded to other use cases later (e.g., in
https://github.com/flutter/flutter/issues/68818).
Part of https://github.com/flutter/flutter/issues/82458
* Add integration test to expected dependencies of a plugin app
* Test fixes
* Make an explicit test case
* Roll Flutter Engine from 67254d6e4b03 to 8d83b98c55b3
* Roll Dart SDK from 35a9facce191 to e517487c5679 (Dart 3.0) (#38105)
* Bump SDK versions.
* Bump Dart SDK version constraints
* Update shrine package to 2.0.1 (null safe version)
* Fix more tests.
* Include patches from Jason for min android sdk version
* Fix analyzer warning
* wip
* add track entire web build output dir size
* add more fields
* migrate metrics to use bytes rather than kb
* update keys
* use -9 on tar cz
* delete tempDir and tar first before measuring size
* [tools]build ipa validate icon size
* add more checks in case apple change the format, and also add device lab tests
* do not depend on collection package
* Alphabetize setup calls
* Add --flavor as an option for install
* Add verbose logging in install command
* Test that flavors build succeeds with proper flavor and fails with bogus one.
* Remove unused import
* The import was used...
* SQUASH
* Add flavor install test
* Rename test
* Add flavors install integration tests
* correct error message
* remove unused imports
* Delete copy test
* update test target
* Refactor mechanism to read buildInfo
* Remove unused import
* Set affected test targets to bringup: true
Co-authored-by: a-wallen <stephenwallen@google.com>
* Add Windows startup test
* Add new tests for Windows desktop
* Bring up complex layout tests for Win Desktop
* Bring up flutter_view startup test for Windows
* Add Platform View startup test for Windows
* Fix typo in .ci.yaml
* Make basename variable final
* Link to bug in .ci.yaml
* Add Windows compilation test (hello_world_windows__compile)
* Add devicelab target to .ci.yaml
* Bringup on new target
* Add license to generated files
* Indicate test ownership
* Delete asset from repo
* Add Windows compilation test (hello_world_windows__compile)
* Add devicelab target to .ci.yaml
* Bringup on new target
* Add license to generated files
* Indicate test ownership
* Delete icon
* Use path.basename
Re triggering tests
* Update test owner to desktop team
