Commit Graph

373 Commits

Author SHA1 Message Date
Jenn Magder
3d41ed9635
Remove verbose-system-logs on iOS perf tests (#142739)
https://github.com/flutter/flutter/issues/129836 was closed in October.
2024-02-01 19:48:18 +00:00
Jonah Williams
7208d3b73c
[Impeller] opt vulkan tests into GPU tracing. (#142649)
We're going to disable GPU tracing by default. Opt our benchmarks back in so that we have continuity.
2024-02-01 00:48:01 +00:00
Pierrick Bouvier
37c3978b34
Enable native compilation for windows-arm64 (#141930)
It's now possible to natively compile a flutter app for windows-arm64. Cross-compilation is not yet implemented.

Uses arm64 artifacts now available for Dart/Flutter. Platform detection is based on Abi class, provided by Dart. Depending if Dart is an arm64 or x64 binary, the Abi is set accordingly. Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking PROCESSOR_ARCHITECTURE environment variable, which is the way to detect host architecture on Windows.

This is available only for master channel (on other channels, it fallbacks to windows-x64).

On windows-x64, it produces an x64 app. On windows-arm64, it produces an arm64 app.
2024-01-26 00:08:20 +00:00
auto-submit[bot]
1901d6fa10
Reverts "Enable native compilation for windows-arm64 " (#141809)
Reverts flutter/flutter#137618
Initiated by: Jasguerrero
This change reverts the following previous change:
Original Description:
It's now possible to natively compile a flutter app for
windows-arm64. Cross-compilation is not yet implemented.

Uses arm64 artifacts now available for Dart/Flutter.
Platform detection is based on Abi class, provided by Dart. Depending if
Dart is an arm64 or x64 binary, the Abi is set accordingly.
Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking
PROCESSOR_ARCHITECTURE environment variable, which is the way to detect
host architecture on Windows.

This is available only for master channel (on other channels, it
fallbacks to windows-x64).

On windows-x64, it produces an x64 app. On windows-arm64, it produces an
arm64 app.
2024-01-18 22:48:16 +00:00
Pierrick Bouvier
540559204e
Enable native compilation for windows-arm64 (#137618)
It's now possible to natively compile a flutter app for
windows-arm64. Cross-compilation is not yet implemented.

Uses arm64 artifacts now available for Dart/Flutter.
Platform detection is based on Abi class, provided by Dart. Depending if
Dart is an arm64 or x64 binary, the Abi is set accordingly.
Initial bootstrap of dart artifacts (update_dart_sdk.ps1) is checking
PROCESSOR_ARCHITECTURE environment variable, which is the way to detect
host architecture on Windows.

This is available only for master channel (on other channels, it
fallbacks to windows-x64).

On windows-x64, it produces an x64 app. On windows-arm64, it produces an
arm64 app.
2024-01-18 19:15:23 +00:00
Jenn Magder
0cd2ea960b
Set compile test iOS app target version to not embed Swift runtime (#140188)
ObjC->Swift plugin migration caused a size regression in the gallery app because the Swift runtime was also pulled in.  

The gallery app minimum target version is iOS 11.0, which predates Swift ABI compatibility.  Pre iOS 12.2 apps embedded the Swift runtime since there wasn't one available to use in the OS.  

Add  `FLUTTER_XCODE_IPHONEOS_DEPLOYMENT_TARGET` to the compile perf test environment, which gets translated by the tool to an Xcode build setting:
```
[2023-12-14 15:52:14.797318] [STDOUT] stdout:                IPHONEOS_DEPLOYMENT_TARGET = 12.2
```

On my machine on main
```
    "release_size_bytes": 43717389,
```
becomes
```
    "release_size_bytes": 40679432,
```

Fixes https://github.com/flutter/flutter/issues/139605
2023-12-15 05:04:33 +00:00
Jonah Williams
c421f65b3b
Add Impeller complex layout impeller benchmarks. (#139490)
These benchmarks do a nice job of showing CPU overhead, which is useful for many of the microoptimizations we've been doing.
2023-12-04 19:23:26 +00:00
Nate
3267fbc07a
Implement switch expressions in dev/ (#139048)
I previously made a PR (#136140) that used `switch` expressions to make some parts of the Flutter codebase easier to understand. It was assigned to the framework team, and @christopherfujino let me know that it was too large to effectively review and recommended breaking it up into smaller pull requests.

Here's a PR that only targets files in the `dev/` directory. Hopefully this will be easier to work with!

(solves issue https://github.com/flutter/flutter/issues/136139)
2023-11-28 23:40:28 +00:00
Jonah Williams
925f3eee66
[benchmarks] disable partial repaint for multiple backdrop blur iOS macrobenchmarks. (#137902)
Partial repaint is too effective, and we'd like to be able to measure performance without carefully structuring the benchmarks. For example, right now partial repaint is culling any blurs in the multibackdrop case, which we should be using to track  https://github.com/flutter/flutter/issues/132735
2023-11-06 19:49:15 +00:00
Jonah Williams
841cc675b3
[Impeller] add OpenGL GPU tracing to devicelab tests. (#136974)
Adds the metadata key required to enable OpenGLES GPU tracing. This is off by default because the API crashes on some GPU models, but it should be safe on the Pixel 7 (others TBD based on testing results).
2023-10-20 20:23:25 +00:00
Victoria Ashworth
3c3d414e7c
[Reland] Skip injecting Bonjour settings when port publication is disabled (#136842)
Reland https://github.com/flutter/flutter/pull/136751 with fixes.
2023-10-19 17:38:52 +00:00
auto-submit[bot]
c32ffa65c9
Reverts "[Reland] Skip injecting Bonjour settings when port publication is disabled" (#136839)
Reverts flutter/flutter#136751
Initiated by: vashworth
This change reverts the following previous change:
Original Description:
Reland of https://github.com/flutter/flutter/pull/136562 with fixes.
2023-10-18 20:33:17 +00:00
Victoria Ashworth
1599cbebc3
[Reland] Skip injecting Bonjour settings when port publication is disabled (#136751)
Reland of https://github.com/flutter/flutter/pull/136562 with fixes.
2023-10-18 20:08:19 +00:00
auto-submit[bot]
54c0a350dd
Reverts "Skip injecting Bonjour settings when port publication is disabled" (#136750)
Reverts flutter/flutter#136562
Initiated by: vashworth
This change reverts the following previous change:
Original Description:
Some of our tests in CI are triggering the `NSLocalNetworkUsageDescription` dialog when they're not supposed to (https://github.com/flutter/flutter/issues/129836) since it's disabled via flags (`--no-publish-port` for flutter/flutter and `--disable-vm-service-publication` for flutter/engine).

Normally, we inject `NSLocalNetworkUsageDescription` (and other bonjour settings) to the Info.plist during the project build for debug and profile mode since by default they will publish the VM Service port over mDNS.

To help diagnose the issue, though, this PR changes it so that we don't inject `NSLocalNetworkUsageDescription` (and other bonjour settings) when port publication is disabled since it shouldn't be needed. Hopefully, this will give us better error messages or cause the app to crash and end the test early (rather than timeout after 30 minutes).
2023-10-17 19:27:19 +00:00
Victoria Ashworth
0383d8ba9e
Skip injecting Bonjour settings when port publication is disabled (#136562)
Some of our tests in CI are triggering the `NSLocalNetworkUsageDescription` dialog when they're not supposed to (https://github.com/flutter/flutter/issues/129836) since it's disabled via flags (`--no-publish-port` for flutter/flutter and `--disable-vm-service-publication` for flutter/engine).

Normally, we inject `NSLocalNetworkUsageDescription` (and other bonjour settings) to the Info.plist during the project build for debug and profile mode since by default they will publish the VM Service port over mDNS.

To help diagnose the issue, though, this PR changes it so that we don't inject `NSLocalNetworkUsageDescription` (and other bonjour settings) when port publication is disabled since it shouldn't be needed. Hopefully, this will give us better error messages or cause the app to crash and end the test early (rather than timeout after 30 minutes).
2023-10-17 19:09:08 +00:00
Jonah Williams
1956474409
Upload GPU frame times for Impeller on Android/iOS. (#136565)
These values will be 0 until https://github.com/flutter/engine/pull/46846 and https://github.com/flutter/engine/pull/46796 roll into the framework.
2023-10-14 17:09:26 +00:00
Callum Moffat
2925525e35
Upload frame_request_pending_latency [attempt #2] (#135890)
These values are generated since https://github.com/flutter/flutter/pull/135279, but I didn't know to add the new keys to this list to get them to upload. 

Failed to do so in #135645, I believe the mistake there was putting them in `_kCommonScoreKeys`, which is also used in "E2E" tests, that don't get full trace data, only high level `FrameTiming` packets.

Part of https://github.com/flutter/flutter/issues/129150
2023-10-05 21:54:27 +00:00
Ricardo Amador
4e5e47e76a
Add device ready check (#135526)
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*

*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/121420

*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
2023-09-30 01:58:52 +00:00
Zachary Anderson
63b98d55b0
Revert "Upload generated frame-request-pending stats" (#135672)
Reverts flutter/flutter#135645
2023-09-28 07:23:25 -07:00
Callum Moffat
dd341d4295
Upload generated frame-request-pending stats (#135645)
These values are generated since https://github.com/flutter/flutter/pull/135279, but I didn't know to add the new keys to this list to get them to upload.

Part of https://github.com/flutter/flutter/issues/129150
2023-09-28 13:05:10 +00:00
Zachary Anderson
579e196073
Stop reporting new_gen_gc_count to SkiaPerf (#135324)
Movement in this metric is mostly noise, creates many false alerts, and is neither the cause nor effect of changes in other metrics.
2023-09-22 21:55:28 +00:00
Jim Graham
b51859ec33
Add benchmark for scrolling very long pictures (#133434)
This benchmark will track the performance of the RTree implementation to cull very large pictures down to just the portion visible on the screen.
2023-08-31 19:03:14 +00:00
Pierrick Bouvier
792e26df95
[Windows] Add target architecture to build path (#131843)
To implement windows-arm64 support, it is needed to add architecture as a subdirectory (https://github.com/flutter/flutter/issues/129805).

In short, when performing a flutter windows build, we have:
- Before: build/windows/runner/Release/gallery.exe
- After: build/windows/x64/runner/Release/gallery.exe

This convention follows what flutter linux build does.

Addresses: https://github.com/flutter/flutter/issues/129805
Addresses: https://github.com/flutter/flutter/issues/116196

Design doc: [flutter.dev/go/windows-arm64](https://flutter.dev/go/windows-arm64)
2023-08-31 09:09:02 -07:00
Jonah Williams
51a0ec006f
[devicelab] fix name of flutter driver benchmark. (#132527)
Missing the _test.dart
2023-08-14 17:03:48 -07:00
Jonah Williams
f7bd03202f
[devicelab] boot up benchmarks. (#132148)
Enable Impeller benchmarks for drawAtlas/drawVertices on iOS/Metal, Android/GLES, and Android/Vulkan.

Enable impeller tessellation benchmarks on iOS/Metal and Android/Vulkan - not GLES as this is measuring backend agnostic performance.
2023-08-14 18:47:08 +00:00
Matan Lurey
a7997f606e
Update dev/devicelab/** to provide --local-engine-host. (#132342)
Partial work towards https://github.com/flutter/flutter/issues/132245.

I have to admit I don't totally understand what I've updated, or whether
there are more integration points needed.
2023-08-10 17:25:30 -07:00
Victoria Ashworth
d631b26285
New tooling for iOS 17 physical devices (#131865)
This PR includes the following changes. These changes only apply to iOS 17 physical devices.

| Command | Change Description  | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode.  | No change.  |
| `flutter run`  | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal.  |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app.  | No change.  |
| `flutter screenshot` | Will return error.  | Will return error.  |

Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs

Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.

Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
2023-08-09 19:25:12 +00:00
gaaclarke
1ff82af540
[Impeller] Adds advanced blend benchmark (#131893)
issue: https://github.com/flutter/flutter/issues/131784

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-08-09 10:57:35 -07:00
gaaclarke
f0e39349c9
[Impeller] Add animated_blur_backdrop_filter_perf execution on samsung vulkan and opengles (#131864)
issue: https://github.com/flutter/flutter/issues/131784
2023-08-03 22:31:06 +00:00
gaaclarke
4247e1eff1
Added new gallery benchmark test that forces opengles. (#131796)
fixes https://github.com/flutter/flutter/issues/131782

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat

---------

Co-authored-by: Xilai Zhang <xilaizhang@google.com>
2023-08-03 12:25:29 -07:00
Victoria Ashworth
a903f1defb
Add verbose engine logs for iOS start up tests (#130511)
Added to be able to view logs from https://github.com/flutter/engine/pull/43616.

Added to debug https://github.com/flutter/flutter/issues/129836
2023-07-14 16:51:50 +00:00
Victoria Ashworth
6683468f0b
Add debugging for iOS startup test flakes (#130099)
Adding debugging for https://github.com/flutter/flutter/issues/129836.

Takes a screenshot when startup test takes too long (10 minutes).

Also, removes some old debugging and add new debugging message.
2023-07-07 16:49:00 +00:00
Brandon DeRosier
b898d1d270
[devicelab] Add platform view scroll benchmarks for Impeller on Android (#129455)
For tracking perf improvements with changes like
https://github.com/flutter/engine/pull/43152.
2023-06-23 18:40:17 -07:00
Jonah Williams
41abe998ee
[devicelab] add drawPoints benchmark (#126728)
Add a benchmark that measures the improvements from https://github.com/flutter/engine/pull/41803
2023-05-15 19:14:58 +00:00
Jenn Magder
4f3c989bd3
Remove complex_layout_*__compile benchmarks (#126023)
Fixes https://github.com/flutter/flutter/issues/126020.
2023-05-03 23:49:01 +00:00
Zachary Anderson
aef7929f3d
Report total_ui_gc_time to SkiaPerf (#125919)
Not sure why this wasn't already being reported.
2023-05-02 22:09:33 -07:00
Zachary Anderson
50f83fc277
Add back one Skia test on iOS (#125663)
Fixes https://github.com/flutter/flutter/issues/125648
2023-04-28 10:41:33 -07:00
Taha Tesser
457e98a1e7
Add Sliders to macrobenchmarks (#125296)
This adds a performance test for `Slider` and `RangeSlider` to run on Android.
This is to measure upcoming changes to these widgets and compare their performance. 

https://user-images.githubusercontent.com/48603081/233633349-0bcad3c3-04a9-42dd-acd0-46b76ce51178.mp4
2023-04-24 23:25:28 +00:00
Harry Terkelsen
daaba8a4aa
Add --local-web-sdk in devicelab runner to make --ab testing work for web (#123825)
This allows us to check for performance differences in local Web SDKs.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-03-31 14:25:00 -07:00
Jonah Williams
54dbbd76b8
[devicelab] add regular old blur benchmark (#123879)
[devicelab] add regular old blur benchmark
2023-03-31 20:54:51 +00:00
Michael Goderbauer
fda9ecfef7
Remove 1745 decorative breaks (#123259)
Remove 1745 decorative breaks
2023-03-22 21:12:22 +00:00
Jonah Williams
67810d672b
delete sksl benchmarks (#123163)
delete sksl benchmarks
2023-03-21 22:52:18 +00:00
Jonah Williams
e98c53751e
Revert "[Impeller] Temporary flag flip for devicelab tests to use Impeller redux. (#122340)" (#122362)
Revert "[Impeller] Temporary flag flip for devicelab tests to use Impeller redux."
2023-03-10 16:16:30 +00:00
Jonah Williams
de399511d6
[Impeller] Temporary flag flip for devicelab tests to use Impeller redux. (#122340) 2023-03-09 17:45:17 -08:00
Jonah Williams
4adb299566
Revert "[Impeller] Temporary flag flip for devicelab tests to use Impeller. (#122224)" (#122236)
This reverts commit 21b8b7297f.
2023-03-08 14:04:25 -08:00
Jonah Williams
21b8b7297f
[Impeller] Temporary flag flip for devicelab tests to use Impeller. (#122224)
[Impeller] Temporary flag flip for devicelab tests to use Impeller.
2023-03-08 21:41:51 +00:00
Jenn Magder
a16e620ec2
Funnel devicelab tests through utils process methods (#122161)
Funnel devicelab tests through utils process methods
2023-03-08 19:44:40 +00:00
Ben Konyi
ecd7518df5
Reland "Remove references to Observatory (#118577)" (#121606)
This reverts commit 275ab9c69b.
2023-02-28 11:57:04 -05:00
Michael Goderbauer
275ab9c69b
Revert "Reland "Remove references to Observatory (#118577)" (#121215)" (#121555)
Revert "Reland "Remove references to Observatory (#118577)""
2023-02-27 23:46:53 +00:00
Ben Konyi
fbae472fc3
Reland "Remove references to Observatory (#118577)" (#121215)
This reverts commit 298d8c76ba.
2023-02-27 09:26:43 -05:00