Commit Graph

350 Commits

Author SHA1 Message Date
Jonah Williams
51a0ec006f
[devicelab] fix name of flutter driver benchmark. (#132527)
Missing the _test.dart
2023-08-14 17:03:48 -07:00
Jonah Williams
f7bd03202f
[devicelab] boot up benchmarks. (#132148)
Enable Impeller benchmarks for drawAtlas/drawVertices on iOS/Metal, Android/GLES, and Android/Vulkan.

Enable impeller tessellation benchmarks on iOS/Metal and Android/Vulkan - not GLES as this is measuring backend agnostic performance.
2023-08-14 18:47:08 +00:00
Matan Lurey
a7997f606e
Update dev/devicelab/** to provide --local-engine-host. (#132342)
Partial work towards https://github.com/flutter/flutter/issues/132245.

I have to admit I don't totally understand what I've updated, or whether
there are more integration points needed.
2023-08-10 17:25:30 -07:00
Victoria Ashworth
d631b26285
New tooling for iOS 17 physical devices (#131865)
This PR includes the following changes. These changes only apply to iOS 17 physical devices.

| Command | Change Description  | Changes to User Experience |
| ------------- | ------------- | ------------- |
| `flutter run --release` | Uses `devicectl` to install and launch application in release mode.  | No change.  |
| `flutter run`  | Uses Xcode via automation scripting to run application in debug and profile mode. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal. |
| `flutter run --use-application-binary=xxxx` | Creates temporary empty Xcode project and use Xcode to run via automation scripting in debug and profile. | Xcode will be opened in the background. Errors/crashes may be caught in Xcode and therefore may not show in terminal.  |
| `flutter install` | Uses `devicectl` to check installed apps, install app, uninstall app.  | No change.  |
| `flutter screenshot` | Will return error.  | Will return error.  |

Other changes include:
* Using `devicectl` to get information about the device
* Using `idevicesyslog` and Dart VM logging for device logs

Note:
Xcode automation scripting (used in `flutter run` for debug and profile) does not work in a headless (without a UI) interface. No known workaround.

Fixes https://github.com/flutter/flutter/issues/128827, https://github.com/flutter/flutter/issues/128531.
2023-08-09 19:25:12 +00:00
gaaclarke
1ff82af540
[Impeller] Adds advanced blend benchmark (#131893)
issue: https://github.com/flutter/flutter/issues/131784

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-08-09 10:57:35 -07:00
gaaclarke
f0e39349c9
[Impeller] Add animated_blur_backdrop_filter_perf execution on samsung vulkan and opengles (#131864)
issue: https://github.com/flutter/flutter/issues/131784
2023-08-03 22:31:06 +00:00
gaaclarke
4247e1eff1
Added new gallery benchmark test that forces opengles. (#131796)
fixes https://github.com/flutter/flutter/issues/131782

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat

---------

Co-authored-by: Xilai Zhang <xilaizhang@google.com>
2023-08-03 12:25:29 -07:00
Victoria Ashworth
a903f1defb
Add verbose engine logs for iOS start up tests (#130511)
Added to be able to view logs from https://github.com/flutter/engine/pull/43616.

Added to debug https://github.com/flutter/flutter/issues/129836
2023-07-14 16:51:50 +00:00
Victoria Ashworth
6683468f0b
Add debugging for iOS startup test flakes (#130099)
Adding debugging for https://github.com/flutter/flutter/issues/129836.

Takes a screenshot when startup test takes too long (10 minutes).

Also, removes some old debugging and add new debugging message.
2023-07-07 16:49:00 +00:00
Brandon DeRosier
b898d1d270
[devicelab] Add platform view scroll benchmarks for Impeller on Android (#129455)
For tracking perf improvements with changes like
https://github.com/flutter/engine/pull/43152.
2023-06-23 18:40:17 -07:00
Jonah Williams
41abe998ee
[devicelab] add drawPoints benchmark (#126728)
Add a benchmark that measures the improvements from https://github.com/flutter/engine/pull/41803
2023-05-15 19:14:58 +00:00
Jenn Magder
4f3c989bd3
Remove complex_layout_*__compile benchmarks (#126023)
Fixes https://github.com/flutter/flutter/issues/126020.
2023-05-03 23:49:01 +00:00
Zachary Anderson
aef7929f3d
Report total_ui_gc_time to SkiaPerf (#125919)
Not sure why this wasn't already being reported.
2023-05-02 22:09:33 -07:00
Zachary Anderson
50f83fc277
Add back one Skia test on iOS (#125663)
Fixes https://github.com/flutter/flutter/issues/125648
2023-04-28 10:41:33 -07:00
Taha Tesser
457e98a1e7
Add Sliders to macrobenchmarks (#125296)
This adds a performance test for `Slider` and `RangeSlider` to run on Android.
This is to measure upcoming changes to these widgets and compare their performance. 

https://user-images.githubusercontent.com/48603081/233633349-0bcad3c3-04a9-42dd-acd0-46b76ce51178.mp4
2023-04-24 23:25:28 +00:00
Harry Terkelsen
daaba8a4aa
Add --local-web-sdk in devicelab runner to make --ab testing work for web (#123825)
This allows us to check for performance differences in local Web SDKs.

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
2023-03-31 14:25:00 -07:00
Jonah Williams
54dbbd76b8
[devicelab] add regular old blur benchmark (#123879)
[devicelab] add regular old blur benchmark
2023-03-31 20:54:51 +00:00
Michael Goderbauer
fda9ecfef7
Remove 1745 decorative breaks (#123259)
Remove 1745 decorative breaks
2023-03-22 21:12:22 +00:00
Jonah Williams
67810d672b
delete sksl benchmarks (#123163)
delete sksl benchmarks
2023-03-21 22:52:18 +00:00
Jonah Williams
e98c53751e
Revert "[Impeller] Temporary flag flip for devicelab tests to use Impeller redux. (#122340)" (#122362)
Revert "[Impeller] Temporary flag flip for devicelab tests to use Impeller redux."
2023-03-10 16:16:30 +00:00
Jonah Williams
de399511d6
[Impeller] Temporary flag flip for devicelab tests to use Impeller redux. (#122340) 2023-03-09 17:45:17 -08:00
Jonah Williams
4adb299566
Revert "[Impeller] Temporary flag flip for devicelab tests to use Impeller. (#122224)" (#122236)
This reverts commit 21b8b7297f.
2023-03-08 14:04:25 -08:00
Jonah Williams
21b8b7297f
[Impeller] Temporary flag flip for devicelab tests to use Impeller. (#122224)
[Impeller] Temporary flag flip for devicelab tests to use Impeller.
2023-03-08 21:41:51 +00:00
Jenn Magder
a16e620ec2
Funnel devicelab tests through utils process methods (#122161)
Funnel devicelab tests through utils process methods
2023-03-08 19:44:40 +00:00
Ben Konyi
ecd7518df5
Reland "Remove references to Observatory (#118577)" (#121606)
This reverts commit 275ab9c69b.
2023-02-28 11:57:04 -05:00
Michael Goderbauer
275ab9c69b
Revert "Reland "Remove references to Observatory (#118577)" (#121215)" (#121555)
Revert "Reland "Remove references to Observatory (#118577)""
2023-02-27 23:46:53 +00:00
Ben Konyi
fbae472fc3
Reland "Remove references to Observatory (#118577)" (#121215)
This reverts commit 298d8c76ba.
2023-02-27 09:26:43 -05:00
Casey Hillers
298d8c76ba
Revert "Remove references to Observatory (#118577)" (#120929)
This reverts commit 2df140f40d.
2023-02-16 21:28:30 -08:00
Ben Konyi
2df140f40d
Remove references to Observatory (#118577)
Observatory is being deprecated for Dart 3.0 so it should no longer be
referenced in tooling messaging / flags.

See https://github.com/dart-lang/sdk/issues/50233
2023-02-13 14:29:30 -05:00
Michael Goderbauer
ab3c82244e
Remove unnecessary null checks in dev/devicelab (#118842) 2023-01-20 18:56:44 +00:00
Loïc Sharma
cea55d99d0
[Linux] Add a 'flutter run' console output test (#118279)
* Add Linux support for the UI integration test project

* Add Linux run console test
2023-01-13 19:17:12 +00:00
Jonah Williams
08a2635e2b
[devicelab] add benchmark for complex non-intersecting widgets with platform views (#116436) 2022-12-02 21:37:45 +00:00
Christopher Fujino
e62b6e7997
Track entire web build directory size in web_size__compile_test (#115682)
* wip

* add track entire web build output dir size

* add more fields

* migrate metrics to use bytes rather than kb

* update keys

* use -9 on tar cz

* delete tempDir and tar first before measuring size
2022-11-30 22:06:58 +00:00
Jonah Williams
dfa3d33328
[devicelab] track performance of animated image filter (#115850)
* [devicelab] track performance of animated image filter

* ++

* make animations repeat and add dispose

* make duration more consistently 10sec
2022-11-29 01:59:19 +00:00
Jonah Williams
9379c3233c
Revert "[devicelab] measure entire release folder size, zipped (#115612)" (#115617)
This reverts commit 0b33b85928.
2022-11-17 18:13:49 -08:00
Jonah Williams
0b33b85928
[devicelab] measure entire release folder size, zipped (#115612)
* [devicelab] measure entire release folder size, zipped

* [devicelab] tar with more settings

* super safe reland

* Update perf_tests.dart
2022-11-18 01:54:54 +00:00
Jonah Williams
ec03f1c8c4
Revert "[devicelab] measure entire release folder size, zipped (#115597)" (#115609)
This reverts commit 75a0a72555.
2022-11-17 16:27:49 -08:00
Jonah Williams
75a0a72555
[devicelab] measure entire release folder size, zipped (#115597) 2022-11-18 00:03:49 +00:00
Christopher Fujino
0211df9cfc
[flutter_tools] provide --timeout option to flutter drive (#114458) 2022-11-02 18:43:42 +00:00
Jason Simmons
b31f41bd1a
Unlock the device before running a StartupTest benchmark (#112243) 2022-09-23 14:38:09 +00:00
Gary Qian
a8a6fe2fb1
Reland "Add initial compile tests #109177" (#111562) 2022-09-14 19:35:16 +00:00
Jenn Magder
99372ba1e4
Revert "Add initial compile tests (#109177)" (#110550)
This reverts commit 3437fd9c03.
2022-08-29 17:46:31 -07:00
Gary Qian
3437fd9c03
Add initial compile tests (#109177) 2022-08-29 23:12:22 +00:00
Stephen (Alex) Wallen
2f3ba1a313
Readability revisions for #109891 (#110348) 2022-08-29 18:23:16 +00:00
Zachary Anderson
056f11060f
Add several new iOS/Impeller benchmarks (#110358) 2022-08-26 14:06:26 -07:00
Jonah Williams
f0ffc85698
Add bad scroller benchmark (#110362) 2022-08-26 18:50:07 +00:00
Stephen (Alex) Wallen
27bfc511ba
[macOS] Bring up "complex_layout" devicelab, startup tests for x64 (#110100) 2022-08-25 23:35:00 +00:00
Stephen (Alex) Wallen
d71a8a7093
[macOS] Bring up "hello_world" devicelab, compilation test for x86. (#109891) 2022-08-24 18:04:42 +00:00
Zachary Anderson
d79a3d7a19
Remove most benchmarks of SkSL warmup (#110040) 2022-08-22 22:31:54 +00:00
yaakovschectman
4e068a613e
Bring up new Windows start-up tests (#109618)
* Add Windows startup test

* Add new tests for Windows desktop

* Bring up complex layout tests for Win Desktop

* Bring up flutter_view startup test for Windows

* Add Platform View startup test for Windows

* Fix typo in .ci.yaml

* Make basename variable final

* Link to bug in .ci.yaml
2022-08-17 09:38:41 -04:00