Commit Graph

644 Commits

Author SHA1 Message Date
gaaclarke
e6f3022890
Adds StandardMessageCodec benchmark (#101767) 2022-04-12 16:34:10 -07:00
Jim Graham
493323506f
add gradient benchmarks (#101549) 2022-04-08 17:20:22 -07:00
Michael Goderbauer
195a1cc413
Prepare dev (minus flutter_gallery) for use_super_parameters (#100516) 2022-03-29 12:52:42 -07:00
gaaclarke
912873baa7
Relands "Starts using the --source flag to compile the dart registrant. (#98046)" (#100572) 2022-03-28 18:15:10 -07:00
wangying
712abb08e2
Add benchmark for ShaderMask RasterCache (#100127) 2022-03-28 00:15:07 -07:00
Zachary Anderson
8acbf54930
Adds an impeller variant of the new gallery transition perf benchmark (#100837) 2022-03-26 22:12:48 -07:00
Chris Bracken
4b819782fb
[macOS] Add run release test in devicelab (#100526)
Adds a test that invokes flutter run in release mode on macOS desktop,
waits for successful launch and the flutter command list, then sends the
'q' command to quit the running app.

This adds an integration test for https://github.com/flutter/flutter/pull/100504.

Issue: https://github.com/flutter/flutter/issues/100348 (fix)
Issue: https://github.com/flutter/flutter/issues/97978 (partial fix)
Issue: https://github.com/flutter/flutter/issues/97977 (partial fix)
Umbrella issue: https://github.com/flutter/flutter/issues/60113
2022-03-22 14:12:59 -07:00
Jenn Magder
a165da236d
Remove device unlock from hostonly compile test (#100571) 2022-03-22 13:55:17 -07:00
Chris Yang
e6bd208196
Reland "Add the refresh rate fields to perf_test #99710" (#99854) 2022-03-21 15:55:13 -07:00
gaaclarke
97258979df
Revert "Starts using the --source flag to compile the dart registrant. (#98046) (#100493) 2022-03-21 13:40:23 -07:00
Emmanuel Garcia
5e6a653865
Flush and close stdin in dart_plugin_registry_test (#100338) 2022-03-17 19:25:14 -07:00
Emmanuel Garcia
9072a09958
Fix race condition in readJsonResults (#100243) 2022-03-17 13:40:15 -07:00
wangying
b4040c867b
Add benchmark for ColorFilter raster cache (#99542) 2022-03-16 19:15:13 -07:00
Emmanuel Garcia
cd725e3640
Ensure dart_plugin_registry shows stderr and exits when process exits (#99936) 2022-03-14 12:15:20 -07:00
Jeff Ward
0aab22807c
First pass at using platform abstraction for plugins (#92672) 2022-03-09 14:10:21 -08:00
Zachary Anderson
5f76ac35bc
Revert "Add the refresh rate fields to perf_test (#99710)" (#99801)
This reverts commit a7c85996b8.
2022-03-08 23:04:49 -08:00
Chris Yang
a7c85996b8
Add the refresh rate fields to perf_test (#99710) 2022-03-08 17:25:15 -08:00
Jonah Williams
59859df126
Add Windows performance benchmark (#99564) 2022-03-04 15:41:14 -08:00
Emmanuel Garcia
f320d14062
Fix lifecycle for API level 28 (#99433) 2022-03-02 18:30:54 -08:00
gaaclarke
fdcd14464c
Starts using the --source flag to compile the dart registrant. (#98046)
* Starts using the `--source` flag to compile the dart registrant.

* updated general.shard tests

* Fixed the resident compiler flow

* added integration test

* made the integration test self contained

* renamed generated_main to dart_plugin_registrant

* cleaned up for review

* added task runner for ci

* added bringup and TESTOWNERS

* updated failure message
2022-03-01 16:24:47 -08:00
Emmanuel Garcia
6c818d772a
Add Android lifecycles test (#99319) 2022-03-01 15:26:21 -08:00
Emmanuel Garcia
e27f714497
Add regression test for #98973 (#99187) 2022-02-28 10:16:21 -08:00
gaaclarke
a7790d8e3a
Added a microbenchmark for Timeline events. (#98763) 2022-02-22 13:42:50 -08:00
George Wright
ff1e9b7b9e
Add a benchmark for simple DisplayLists to exercise the Raster Cache's cache admission algorithm (#97992) 2022-02-07 21:01:01 -08:00
Jim Graham
9c934041bd
Add benchmarks to measure impact of alpha saveLayers in DisplayLists (#97336) 2022-01-27 13:25:17 -08:00
Daco Harkes
0e2f51dfd0
FFI plugins (#96225) 2022-01-26 23:44:45 +01:00
Michael Goderbauer
5684b9f69e
Enable no_leading_underscores_for_local_identifiers (#96422) 2022-01-21 14:43:59 -08:00
stuartmorgan
2df7dca858
Fully support Dart-only mobile and macOS plugins (#96183) 2022-01-13 08:30:22 -08:00
Ian Hickson
57f097b532
Refactor devicelab bin/run.dart and other cleanup (#96320) 2022-01-10 13:40:17 -08:00
Zachary Anderson
764c3c18d4
Revert "Add total_ui_gc_time to benchmarks captured (#96125)" (#96137) 2022-01-04 12:14:36 -08:00
Kaushik Iska
da9e87021d
Add total_ui_gc_time to benchmarks captured (#96125) 2022-01-04 10:02:21 -08:00
Jim Graham
ee204880a5
Opacity Peephole optimization benchmarks (#94447) 2021-12-01 01:39:02 -08:00
Core
d8034538bd
feat: enable flavor option on test command (#89045) 2021-11-11 09:23:02 -08:00
Ian Hickson
7f7f487abc
Feature/cpu gpu memory gallery transition tests (#92626) 2021-10-28 16:03:03 -07:00
Jason Simmons
0bf2c9b3b7
Add local engine flag support to the SkSL caching performance benchmark scripts (#91773) 2021-10-14 10:57:32 -07:00
Ian Hickson
f90b019c68
Enable prefer_relative_imports and fix files. (#91573) 2021-10-11 10:28:07 -07:00
Ian Hickson
61a0add286
Enable avoid_redundant_argument_values lint (#91409) (#91462) 2021-10-08 09:25:14 -07:00
Zachary Anderson
b9d2177da0
Revert "Enable avoid_redundant_argument_values lint (#91409)" (#91461)
This reverts commit 5fd259be24.
2021-10-07 21:11:07 -07:00
Ian Hickson
5fd259be24
Enable avoid_redundant_argument_values lint (#91409) 2021-10-07 20:13:02 -07:00
Jenn Magder
4ef6fc1833
Update number of IPHONEOS_DEPLOYMENT_TARGET in plugin_test_ios (#91324) 2021-10-07 10:13:04 -07:00
Dan Field
22c1ca769b
Add a startup test that delays runApp (#91346) 2021-10-06 09:43:02 -07:00
stuartmorgan
8231592d71
Update minimum iOS version in plugin_test (#91099) 2021-10-05 10:33:04 -07:00
Alexander Aprelev
98120d19d3
Add a microbenchmark that measures performance of flutter compute. (#91257)
* Add a microbenchmark that measures performance of flutter compute.

This benchmark should show performance improvements when compute is switched to Isolate.exit

* Fix analyzer warnings, increase warmup iterations

* Fix license header
2021-10-04 14:16:23 -07:00
Jim Graham
abfb889fff
Add raster cache metrics to timeline summaries (#89393) 2021-09-04 18:31:03 -07:00
Jenn Magder
413f88f209
Do not try to codesign during plugin_test_ios (#88834) 2021-08-24 21:36:36 -07:00
Jonah Williams
060e4608b1
clean up stale or unnecessary TODOS (#88308) 2021-08-16 21:37:03 -07:00
Jenn Magder
f673c5c58f
Remove obsolete codegen_integration tests (#86962) 2021-07-23 14:56:04 -07:00
Jenn Magder
6881fca7b7
Respect plugin excluded iOS architectures (#86905) 2021-07-23 13:56:03 -07:00
Jenn Magder
81aeb33ce3
Increase Flutter framework minimum iOS version to 9.0 (#86840) 2021-07-22 13:21:05 -07:00
Kaushik Iska
df3fe587e8
Validate that min frame number is numeric (#86498) 2021-07-15 17:11:09 -07:00
Darren Austin
4c96cb26d2
Fixed a cast issue with host_mode_tests.dart. (#86392) 2021-07-13 17:31:02 -07:00
Darren Austin
ef5ffd08f6
Migrate devicelab framework code to null safety. (#86325)
(Attempt to reland #85993)
2021-07-13 12:14:50 -07:00
Darren Austin
08ef7752d5
Revert "Migrate core devicelab framework to null safety. (#85993)" (#86269)
This reverts commit 2175e64e4f.
2021-07-11 23:25:25 -07:00
Darren Austin
2175e64e4f
Migrate core devicelab framework to null safety. (#85993)
Migrate core devicelab framework to null safety.
2021-07-11 18:36:30 -07:00
gaaclarke
66fd0b28b0
replaced unawaited with synchronous / await calls where needed (#82839) 2021-06-24 16:22:43 -07:00
Jenn Magder
1d2f6c5296
Revert "Set up iOS native integration_tests in ui example project (#84890)" (#84892)
This reverts commit 05d5515732.
2021-06-18 18:05:00 -07:00
Jenn Magder
05d5515732
Set up iOS native integration_tests in ui example project (#84890) 2021-06-18 17:45:11 -07:00
Jenn Magder
03034e9e4a
Audit devicelab log streaming for nullability (#84820) 2021-06-18 00:14:03 -07:00
Jenn Magder
697928d21f
Revert "Set up iOS native integration_tests in ui example project (#84596)" (#84641)
This reverts commit ef2879a46f.
2021-06-15 10:09:41 -07:00
Jenn Magder
ef2879a46f
Set up iOS native integration_tests in ui example project (#84596) 2021-06-14 18:04:02 -07:00
Greg Spencer
ec5cfc001d
Fix build failure (#84597)
Fixes an incorrect cast that caused a build failure.

Landing on red to fix the build.
2021-06-14 15:17:25 -07:00
Greg Spencer
88f3811055
Turn on avoid_dynamic_calls lint, except packages/flutter tests, make appropriate changes. (#84476)
This adds avoid_dynamic_calls to the list of lints, and fixes all instances where it was violated.

Importantly, this lint is NOT turned on for flutter/packages/test, because those changes are happening in another PR: #84478
2021-06-14 14:16:57 -07:00
Dan Field
5cc7b6c3ea
Add option to stream logs to file for flutter logs and way to use it in devicelab runs (#84008) 2021-06-07 10:33:00 -07:00
Jenn Magder
94a41cde83
Mixed null safety in dev/devicelab (#84017) 2021-06-05 02:29:03 -07:00
Dan Field
83d51ecd1a
Delete obsolete TODO (#83966) 2021-06-03 23:29:05 -07:00
Jonah Williams
85df73d0e3
[flutter] ensure vmservice binding is registered in profile mode (#83913) 2021-06-03 17:59:03 -07:00
Anis Alibegić
c99ed373b3
Fixed large amount of spelling errors (#83744) 2021-06-02 10:14:06 -07:00
Alexandre Ardhuin
34059eec2c
enable lint prefer_interpolation_to_compose_strings (#83407) 2021-06-01 11:14:06 -07:00
gaaclarke
7cdd33fe99
added unawaited function for devicelab and turned on unawaited_futures linter (#82833) 2021-05-18 14:19:03 -07:00
Jonah Williams
d6fa9bbf91
[devicelab] await flutter create call in platform channels benchmark (#82762) 2021-05-18 08:55:38 -07:00
Jonah Williams
a3863b6592
[flutter_tools] support memory profiles from flutter drive (#82739) 2021-05-17 17:39:03 -07:00
Christopher Fujino
d6d3f0e0a8
add --verbose flag to pub global activate (#82719) 2021-05-17 11:39:28 -07:00
Dan Field
304b9c668e
Reland GC tracking benchmarks (#82069) 2021-05-14 19:59:24 -07:00
Christopher Fujino
855ada1d95
pub global deactivate devtools before installing (#82491)
* pub global deactivate devtools before installing
* upgrade devtools to latest
2021-05-13 17:13:03 -07:00
Jacob Richman
f82dedd5b9
Update perf_tests to use devtools 2.2.2 (#81869) 2021-05-10 15:22:50 -07:00
Dan Field
3101374f60
Revert "Reland GC benchmark changes again (#82042)" (#82059)
This reverts commit 99a947c3e5.
2021-05-07 13:19:56 -07:00
Dan Field
99a947c3e5
Reland GC benchmark changes again (#82042) 2021-05-07 12:29:01 -07:00
gaaclarke
a5f57b9e64
Added performance benchmarks for platform channels (#81414) 2021-05-07 10:24:03 -07:00
Zachary Anderson
4056ac057d
Revert "Reland GC related bench update (#81794)" (#82005)
This reverts commit 714dea5e03.
2021-05-06 19:03:59 -07:00
Dan Field
714dea5e03
Reland GC related bench update (#81794) 2021-05-06 16:49:04 -07:00
Kate Lovett
2d89ebb929
Revert "Fix benchmark (#81449)" (#81696)
This reverts commit a3893aea04.
2021-05-03 12:22:46 -05:00
Dan Field
a3893aea04
Fix benchmark (#81449) 2021-05-03 10:02:09 -07:00
Michael Goderbauer
3cbfe82d9d
Rev DevTools to 2.1.1 for tests (#81515) 2021-04-29 16:47:42 -07:00
Ian Hickson
6beafa7b16
Apply style guide regarding createTempSync pattern (#81090) 2021-04-27 14:59:03 -07:00
Phil Quitslund
08d7811f7f
sort directives (#81210) 2021-04-27 10:09:02 -07:00
xster
ff2dde2c7f
Reland double gzip wrapping NOTICES to reduce on-disk installed space (#80897) 2021-04-26 22:35:45 -07:00
Emmanuel Garcia
b0a63c4ffe
Reland the Dart plugin registry (#79669) 2021-04-23 15:34:04 -07:00
Jonah Williams
63ba1b23ca
[flutter_tools] add support for --bundle-sksl-path to flutter run / flutter drive (#80885) 2021-04-21 13:59:02 -07:00
Phil Quitslund
61c30c41b2
fix sort_directives violations (#80817) 2021-04-21 13:49:03 -07:00
Casey Hillers
c571804913
Revert "Reland [devicelab] Migrate Gallery to BuildTestTask (#78361)" (#79903)
This reverts commit 0c9781e3dd.
2021-04-06 16:28:54 -07:00
Casey Hillers
0c9781e3dd
Reland [devicelab] Migrate Gallery to BuildTestTask (#78361) 2021-04-06 16:04:03 -07:00
Jenn Magder
1f19fd7108
Take screenshot when startup perf test fails to run (#79514) 2021-03-31 21:32:49 -07:00
Jia Hao
8158b478d7
Add tests for integration tests to devicelab (#79258) 2021-03-30 22:29:03 -07:00
Jenn Magder
12880f7751
Point gallery test data to FLUTTER_TEST_OUTPUTS_DIR (#79332) 2021-03-29 21:39:03 -07:00
Kaushik Iska
e6920662ad
Add a test to exercise multiple async microtasks in frame workload (#79163) 2021-03-29 20:49:05 -07:00
Dan Field
374993ce88
fix tree (#79329) 2021-03-29 18:52:36 -07:00
Dan Field
750fd1d536
Avoid hard coding "build" for test output files (#79322) 2021-03-29 17:36:34 -07:00
Zachary Anderson
5efc7169eb
Reverts "Implement dartPluginClass support for plugins #74469" (#78623)
* Revert "Enable dart_plugin_registry_test (#76645)"

This reverts commit 109e0bb9f5.

* Revert "Apply changes caused by https://github.com/flutter/flutter/pull/76662 (#77093)"

This reverts commit cdca6485f0.

* Revert "Disable clang format in the plugin registrants (#76662)"

This reverts commit dadbd47d09.

* Revert "Disable warnings for the dart plugin registrant (#76561)"

This reverts commit 098ece522d.

* Revert "Remove dart_plugin_registry_test timeouts (#76838)"

This reverts commit 1610a27476.

* Revert "Implement dartPluginClass support for plugins (#74469)"

This reverts commit b7d4806243.

Kick.
2021-03-23 14:28:11 -07:00
Jenn Magder
3463946448
Take screenshot when drive fails (#78822) 2021-03-22 19:06:28 -07:00
Casey Hillers
c30fdfbe82
Revert "[devicelab] Migrate Gallery to BuildTestTask (#77956)" (#78355)
This reverts commit 93fb258602.
2021-03-16 12:20:36 -07:00
Casey Hillers
93fb258602
[devicelab] Migrate Gallery to BuildTestTask (#77956) 2021-03-16 11:58:03 -07:00
Zachary Anderson
9167083d09
Explain why devtools in pinned (#77657) 2021-03-09 08:34:03 -08:00
Jenn Magder
85e2c52b39
Increase activated devtools version to 2.0 (#77637) 2021-03-08 13:17:21 -08:00
Michael Goderbauer
cb867bbedc
Enable unnecessary_await_in_return lint (#77434) 2021-03-05 18:38:15 -08:00
Jenn Magder
e9e8ca7640
Add debug_second_compile_millis benchmark metric (#77420) 2021-03-05 15:33:28 -08:00
Casey Hillers
b0afe3464e
Create BuildTestTask (#77307) 2021-03-04 18:49:03 -08:00
Casey Hillers
7ed797b053
Revert "[devicelab] Separate build and test from Flutter gallery tests (#76415)" (#77086)
This reverts commit 297c7b5cf2.
2021-03-02 12:17:18 -08:00
Casey Hillers
297c7b5cf2
[devicelab] Separate build and test from Flutter gallery tests (#76415) 2021-03-02 11:49:02 -08:00
Jenn Magder
1610a27476
Remove dart_plugin_registry_test timeouts (#76838) 2021-02-25 20:26:02 -08:00
Jenn Magder
d69e2f10d8
Remove unused android_splash_screens integration test (#76429) 2021-02-22 14:46:04 -08:00
Jenn Magder
7cd99db0e4
Remove unused image_loading integration test (#76430) 2021-02-22 10:51:53 -08:00
Emmanuel Garcia
b7d4806243
Implement dartPluginClass support for plugins (#74469) 2021-02-19 09:22:45 -08:00
chunhtai
815976a8e4
change stack size test to test arm device (#75814) 2021-02-17 12:46:03 -08:00
Jenn Magder
d35058f5b8
Do not fail if 9xth_percentile_memory_usage is not recorded (#76153) 2021-02-16 15:56:02 -08:00
godofredoc
a0e8681452
Add flutter update packages to some tests. (#75530)
* Add flutter update packages to some tests.

Analyzer_benchmark and flutter_test_performance require to pub get
multiple directories. Rather than delegating the recipes to run flutter
update-packages we explicitly running on tests that need it.

Bug: https://github.com/flutter/flutter/issues/75524

* Add pub get directly on the analyis classes.

* Run flutter get in dev/tools.

* Fix directory path.

* Add more dependency directories.

* Fix typo.

* Add more dependencies.

* Add stocks dependency.

* Fix paths to some apps.

* Add more dependencies.

* Complete dependencies definition.

* More analysis dependencies.

* Add examples dependencies.

* Dev tool dependencies.

* Use update-packages for analyzer benchmark.
2021-02-11 15:30:55 -08:00
chunhtai
713fa8e360
Reland "add a benchmark test for stack size (#75039)" (#75434)
* Reland "add a benchmark test for stack size (#75039)"

This reverts commit de683de9d5.

* fix typo
2021-02-04 12:16:31 -08:00
Shi-Hao Hong
de683de9d5
Revert "add a benchmark test for stack size (#75039)" (#75383)
This reverts commit 9d9109aa6f.
2021-02-04 10:26:04 +08:00
chunhtai
9d9109aa6f
add a benchmark test for stack size (#75039) 2021-02-03 15:16:03 -08:00
Jenn Magder
91aad2bb06
Avoid iOS local networking dialog in integration tests (#75163) 2021-02-01 13:33:25 -08:00
Jonah Williams
7ee910b0a8
[flutter_tools] ensure --dart-define can parse args with commas (#73798) 2021-01-15 13:51:32 -08:00
Yuqian Li
c4114a7527
Remove more flaky non-e2e tests (#74062)
Fixes https://github.com/flutter/flutter/issues/73873
2021-01-15 13:22:26 -08:00
LongCatIsLooong
cbe72db135
macrobenchmark: TextField with complex paragraph (#73374) 2021-01-12 15:44:01 -08:00
Jenn Magder
66145fe652
Force plugins to inherit minimum macOS version from Flutter app (#72764) 2021-01-11 11:16:12 -08:00
Yegor
1f8501f0a4
web benchmark choose renderer explicitly (#73582) 2021-01-08 12:04:03 -08:00
nturgut
272b23cabd
[web] make sure tests and benchmarks use correct backend - add autodetect false (#73047)
* add autodetect false

* fix typo
2020-12-28 10:55:53 -08:00
nturgut
14206399c3
make sure tests and becnhmarks use correct backend (#72628) 2020-12-21 09:55:23 -08:00
xster
ca82c572b7
Revert "let NOTICES be double gzip wrapped to reduce on-disk installed space (#71899)" (#72438) 2020-12-16 10:58:19 -08:00
xster
4ed3432e8f
let NOTICES be double gzip wrapped to reduce on-disk installed space (#71899) 2020-12-15 13:29:14 -08:00
Kate Lovett
1c244da28d
Add verbose flag to hot mode tests (#71610) 2020-12-02 16:47:35 -06:00
Jonah Williams
1e0628001e
[devicelab] dont use set modified for hot reload bench (#70965) 2020-11-20 13:02:28 -08:00
Jonah Williams
9424474793
[devicelab] update hot reload benchmark to be more representative (#70643)
update hot reload benchmark to be more representative
2020-11-18 14:54:01 -08:00
Jim Graham
61460d255a
preserve detailFiles entry in TaskResult JSON (#70668)
* preserve detailFiles entry in TaskResult JSON
2020-11-17 16:05:54 -08:00
Yuqian Li
f7811ea541
e2e perf tests don't have CPU/GPU/memory metrics (#70439)
Address comment in https://github.com/flutter/flutter/pull/66386/files#r522671103
2020-11-13 11:16:50 -08:00
Yegor
9e5d22d101
Say [APP] instead of [Gallery] in web benchmarks (#69624) 2020-11-03 10:13:07 -08:00
Jenn Magder
bb495ba90c
Do not fail if average_memory_usage is not recorded (#69339) 2020-10-29 16:09:50 -07:00
Jonah Williams
31c96b5d83
[devicelab] do not fail test if CPU measurement is not recorded (#69268) 2020-10-29 09:32:34 -07:00
Jonah Williams
18341efa2f
[devicelab] retry startup of start up test (#69272) 2020-10-29 09:19:07 -07:00
Jonah Williams
83d4351e0c
[devicelab] work around occasional null values for devtools (#69273)
* [devicelab] work around occasional null values for devtools

* Update perf_tests.dart
2020-10-29 09:18:38 -07:00
Jenn Magder
13896b3bd1
Decrease device discovery timeout from 10 to 5 seconds (#69266) 2020-10-28 19:43:19 -07:00
Jenn Magder
135aab9cf5
Increase device run perf test timeout (#69260) 2020-10-28 18:29:04 -07:00
Jonah Williams
2c12a8d917
[devicelab] remove twc enabled test and uncaught image test (#68518) 2020-10-24 13:19:51 -07:00
Jonah Williams
ab20ae782a
[devicelab] uinstall during memory test (#68548) 2020-10-20 09:49:50 -07:00
Jonah Williams
938fae9c99
remove --machine flag (#68538) 2020-10-19 17:06:28 -07:00
Jonah Williams
a152c0f429
[devicelab] reland: reduce iterations of startup test, use application binary (#68532)
These devicelab tests are incredibly flaky. Currently they will build & install the same application 15 times. This causes the temp storage to fill up on android, and has a good chance of flaking on iOS due to install issues.

The change from 3 to 15 increased total test time for 2 -> 12 minutes on iOS, or roughly 5 more test equivalents for every single iOS test. Reduce the iteration count back to 5

Uninstall the app after each run so temp storage does not fill up.
2020-10-19 16:51:28 -07:00
Jonah Williams
4a3e2a1af5
Revert "[devicelab] reduce iterations, uninstall at end, and use --application-binary in all startup tests (#68516)" (#68531)
This reverts commit a4c2075a45.
2020-10-19 15:05:46 -07:00
Jonah Williams
8262f808ba
Revert "[devicelab] fix flutter gallery app name (#68528)" (#68530)
This reverts commit 0f69f8e53b.
2020-10-19 15:05:10 -07:00
Jonah Williams
0f69f8e53b
[devicelab] fix flutter gallery app name (#68528) 2020-10-19 14:48:05 -07:00
Jonah Williams
a4c2075a45
[devicelab] reduce iterations, uninstall at end, and use --application-binary in all startup tests (#68516)
These devicelab tests are incredibly flaky. Currently they will build & install the same application 15 times. This causes the temp storage to fill up on android, and has a good chance of flaking on iOS due to install issues.

The change from 3 to 15 increased total test time for 2 -> 12 minutes on iOS, or roughly 5 more test equivalents for every single iOS test. Reduce the iteration count back to 5

Uninstall the app after each run so temp storage does not fill up.
2020-10-19 14:16:04 -07:00
Jonah Williams
03db9bf918
[devicelab] migrate defines test to DriverTest (#68407)
Removes one-off handling for defines test and forces it to run through the DriverTest interface
2020-10-19 12:28:43 -07:00
Jonah Williams
223df0c351
[devicelab] split end to end tests into driver targets (#68410)
Split the integration_ui target into individual driver tests. Remove disabled screenshot tests
2020-10-19 12:28:13 -07:00