Commit Graph

332 Commits

Author SHA1 Message Date
liyuqian
19a6a6f431
Reland "Shader warm up (#27660)" (#28537)
This reverts commit adc8e159a5.

This should be safe to land once https://github.com/flutter/flutter/pull/28530 gets merged

Merge on yellow doc test because the doc test is actually green.
2019-02-27 09:33:08 -08:00
liyuqian
adc8e159a5
Revert "Shader warm up (#27660)" (#28376)
This reverts commit a44f174efc.

Reason: start_up tests become flaky.

See https://github.com/flutter/flutter/issues/28374

TBR: xster

Merge on red to fix the tree
2019-02-22 17:32:37 -08:00
liyuqian
a44f174efc
Shader warm up (#27660)
This patch adds a default shader warm up process which moves shader compilation from the animation time to the startup time. This also provides an extension for `runApp` so developers can customize the warm up process.

This should reduce our worst_frame_rasterizer_time_millis from ~100ms to ~20-30ms for both flutter_gallery and complex_layout benchmarks. Besides, this should also have a significant improvement on 90th and 99th percentile time (50%-100% speedup in some cases, but I haven't tested them thoroughly; I'll let our device lab collect the data afterwards).

The tradeoff the is the startup time (time to first frame). Our `flutter run --profile --trace-startup` seems to be a little noisy and I see about 100ms-200ms increase in that measurement for complex_layout and flutter_gallery. Note that this only happens on the first run after install or data wipe. Later the Skia persistent cache will remove the overhead.

This also adds a cubic_bezier benchmark to test the custom shader warm up process.

This should fix https://github.com/flutter/flutter/issues/813 (either by `defaultShaderWarmUp`, or a `customShaderWarmUp`).
2019-02-22 15:37:02 -08:00
Dan Field
04b7a5b366
Reduce memory benchmark perf tests outer loop to 10. (#28281)
* reduce iterations to avoid test timing out
2019-02-21 14:46:24 -08:00
Jonah Williams
67cf21577f
Add basic codegen app to be used for integration testing and benchmarks (#27257) 2019-02-14 23:17:16 -08:00
liyuqian
de321b9cc1
Add scroll performance test for flutter_gallery (#27513)
Previously, we had a scroll_perf drive test but we were not exercising
it on our device lab. This helps us tracking issues like:
https://github.com/flutter/flutter/issues/24782
2019-02-04 22:54:11 -08:00
Jason Simmons
b5ca595d61
Remove icudtl.dat from APK asset size checks (#27505) 2019-02-04 12:40:43 -08:00
Michael Goderbauer
6d20ff2446
Remove all obsolete "// ignore:" (#27271) 2019-01-30 08:56:12 -08:00
liyuqian
8ad4cbe06e
Add cull opacity perf test to device lab (#25381)
For https://github.com/flutter/flutter/issues/24712

This test verifies that https://github.com/flutter/engine/pull/6923 will speedup the average rasterize time of this test from ~150ms to ~10ms

Please see non-auto-generated files in 37b21d9fb4
2018-12-17 22:01:07 -08:00
Jonah Williams
938dd5a4aa
Revert "Update driver script to execute test through test_core" (#24401)
* Revert "Add dashing config file for generating docset from flutter docs (#24374)"

This reverts commit ec8ca8606c.

* Revert "Update driver script to execute test through test_core (#24168)"

This reverts commit 6c62cf337f.
2018-11-15 11:17:00 -08:00
Jonah Williams
6c62cf337f
Update driver script to execute test through test_core (#24168) 2018-11-15 10:50:25 -08:00
Michael Goderbauer
b8c29a1490
Add benchmark for LICENSE file size (#23948) 2018-11-05 13:57:40 -08:00
Michael Klimushyn
43c99404c5
Test flutter create --offline (#23278)
Verifies that flutter tools isn't rebuilt. Doesn't truly check that the
created output is runnable.
2018-10-22 16:39:28 -07:00
Greg Spencer
abb1758edf
Fix places in devicelab tests where it expects the app template. (#22696)
I missed some create template conversions in the devicelab directory.
2018-10-04 15:30:08 -07:00
Greg Spencer
3246a66374
Make devicelab tests use 'app' template (#22694)
Fixes the build after my module rename broke the Windows tests.
2018-10-04 14:30:59 -07:00
Alexandre Ardhuin
d340e2f229
apply lint prefer_void_to_null in dev/ (#22661) 2018-10-04 18:44:23 +02:00
Alexandre Ardhuin
2ea1d81cdc
sort_constructors_first (#22575) 2018-10-04 07:28:07 +02:00
Alexandre Ardhuin
f62afdcf57
add missing type parameter on methods (#22096) 2018-10-01 21:29:08 +02:00
liyuqian
872eba2d4c
Track 90th, 99th percentile instead of missed_frame_build_budget_count (#21754)
See https://github.com/flutter/flutter/pull/19121#issuecomment-419520765
2018-09-14 13:24:13 -07:00
Alexandre Ardhuin
d927c93310
Unnecessary new (#20138)
* enable lint unnecessary_new

* fix tests

* fix tests

* fix tests
2018-09-12 08:29:29 +02:00
Jonah Williams
c6e8a513f0
Add android semantics integration test to device lab (#20971) 2018-09-06 12:31:18 -07:00
Chris Bracken
9f9676c4c6
Eliminate Dart 1 support in devicelab tests (#21240)
Flutter commands all default to --preview-dart-2. Eliminate support for
the use of this option and the previewDart2 parameter in dev/devicelab
tests.
2018-08-30 17:17:37 -07:00
Ian Hickson
53b635817c
Re-enable missing_return (#20844) 2018-08-28 11:22:57 -07:00
amirh
1f31c3b354
Run the embedded Android views integration test on the device lab. (#20841) 2018-08-23 10:26:00 -07:00
Ian Hickson
1993a67381
Performance test cleanup (#20652)
* Fix TODO syntax.
* Clarify messages for some timeouts, to aid debugging.
* Increase some other timeouts that were a needlessly short, to reduce sources of flakes.
* Remove some more timeouts that were mostly redundant, to remove complexity.
* Minor style cleanup.
* Remove some dangerous traps (specifically, hide the explicit start/end times in TimedEvent since they shouldn't matter).
2018-08-17 13:19:28 -07:00
Ian Hickson
3dec6a6930
Clean up usage of temporary directories (#20682)
All temporary directory start with `flutter_` and have their random component separated from the name by a period, as in `flutter_test_bundle.YFYQMY`.

I've tried to find some of the places where we didn't cleanly delete temporary directories, too. This greatly reduces, though it does not entirely eliminate, the directories we leave behind when running tests, especially `flutter_tools` tests.

While I was at it I standardized on `tempDir` as the variable name for temporary directories, since it was the most common, removing occurrences of `temp` and `tmp`, among others.

Also I factored out some common code that used to catch exceptions that happen on Windows, and made more places use that pattern.
2018-08-17 13:17:23 -07:00
Ian Hickson
9e9ac1a16c
Refactor analysis benchmark and collect more data (#20282)
* Refactor analysis benchmark and collect more data (#20169)

This does some code cleanup to avoid duplication, improves the logs,
and records min and max values as well as the mean.

* Fix typo that led to infinite loop
2018-08-06 18:15:44 -07:00
Ian Hickson
b81e361192
Revert "Refactor analysis benchmark and collect more data" (#20280)
* Revert "increase size of user account drawer headers to 48 by 48 (#20266)"

This reverts commit 4a7b4a4dde.

* Revert "EditableText Cursor can be set to not blink for testing (#20004)"

This reverts commit d041b319e8.

* Revert "Refactor analysis benchmark and collect more data (#20169)"

This reverts commit 5ea0a13598.
2018-08-06 15:34:08 -07:00
Ian Hickson
5ea0a13598
Refactor analysis benchmark and collect more data (#20169)
This does some code cleanup to avoid duplication, improves the logs,
and records min and max values as well as the mean.
2018-08-06 12:47:09 -07:00
Ian Hickson
872e88cec7
Reduce noise on the memory benchmarks (#19630) (#20163)
- Check memory usage in release builds, not profile.
- Use multiple runs and average the results.
2018-08-06 12:46:51 -07:00
Devon Carew
3ead9c0f0c
reduce the number of runs on the analyze benchmakr (#20155) 2018-08-02 16:13:05 -07:00
Ian Hickson
a2ca14d71e
Revert "Reduce noise on the memory benchmarks" (#20158)
* Revert "Add pub cache, artifacts, pkgs to Cirrus cache (#20080)"

This reverts commit 07e93b385c.

* Revert "Reduce noise on the memory benchmarks (#19630)"

This reverts commit 8eb5cb7dc0.
2018-08-02 13:56:44 -07:00
Ian Hickson
8eb5cb7dc0
Reduce noise on the memory benchmarks (#19630)
- Check memory usage in release builds, not profile.
- Use multiple runs and average the results.
2018-08-02 12:22:15 -07:00
Alexandre Ardhuin
eda03e2586
re-re-enable lint unnecessary_const (#20103) 2018-08-02 12:02:32 +02:00
Devon Carew
1393c73077
use the best of four results for flutter analyze --benchmark (#20058) 2018-08-01 11:24:41 -07:00
liyuqian
a4cc34dee6
Add tiles_scroll_perf test (#19833) 2018-07-27 14:59:38 -07:00
Todd Volkert
00aac68e2d
Revert flutter/flutter#19592 (#19861)
It was causing problems rolling Flutter into Fuchsia
2018-07-27 08:44:39 -07:00
Danny Tuppeny
868d8c1458
In microbenchmarks, ensure flutter quits by sending q to stdin (#19400)
* Tell flutter to quit directly

Fixes #19208.

* Tweak comment
2018-07-23 18:53:19 +01:00
Alexandre Ardhuin
27018359d2
re-enable lint unnecessary_const (#19592)
* re-enable lint unnecessary_const

* remove trailling whitespaces

* remove unnecessary const (after merge)
2018-07-23 08:31:48 +02:00
Leaf Petersen
32f94443cc
Remove uses of deprecated constants and change int.parse to int.tryParse (#19575)
* Remove uses of deprecated constants
* Change int.parse to int.tryParse where appropriate
2018-07-20 15:07:24 -07:00
Chris Bracken
c0294e3a02
Eliminate Dart 1 benchmarks (#19430)
Flutter app development has been Dart 2 only for a significant amount of
time, and we're about to remove all support for Dart 1.
2018-07-18 15:56:51 -07:00
Ian Hickson
f630f90d6d
Revert "enable lint unnecessary_const (#19342)" (#19423)
This reverts commit cc1cf13eec.
2018-07-16 13:30:27 -07:00
Alexandre Ardhuin
cc1cf13eec
enable lint unnecessary_const (#19342) 2018-07-16 21:43:48 +02:00
Alexander Aprelev
7ebf2728dc
Revert "Revert "Roll engine to fed2ea458ed49088d33eddabc546ba56d600c717 (includes dart roll) again (#19277)" (#19295)" (#19322)
This reverts commit cc122f1672.
2018-07-12 14:59:22 -07:00
Danny Tuppeny
2e215fdfe4
Align expressions better (#19256) 2018-07-12 16:24:39 +01:00
Alexander Aprelev
cc122f1672
Revert "Roll engine to fed2ea458ed49088d33eddabc546ba56d600c717 (includes dart roll) again (#19277)" (#19295)
This reverts commit 700cb767c9 as it
breaks Windows tests.
2018-07-11 19:52:28 -07:00
Alexander Aprelev
700cb767c9
Roll engine to fed2ea458ed49088d33eddabc546ba56d600c717 (includes dart roll) again (#19277)
* Revert "Revert "Roll engine to fed2ea458ed49088d33eddabc546ba56d600c717 (includes dart roll) (#19044)" (#19276)"

This reverts commit cf932490b7 as it also
includes fix for type error that broke tests.

* Add type cast for dart2 type checks.

* Move up to latest goldens

* Make inDirectory() type-parameterized.

* Add typecasting to transitions_perf_test.dart and microbenchmarks.

* Add boolean flag initialization in save_catalog_screenshots.dart

* Add type conversion to gallery transition test
2018-07-11 19:26:33 -07:00
Danny Tuppeny
127d1f85dd
Improve error message in microbenchmarks when flutter run lives too long (#19200)
This should make it more obvious from the error what's gone wrong if we see a repeat of #19096 (previously the error was something like "Bad state: future already completed").
2018-07-10 17:09:39 +01:00
Danny Tuppeny
5413560415 Revert "Add temporary debug prints to identify devicelab failures (#19093)" (#19095)
This reverts commit edc16473bf.
2018-07-05 18:58:26 +02:00
Danny Tuppeny
edc16473bf
Add temporary debug prints to identify devicelab failures (#19093) 2018-07-05 14:16:44 +01:00
Danny Tuppeny
e1d396d5ca
Attempt to fix race in microbenchmark error handling code (#19092)
Await stream cancellations before trying to complete
2018-07-05 13:06:43 +01:00
Mikkel Nygaard Ravn
c34daae2d6
Avoid repeating microbenchmark (#19087) 2018-07-05 11:11:17 +02:00
liyuqian
8c1d8a6a6b
Add 90th and 99th percentiles to dashboard (#18930)
This fixes #18727
2018-06-28 16:19:23 -07:00
Ian Hickson
35ad2a786d
Remove race conditions involving finding available ports (#18698)
This is an attempt to reland #18488 with less breakage on macOS.
2018-06-27 16:44:28 -07:00
Jason Simmons
43e6372128
Update the ICU data file asset path in the devicelab tests (#18854) 2018-06-26 16:01:31 -07:00
Michael Goderbauer
f11b790b8c
Track size of App.framework and Flutter.framework for Hello World (#18701) 2018-06-22 09:27:31 -07:00
Greg Spencer
08d1ec36de Roll engine to 0c119932c0d6cb30dafd75ec717d1eda76fd7651 (#18575) 2018-06-19 09:58:20 +02:00
Michael Goderbauer
9a8c4dfef6
Report compressed/uncompressed size of icudtl, snapshots, and libflutter (#18583)
Reported for the hello_world compile benchmark in devicelab for now.
2018-06-18 17:12:14 -07:00
Michael Goderbauer
ebb174b0bf
Add size benchmark for hello_world (#18522) 2018-06-16 00:08:33 -07:00
Michael Goderbauer
2a19dae28b
Fix aot_snapshot_size_* benchmarks (#18520)
... and make sure they fail if they can't report the size.
2018-06-15 17:17:28 -07:00
Ian Hickson
0fa5ba43a1
Revert "Remove race conditions involving finding available ports (#18488)" (#18521)
This reverts commit 7750872251.
2018-06-15 15:37:20 -07:00
Ian Hickson
7750872251
Remove race conditions involving finding available ports (#18488) 2018-06-15 14:16:18 -07:00
Michael Goderbauer
409007827b
Keep TextFields visible when keyboard comes up (#18291)
Fixes #10826.

Also in this PR: If you focus a text field, scroll it out of view and then start typing it will scroll back into view.
2018-06-15 10:11:10 -07:00
Chris Bracken
2ae48845a8
Revert elimination of Dart 1 (#18460)
fuchsia_tester.dart still assumes Dart 1. Previously, it ran tests directly
from source, flutter_platform.dart automatically runs a kernel compile when
operating in Dart 2 mode, but this assumes a functional Dart SDK is available
in the artifacts directly, and fuchsia_tester.dart mocks out the artifacts
directory with an empty temp dir.

Remaining work is:
1. Get the frontend server building as a dependency on Fuchsia.
2. Patch fuchsia_tester.dart to use a valid Dart SDK and frontend server.

This also reverts migration to Dart 2 typedef syntax.

This reverts commit 6c56bb2. (#18362)
This reverts commit 3daebd0. (#18316)
2018-06-13 12:46:39 -07:00
Greg Spencer
6c56bb2420
Update typedef syntax to use Function notation and turn on lint for old notation. (#18362)
Now that Dart 1 is turned off, reapplying my change to turn on the prefer_generic_function_type_aliases analysis option, and fix all the typedefs to Dart 2 preferred syntax.

Also eliminated the unused analysis_options_repo.yaml file and turned on public_member_api_docs in analysys_options.yaml.

No logic changes, just changing the typedef syntax for all typedefs, and updating analysis options.
2018-06-11 15:51:45 -07:00
Chris Bracken
3daebd059c
It's time to #deleteDart1 (again) (#18316)
* It's time to #deleteDart1 (#18293)

Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.

* Fix indentation, remove no longer necessary .toList()

* Only push udpated kernel if >0 invalidated srcs
2018-06-08 15:49:24 -07:00
Chris Bracken
9495df4bc3
Revert "It's time to #deleteDart1 (#18293)" (#18313)
Several benchmarks started failing during kernel compile:
1. Timeouts:
   * commands_test
   * hot_mode_dev_cycle__benchmark
   * run_machine_concurrent_hot_reload
   * service_extensions_test

2. Kernel compiler crash:
   * hot_mode_dev_cycle_linux__benchmark
   * routing_test

3. Unexpected termination:
   * hot_mode_dev_cycle_win__benchmark

This reverts commit ed63e7082a.
2018-06-08 12:43:11 -07:00
Chris Bracken
ed63e7082a
It's time to #deleteDart1 (#18293)
Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.
2018-06-08 11:04:57 -07:00
Alexandre Ardhuin
09276bea25
enable lint prefer_equal_for_default_values (#18156) 2018-06-05 08:50:40 +02:00
Greg Spencer
b921fdc58e
Revert "Update typedef syntax to use Function notation and turn on lint for old notation. (#18035)" (#18041)
This reverts commit 3258602073.
2018-05-30 13:51:14 -07:00
Greg Spencer
3258602073
Update typedef syntax to use Function notation and turn on lint for old notation. (#18035)
Fixes #18028

Just changes typedef declarations, no logic changes.
2018-05-30 12:13:58 -07:00
Sarah Zakarias
a3e71bf11f
Add startup test for Platform View example (#17645) 2018-05-16 13:24:47 +02:00
Todd Volkert
d820e5f3b1
Use deprecated io constants (#17278)
The new values are not ready for use yet inside Google
2018-05-03 22:27:29 -07:00
Yegor
b2c98f9a4e
Roll engine to e976be13c51448f89107d082ec81e2b6731671fa (#17266)
* Roll engine to e976be13c51448f89107d082ec81e2b6731671fa

* move away from deprecated constants
2018-05-03 19:24:48 -07:00
Yegor
a90a850462
bump benchmark timeouts (#16817) 2018-04-20 13:57:06 -07:00
Chinmay Garde
9a0b562524
When parsing logs, filter away lines that are irrelevant to the microbenchmark. (#16678) 2018-04-17 16:57:47 -07:00
Todd Volkert
ec26a7baca
Fail with better error message in microbenchmarks (#16676) 2018-04-17 14:34:09 -07:00
Alexander Aprelev
254c45e471
Add hotReloadMillisecondsToFrameAfterChange benchmark. (#16406)
This fixes https://github.com/dart-lang/sdk/issues/32598.
2018-04-10 07:25:22 -07:00
Mikkel Nygaard Ravn
77403ba6ec
Experiment to understand test flakiness (#15872) 2018-03-29 00:24:05 +02:00
Mikkel Nygaard Ravn
ce28378178
Change to absolute path (#15827) 2018-03-23 11:37:56 +01:00
Mikkel Nygaard Ravn
f90c54c6ed
Fix path to .bat file (#15824) 2018-03-22 13:22:44 +01:00
Mikkel Nygaard Ravn
145abb6dbf
Stop Gradle daemon before attempting to delete Flutter project (#15819) 2018-03-22 12:33:29 +01:00
Alexander Aprelev
ffc7fac288
Pass --no-preview-dart-2 for dart1 tests. (#15800) 2018-03-21 13:40:17 -07:00
Alexander Aprelev
21ad09fe57
With dart2 being default, move preview_dart_2 tests to dart1. (#15667)
* With dart being default, move preview_dart_2 tests to dart1.

* Swap out defaults for preview-dart-2 option in tests. Make dart1 explicit

* Move suffix around

* Rename files
2018-03-19 12:11:18 -07:00
Alexander Aprelev
2f8474f9aa
Roll engine to b6df7a637498ca9beda1fa9cd7210e3202ea599f. (#15444)
* Roll engine to b6df7a637498ca9beda1fa9cd7210e3202ea599f.

Changes since last roll:
```
b6df7a637 Roll dart to 290c576264faa096a0b3206c71b2435309d9f904. (#4771)
a6764dbd5 Add sources for Fuchsia target. (#4763)
2d5900615 [fuchsia] Remove unused header file. (#4769)
9717063b7 Revert "Roll dart to c080951d45e79cd25df98036c4be835b284a269c. (#4767)" (#4768)
9a9814312 Roll dart to c080951d45e79cd25df98036c4be835b284a269c. (#4767)
e74e8b35c [async] Update includes of async headers to new path (#4760)
e2c4b2760 Use Dart 2 camel case constants in the engine Dart libraries (#4766)
9c1e48434 Updates for Fuchsia roll. (#4765)
14c940e27 Switch from fxl::Mutex to std::mutex (#4764)
debf82c0b Roll Garnet (#4759)
5bffdefbb Use weak pointers to the accesibility bridge from objects vended to the UIKit accessibility framework. (#4761)
```
2018-03-12 15:44:25 -07:00
Mikkel Nygaard Ravn
658f9e6c83
Make plugins work in Swift projects (#15437) 2018-03-12 17:33:10 +01:00
Alexander Aprelev
758b302c8d
Add --preview-dart-2 relaunch test. (#15117)
* Add a test that measures/verifies start from warm state(app was previously compiled).

* Remove json file from the first run.
2018-03-06 14:33:26 -08:00
Mikkel Nygaard Ravn
2000435728
Move plugin injection to just after pub get (#14743) 2018-02-16 10:17:28 +01:00
Mikkel Nygaard Ravn
c2e1bc0322
Revert "Move plugin injection to just after pub get (#14560)" (#14734)
This reverts commit b3e4976ca2.
2018-02-15 22:55:36 +01:00
Mikkel Nygaard Ravn
b3e4976ca2
Move plugin injection to just after pub get (#14560) 2018-02-15 22:17:12 +01:00
Alexandre Ardhuin
c02b6a8bcf
some whitespace cleanup (#14443) 2018-02-02 23:27:29 +01:00
xster
82ddabd32a
Fastlane instructives for tool publishing the Gallery (#14347)
* first round fastlane working on Android

* clean up and start ios

* Partial iOS

* Got a fastfile that works on iOS

* Finalize Android push instructions

* Finalize instructions for iOS

* Extra cleanup and optipng

* Disable readme autogen and add a default readme
2018-01-31 11:21:37 -08:00
Alexander Aprelev
5b02a5ec58
Fix iOS automatic signing on buildbots. (#14024) 2018-01-10 14:01:20 -08:00
Leaf Petersen
b3cfa785c4
Uncontroversial and backwards compatible 2.0 SDK fixes (#13723)
Small code changes as part of moving the framework SDK forward to a 2.0 dev version.
2017-12-21 13:39:03 -08:00
Yegor
1c6decaa5e
update APK path in devicelab tests (#13540)
* update APK path in devicelab tests

* add missing transitive dependency on archive

* use path used by flutter tools
2017-12-13 13:12:10 -08:00
Vyacheslav Egorov
41a5e5d8f6 Change few devicelab tests to support testing against local engine (#13268)
* Change some of the dev/devicelab tests to support testing against local engine.

We can already configure flutter tools to use local engine by setting
FLUTTER_ENGINE environment variable. However when this variable is set
this also requires setting --local-engine to specify which flavor of
engine to use.

This change changes tests in dev/devicelab to pass a sensible default for
--local-engine, e.g. when testing hot reload on Android we pass android_debug
and when testing release AOT build for IOS we pass ios_release.

* Fix analysis issues

* Update utils.dart
2017-11-30 12:15:27 -08:00
Ian Hickson
1bef312fb1
Actually run "flutter create" before compiling the default app... (#13246) 2017-11-29 15:49:50 -08:00
Alexander Markov
d2fc828aa1
Run Flutter microbenchmarks in --preview-dart-2 mode (#13225)
Existing microbenchmarks task is extended instead of adding a new one to consume less horizontal space on Flutter dashboard.
2017-11-28 20:59:57 -08:00
Ian Hickson
216ff81cd2
Fix devicelab (#13239) 2017-11-28 17:41:40 -08:00
Ian Hickson
d104106e78
Refactor compilation devicelab tests (#13218)
This fills out the following matrix:

App: `flutter create` default app, Complex Layout, and Gallery.
Platforms: Linux/Android, Mac/iOS, Windows/Android
Build modes: full release, aot blobs, debug
Dart: normal, 2.0 preview

It also renames all the tests that used "build" to mean compilation
build to use "compile" instead, so that it's much clearer on the
dashboard.

Also, mark some flaky tests that are no longer flaky as not flaky.
Also, remove uses of call() since that's probably going to go away at
some point.
2017-11-28 17:21:01 -08:00
Mikkel Nygaard Ravn
d06482cc79
Add integration test of textures (#13122) 2017-11-22 14:16:25 +01:00
Greg Spencer
0259be90b8
Fix spelling errors in all the dartdocs. (#13061)
I got tired of drive-by spelling fixes, so I figured I'd just take care of them all at once.

This only corrects errors in the dartdocs, not regular comments, and I skipped any sample code in the dartdocs. It doesn't touch any identifiers in the dartdocs either. No code changes, just comments.
2017-11-17 10:05:21 -08:00
Yegor
91bd9bc4f8
delay taking screenshot to allow GPU thread to render the frame (#12896)
* delay taking screenshot to allow GPU thread to render the frame

* address comments
2017-11-06 17:33:48 -08:00
Yegor
35bb855c0d
make microbenchmark logging verbose (#12858)
* make microbenchmark logging verbose

* prefix JSON so it can be parsed in verbose mode
2017-11-02 18:11:34 -07:00
Michael Goderbauer
2a11e896ee Update clean command in devicelab tests (#12640) 2017-10-19 13:20:32 -07:00
Alexandre Ardhuin
700dc9f752 enable lint avoid_function_literals_in_foreach_calls (#12607) 2017-10-19 08:16:16 +02:00
Alexander Aprelev
e20bff4d79 Add benchmark score keys to the hot mode test report. (#12547)
This is follow-up to be1467e0cd.
2017-10-14 11:29:37 -07:00
Alexander Aprelev
c2dfa6a186 Change pattern triggering next step. (#12543)
Previously chosen pattern was generated in the output not quite at the end of the hot reload process.
2017-10-13 21:14:51 -07:00
Alexander Aprelev
be1467e0cd Restructure hot mode test so it runs interactively. (#12519)
* Restructure hot mode test so it runs interactively.

This allows to add a benchmark for hot reload after actual source code change.

* Add curly braces, refactory copyRecursive
2017-10-13 20:06:35 -07:00
Alexander Aprelev
ac1546c794 Add hotReloadInitialDevFSSyncMilliseconds to track how long user have… (#12417)
* Add hotReloadInitialDevFSSyncMilliseconds to track how long user have to wait before being able to do first reload.

This stat is significantly different between existing and preview-dart-2 setting (for the latter, this stat is ~3x slower: 13s vs ~4s).

* Remove ws

* Cleanup timer-related code
2017-10-06 08:39:46 -07:00
Alexander Aprelev
1a3097025d Clone hot reload benchmark for --preview-dart-2 option. (#12251)
* Clone hot reload benchmark for --preview-dart-2 option.

* Get rid of linux and win preview_dart_2 (only android would be sufficient for now). Refactor code into lib/tasks

* Revert 2016 to 2017

* Mark new test as flaky
2017-09-28 12:45:54 -07:00
Ian Hickson
efb45ea788 Trivial nit fixes (#12285) 2017-09-27 16:13:48 -07:00
Yegor
ba5b5e7f6f only tap on widgets reachable by hit testing (#11767)
* only tap on widgets reachable by hit testing

* use FractionalOffset

* added tests

* check finder finds correct widget

* undo unintentional changes

* address comments

* style fix

* add Directionality in test

* fix analysis warning
2017-09-11 09:46:42 -07:00
Alexander Aprelev
c5750cd7ea Introduce --preview-dart-2 option to run new frontend compiler in flutter tools. (#11741)
This adds --preview-dart-2 flag that enables use of Dart 2.0 Frontend in Flutter tools.
2017-08-31 13:35:55 -07:00
Mikkel Nygaard Ravn
9496e6dfa7 Support for app flavors in flutter tooling, #11676 retake (#11734) 2017-08-23 10:55:35 +02:00
Mikkel Nygaard Ravn
7c3a45dc6e Revert "Support for app flavors in flutter tooling (#11676)" (#11729)
This reverts commit 8d07d3f652.
2017-08-22 12:24:29 +02:00
Mikkel Nygaard Ravn
8d07d3f652 Support for app flavors in flutter tooling (#11676) 2017-08-22 10:38:23 +02:00
Sarah Zakarias
f1f5d4f3a8 Add integration test for platform interaction (#11580) 2017-08-11 10:38:12 +02:00
Yegor
764515ec78 temporarily disable broken driver test in integration_ui (#11440) 2017-07-28 14:30:10 -07:00
Yegor Jbanov
31d2ee9c0c tests for waitFor/waitForAbsent 2017-07-28 13:10:03 -07:00
Hans Muller
3045c289ea Fix sample page generator: add the commit arg (#11038) 2017-06-29 11:22:22 -07:00
Hans Muller
3d6e36d056 Updates Sample Catalog v0.0 (#11022) 2017-06-29 10:44:45 -07:00
Ian Hickson
dbe038ef6c Make this (working) test more verbose (#10973)
...so that we can compare it to the not-working test next to it.
2017-06-26 11:19:11 -07:00
Ian Hickson
9adb4a78a6 Deep linking: automatically push the route hiearchy on load. (#10894)
The main purpose of this PR is to make it so that when you set the
initial route and it's a hierarchical route (e.g. `/a/b/c`), it
implies multiple pushes, one for each step of the route (so in that
case, `/`, `/a`, `/a/b`, and `/a/b/c`, in that order). If any of those
routes don't exist, it falls back to '/'.

As part of doing that, I:

 * Changed the default for MaterialApp.initialRoute to honor the
   actual initial route.

 * Added a MaterialApp.onUnknownRoute for handling bad routes.

 * Added a feature to flutter_driver that allows the host test script
   and the device test app to communicate.

 * Added a test to make sure `flutter drive --route` works.
   (Hopefully that will also prove `flutter run --route` works, though
   this isn't testing the `flutter` tool's side of that. My main
   concern is over whether the engine side works.)

 * Fixed `flutter drive` to output the right target file name.

 * Changed how the stocks app represents its data, so that we can
   show a page for a stock before we know if it exists.

 * Made it possible to show a stock page that doesn't exist. It shows
   a progress indicator if we're loading the data, or else shows a
   message saying it doesn't exist.

 * Changed the pathing structure of routes in stocks to work more
   sanely.

 * Made search in the stocks app actually work (before it only worked
   if we happened to accidentally trigger a rebuild). Added a test.

 * Replaced some custom code in the stocks app with a BackButton.

 * Added a "color" feature to BackButton to support the stocks use case.

 * Spaced out the ErrorWidget text a bit more.

 * Added `RouteSettings.copyWith`, which I ended up not using.

 * Improved the error messages around routing.

While I was in some files I made a few formatting fixes, fixed some
code health issues, and also removed `flaky: true` from some devicelab
tests that have been stable for a while. Also added some documentation
here and there.
2017-06-23 14:58:29 -07:00
Michael Goderbauer
4b957bf028 Keep provisioning step (#10736) 2017-06-15 18:23:37 -07:00
Hans Muller
ede575a92e Refactor sample catalog screenshot production (#10676) 2017-06-14 09:05:53 -07:00
Michael Goderbauer
e2d4f9242e Benchmark for semantic overhead during transitions (#10678)
* Benchmark for semantic overhead during transitions

* review comments
2017-06-14 08:09:05 -07:00
Yegor
fde985b37d resurrect analyzer benchmarks (#10668)
* resurrect analyzer benchmarks

* move analyzer_benchmark to the more stable linux/android

* report average rather than best result
2017-06-13 13:14:16 -07:00
Hans Muller
1b29312ad2 Upload sample catalog screenshots to cloud storage (#10462) 2017-06-05 12:53:27 -07:00
yjbanov
b9d5327919 perf_tests.dart: const constructors; constructors first 2017-05-26 09:44:33 -07:00
yjbanov
f59a67707d track debug build times; switch from --profile to --release 2017-05-26 09:44:33 -07:00
Hans Muller
77c07871b4 Updated sample app production device labs task (#10326) 2017-05-25 13:49:31 -07:00
Hans Muller
b4e41ff7d3 Add a devicelab task that generates sample app screenshots, etc (#10303) 2017-05-25 11:47:36 -07:00
Chris Bracken
1dc3f88f57 Add -t flag to integration_ui keyboard resize test (#10012)
Update Android package name.
2017-05-11 15:19:44 -07:00
Chris Bracken
26a712cc12 Explicitly build ios keyboard_resize test (#10011)
Since lib/main.dart does not exist, update test to explicitly build lib/keyboard_resize.dart.
2017-05-11 14:55:35 -07:00
Chris Bracken
2d437f51c3 Add end-to-end test to verify resize on keyboard dismissal (#10005)
Adds an initial integration test that:
1. captures laid-out widget size
2. focuses a text field to show the keyboard and resize the view
3. unfocuses the text field to dismiss the keyboard and resize the view
4. verify that the final widget size matches the initial size
2017-05-11 14:29:20 -07:00
Chris Bracken
28feba043f Update Gallery plugin integration (#9823)
Enable plugin infra for both iOS and Android post-#9791.
2017-05-04 19:02:00 -07:00
Ian Hickson
2b04e300af Fix microbenchmarks tests. (#9782) 2017-05-03 17:33:29 -07:00
Chris Bracken
eacf793b35 Use default FlutterActivity in non-plugin examples (#9770)
Brings hello_world and flutter_gallery Android code in line with the
corresponding iOS code.
2017-05-03 14:20:17 -07:00
Chris Bracken
7f9d859e68 Pass package/activity in AndroidBackButtonMemoryTest (#9738)
Eliminates Gallery-specific hardcoding of app package and activity name
in the test.

Fixes breakage introduced with 1dc2d4eb43
due to main activity class now being MainActivity.
2017-05-02 15:44:03 -07:00
xster
1a4bf9b66d Fix flutter view project's automated tests (#9712)
Remove checked in .lock, remove checked in cocoapods scripts in xcode project, let the automated test run pod install specifically for this test
2017-05-01 18:23:06 -07:00
Chris Bracken
ffa8483f3b Update hello_world_memory_test package name (#9704)
Broken by 89af5ec8c7.
2017-05-01 14:18:00 -07:00
Mikkel Nygaard Ravn
70ff50f929 Integration test for channel communication (#9621) 2017-04-27 08:44:28 +02:00
Alexandre Ardhuin
35803c2297 comprehensive list of lints (#9330)
* comprehensive list of lints
* add comments to commented out lint
* fix unnecessary_this lints
* exclude prefer_final_fields
2017-04-20 09:38:57 +02:00
Alexandre Ardhuin
189028a958 Separate the 'if' expression from its statement (#9177) 2017-04-04 21:45:50 +02:00
xster
4245bce545 Add full platform build speed instrumentations (#9162)
* Add new metric to size_tests instead

* Small lints
2017-04-04 01:05:53 -07:00
Yegor
e067041279 increase vmservice timeout; log stderr in microbenchmarks (#9088) 2017-03-29 15:40:17 -07:00
Jason Simmons
f7d0aa03f8 Throw an exception if flutter run fails during a microbenchmark (#9061) 2017-03-28 15:46:53 -07:00
Yegor
0ec3ffb4bc increase total task timeout; decrease individual benchmark timeout (#9063) 2017-03-28 15:42:06 -07:00
Sarah Zakarias
d274888be6 Rename platform_services to platform_channel (#9018)
* Rename platform_services to platform_channel

* rename in README

* renamed tests

* update modules.xml
2017-03-27 11:17:31 +02:00
Yegor
bc8cc80b9f Xcode 8-compatible provisioning profiles in devicelab tests (#8987) 2017-03-23 17:35:04 -07:00
Sarah Zakarias
18db5a2e68 Add start-up test for flutter_view sample (#8892)
* Add start-up test for flutter_view sample

* add reportMetrics flag to startup test

* rewording in manifest.yaml
2017-03-23 13:20:00 +01:00
Sarah Zakarias
d1f73fd0ad Add integration test for the Platform Services app (#8826) 2017-03-20 22:19:03 +01:00
Ian Hickson
2bbe82ef30 Increase timeout and verbosity to diagnose devicelab failure (#8832) 2017-03-16 15:02:43 -07:00
Yegor
f8238185cc fix size Android tests broken by #8759 (#8796) 2017-03-15 10:08:22 -07:00
Chris Bracken
6c97dd2c6c Declare locals as final where not reassigned (dev) (#8574) 2017-03-03 18:06:08 -08:00
Alexandre Ardhuin
8c043d06de fix lints for directives_ordering rule (#8382) 2017-02-23 13:37:26 -08:00
Alexandre Ardhuin
e9a775bf38 prefer const constructors (#8316) 2017-02-21 14:54:29 -08:00
Jakob Andersen
eebc72a343 Update path to built .apk. (#8137) 2017-02-14 13:46:45 +00:00
Ian Hickson
21434fcf1a Refactor 'flutter drive' to get the observatory port from the logs (#7695)
This remove a very brittle aspect of flutter drive, whereby it would
assume a known port instead of explicitly finding out what it was.

Fixes #7692 and hopefully fixes the devicelab tests.
2017-01-27 01:37:28 -08:00
Ian Hickson
a4a1392c44 --debug-port is now --observatory-port (#7680) 2017-01-26 18:01:34 -08:00
Ian Hickson
aff4e82891 Technical Debt tracker (#7667) 2017-01-26 17:04:37 -08:00
Yegor
4bec9cd940 update build benchmark to match #7589 (#7600) 2017-01-23 17:49:01 -08:00
Jason Simmons
584dbe3d32 Fix the activity name in the Gallery back button memory metric (#7527) 2017-01-18 14:14:40 -08:00
Yegor
0c7a9da98f microbenchmarks: single-line JSON; consolidate #7433 workaround (#7450)
- output single-line JSON because iOS 9.x cannot output more than one line
- move the workaround for #7433 into LiveTestWidgetsFlutterBinding so that it works in all cases (e.g. animation_bench.dart on iOS)
2017-01-11 14:29:21 -08:00
Yegor
42ccffca36 fix microbenchmarks on iOS (#7429) 2017-01-10 18:16:00 -08:00
Yegor
b6ac86435a export missed transition count to the dashboard (#7426) 2017-01-10 13:09:43 -08:00
Yegor
cf61905c04 publish GPU metrics to dashboard; compact transition event printout (#7254) 2016-12-13 15:22:23 -08:00
Yegor
adf998cb2a remove mega_gallery__refresh_time superseded by hot_mode_dev_cycle__benchmark (#7171) 2016-12-06 12:20:54 -08:00
Jason Simmons
2fd786b567 A devicelab memory test for Android app suspend and resume (#7164) 2016-12-06 10:13:21 -08:00
Yegor
568a6e1e65 measure IPA size (#7158) 2016-12-05 12:46:59 -08:00
Jason Simmons
7da3ab634a Devicelab memory usage test that navigates repeatedly to a screen within Gallery (#7137) 2016-12-05 11:08:25 -08:00
Jason Simmons
2563636d90 A devicelab test of memory usage for a minimal app (#7126) 2016-12-02 10:27:43 -08:00
Yegor
45a4f639c9 devicelab: make specifying device OS less confusing (#7132) 2016-12-01 18:35:40 -08:00
Jason Simmons
4da4ca89aa Add a devicelab test that captures memory usage metrics on Android (#7120) 2016-12-01 11:37:16 -08:00
Yegor
80627a5ccd unlock device before running perf test (#6540) 2016-10-25 18:05:53 -07:00
Yegor
33c6526462 fix frame info extraction logic; track frames in gallery test (#6505) 2016-10-25 11:02:18 -07:00
Yegor
1eda886e7e unlock device before running transition test (#6371) 2016-10-18 10:20:05 -07:00
Yegor
8c8be6b5df remove engineEnterTimestampMicros from benchmark list; not useful (#6242) 2016-10-06 22:31:32 -07:00
Yegor
74804e6f1d fix analyzer_server__analysis_time benchmark broken in #6093 (#6205) 2016-10-04 13:27:41 -07:00
Yegor
6cdab85d28 Sync test code with Cocoon (#6129)
* fix flutter watch benchmark

Syncs d2d7950ecd

* Split Android/iOS impl behind a unified interface

Syncs db87e10fa5

* Switch from pub get to flutter packages get

Syncs b378005cbb

* "silent" option in test runner; fix analysis errors;
2016-09-29 10:19:15 -07:00
Yegor
1ba1562293 move CI tests into the main repo (#5758) 2016-09-14 13:22:53 -07:00