Commit Graph

301 Commits

Author SHA1 Message Date
Jenn Magder
e58dc16d7b
Revert "Add a FlutterDriver screenshot test to device lab. (#45411)" (#46281)
This reverts commit adec6977be.
2019-12-06 18:21:52 -08:00
Chris Yang
adec6977be
Add a FlutterDriver screenshot test to device lab. (#45411) 2019-12-06 14:17:02 -08:00
Alexandre Ardhuin
ec1a015045
implicit-casts:false in dev (#45787) 2019-12-05 22:34:06 +01:00
Jonah Williams
0b2bf99491
Reland: Update macOS configuration settings (#45966) 2019-12-03 08:13:08 -08:00
Ian Hickson
449f4a6673
License update (#45373)
* Update project.pbxproj files to say Flutter rather than Chromium

Also, the templates now have an empty organization so that we don't cause people to give their apps a Flutter copyright.

* Update the copyright notice checker to require a standard notice on all files

* Update copyrights on Dart files. (This was a mechanical commit.)

* Fix weird license headers on Dart files that deviate from our conventions; relicense Shrine.

Some were already marked "The Flutter Authors", not clear why. Their
dates have been normalized. Some were missing the blank line after the
license. Some were randomly different in trivial ways for no apparent
reason (e.g. missing the trailing period).

* Clean up the copyrights in non-Dart files. (Manual edits.)

Also, make sure templates don't have copyrights.

* Fix some more ORGANIZATIONNAMEs
2019-11-27 15:04:02 -08:00
sjindel-google
43a8a1902e
Fix erroneous comments referring to blobs snapshots. (#41378) 2019-11-21 12:09:43 +01:00
Jonah Williams
0e36a91d18
Add macOS hot reload test (#45264) 2019-11-20 12:35:53 -08:00
liyuqian
a32fc986f3
Add a perf test for picture raster cache (#45050)
This will catch issues like
https://github.com/flutter/flutter/issues/44252, and this test is
inspired by the `list_demo` sample app in
https://github.com/flutter/flutter/issues/43083#issue-509586473

This is tested locally on a Moto G4 before and after the fix
https://github.com/flutter/engine/pull/13710

The `average_frame_rasterizer_time_millis` of this test drops from
~5.4ms to ~4.9ms after that fix.
2019-11-19 15:48:41 -08:00
Jenn Magder
6e5769d425
Remove FLUTTER_DEVICELAB_XCODE_PROVISIONING_CONFIG code paths (#45136) 2019-11-19 11:19:44 -08:00
Ian Hickson
1ce4a4f36f
Make sure all our .dart files have license headers (#44467) 2019-11-08 16:53:21 -08:00
Jonah Williams
cc51ad5d08
Reland: enable usage of experimental web compiler (#44400) 2019-11-07 20:13:02 -08:00
Jonah Williams
7535ef735f
Revert "Enable usage of experimental incremental compiler for web (#43576)" (#44396)
This reverts commit e5d6d92498.
2019-11-07 16:32:20 -08:00
Jonah Williams
e5d6d92498
Enable usage of experimental incremental compiler for web (#43576) 2019-11-07 16:29:51 -08:00
xster
cf95cd4394
Add v1 plugin register function into v2 plugin template (#44166) 2019-11-05 20:03:50 -08:00
Jonah Williams
e778686e72
Re-enable chrome dev mode tests (#43691) 2019-10-29 13:23:27 -07:00
liyuqian
9cc29c61df
Reland "Measure iOS CPU/GPU percentage #41426" (#41578)
This reverts commit baea9bf7cc.

Additionally,   we let the test run on mac8 with iphonexs because the test won't run on Xcode 10.1 (mac3-7). Hence we force it to run on mac8 which currently has Xcode 10.2.
2019-09-29 11:33:43 -07:00
liyuqian
baea9bf7cc
Revert "Reland "Measure iOS CPU/GPU percentage (#41234)" (#41426)" (#41575)
This reverts commit 67b5bdf99b.

Reason: it broke device lab.

TBR: @goderbauer
2019-09-29 10:58:13 -07:00
liyuqian
67b5bdf99b
Reland "Measure iOS CPU/GPU percentage (#41234)" (#41426)
This reverts commit f1e7fe8142.

This fix is in https://github.com/flutter/packages/pull/39
2019-09-28 14:24:46 -07:00
Jonah Williams
f1e7fe8142
Revert "Reland "Measure iOS CPU/GPU percentage (#39439)" (#41234)" (#41400) 2019-09-26 12:27:01 -07:00
liyuqian
406b44983a
Reland "Measure iOS CPU/GPU percentage (#39439)" (#41234)
This reverts commit 652be88ecd.

This fix is in https://github.com/flutter/packages/pull/37
2019-09-26 10:39:24 -07:00
Jonah Williams
39eecd5226
give benchmark output dill path (#40857) 2019-09-24 19:10:34 -04:00
Alexandre Ardhuin
a36f809e27
add missing trailing commas (in dev/) (#40704) 2019-09-19 07:54:46 +02:00
Alexandre Ardhuin
df4bf453ab
more UI-as-code (#35516) 2019-09-17 16:23:44 +02:00
liyuqian
652be88ecd
Revert "Measure iOS CPU/GPU percentage (#39439)" (#40624)
This reverts commit af9f424d5d.

Reverts flutter/flutter#39439

Reason: this passed the local device lab test on my Macbook but it failed in the actual device lab. Will investigate, fix, and reland.

TBR: @dnfield @goderbauer @tvolkert @Hixie
2019-09-16 15:00:27 -07:00
liyuqian
af9f424d5d
Measure iOS CPU/GPU percentage (#39439)
For https://github.com/flutter/flutter/issues/33899

Test added:
- simple_animation_perf_ios

Test modified:
- backdrop_filter_perf_ios__timeline_summary

We'll add the CPU/GPU measurement to more iOS tests
once it's proven to be non-flaky.
2019-09-16 13:14:38 -07:00
Phil Quitslund
37846b032f
migrate from slow async io calls (#39515)
* migrate from slow async io calls
2019-08-29 16:47:10 -07:00
Jonah Williams
d2f70a6d20
kill leaked chrome processes (#39514) 2019-08-29 14:28:37 -07:00
Collin Jackson
a1c185f37a
Fix plugin template app's tests (#39080)
* Fix plugin template

* Add test of plugin template working
2019-08-26 14:57:16 -07:00
Christopher Fujino
65f8f9c0b8
add --exit and --match-host-platform defaults to devicelab runner (#37832) 2019-08-20 14:37:42 -07:00
liyuqian
e77237d6d3
Reland "Fix the first frame logic in tracing and driver (#35297)" (#37192)
This relands https://github.com/flutter/flutter/pull/35297

The followings have been done to fix the broken tests:
  1. Add `didSendFirstFrameRasterizedEvent` extension and its tests
  2. Wait for `didSendFirstFrameRasterizedEvent` instead of
     `didSendFirstFrameEvent` during start up tests
  3. Mark missed (probably newly added) start up tests as flaky
2019-07-31 11:01:52 -07:00
Jonah Williams
3068fc4f7c
Revert "Fix the first frame logic in tracing and driver (#35297)" (#37027)
This reverts commit 68fc7231b3.
2019-07-26 13:08:36 -07:00
Jason Simmons
a3a350df1c
devicelab: replace the FLUTTER_ENGINE environment variable with the new local engine flags (#36969) 2019-07-26 10:06:56 -07:00
liyuqian
68fc7231b3
Fix the first frame logic in tracing and driver (#35297)
This should fix https://github.com/flutter/flutter/issues/31442 and https://github.com/flutter/flutter/issues/34867
2019-07-26 09:38:12 -07:00
Matt Carroll
a7387b127d
Added demo projects for splash screen support on Android. (#35728) 2019-07-24 13:44:13 -07:00
Jonah Williams
c4cab5dc92
add missing config to create (#36727) 2019-07-22 19:48:55 -07:00
Jonah Williams
819538704a
fix devicelab tests (#36717) 2019-07-22 17:08:42 -07:00
Todd Volkert
0f625196cc
Add plumbing for hello world startup test in devicelab (#36410)
This is the first step in getting esoteric devices wired
up to device lab to ensure that we can at least start Flutter
apps in profile mode on such devices.

https://github.com/flutter/flutter/issues/35838
2019-07-18 11:13:14 -07:00
Jonah Williams
f711308376
Add sync star benchmark cases (#36303) 2019-07-17 16:19:49 -07:00
Jim Graham
6372250995
add the transformPoint and transformRect benchmarks (#36026) 2019-07-17 13:39:13 -07:00
Jenn Magder
b5c1b61c73
Add platform_interaction_test_swift to devicelab (#35775) 2019-07-10 14:13:01 -07:00
Jonah Williams
91e3b2d671
add iOS build benchmarks (#35749) 2019-07-08 18:09:00 -07:00
Alexander Aprelev
9f285aa9d9
Introduce image_list performance benchmark that runs on jit(debug) build. (#35424)
* Introduce image_list performance benchmark that runs on jit(debug) build.

We want to track performance of debug build because ongoing work on reducing the cost of spanwing new isolate will impact jit(debug) more than it will aot(release) configurations.

* Newlines and spaces
2019-07-08 08:23:06 -07:00
Jonah Williams
05b4c67550
add APK build time benchmarks (#35481) 2019-07-03 10:54:04 -07:00
Alexandre Ardhuin
758009ba70
more ui-as-code (#35393)
* more ui-as-code

* address review comments
2019-07-02 21:11:56 +02:00
Alexander Aprelev
3badcf51a4
Introduce image loading performance test. (#34823)
* Introduce image loading performance test.

This example reports how long it takes to load images over the network(locally spawned server).
2019-06-28 15:14:03 -07:00
Tong Mu
03ed06f44d
New benchmark: Gesture semantics (#35232)
* Add semanticsEnabled to widgetBenchmark
* Add button_matrix_app and gesture benchmark
2019-06-28 00:37:01 -07:00
Alexandre Ardhuin
919dcf53f3
enable lints prefer_spread_collections and prefer_inlined_adds (#35189) 2019-06-27 21:23:16 +02:00
Jim Graham
f83fd9d467
Add test case for Flutter Issue #27677 as a benchmark. (#34870)
* Add test case for Flutter Issue #27677 as a benchmark.

See https://github.com/flutter/flutter/issues/27677

I got the following results running the test on a Moto E2 which
will help us determine how much we can gain by analyzing the
operations and eliminating unnecessary repaints based on
dirty rectangles/regions.

no blurs       - avg 216.0 fps over 3 samples
blur the group - avg  22.9 fps over 3 samples
blur each txt  - avg   3.4 fps over 3 samples

* Added the new benchmark to be tracked on the dashboard.
2019-06-24 20:53:22 -07:00
Jonah Williams
25b5656969
Reland: rename web device (#34885) 2019-06-24 11:09:07 -07:00
Todd Volkert
fb2f18e7e8
Prepare for Uint8List SDK breaking changes (#34295)
https://github.com/dart-lang/sdk/issues/36900
2019-06-12 10:22:50 -07:00
Emmanuel Garcia
8627ff433b
Allow multi-abi shared libraries in APKs and App bundles (#34123)
* Gradle generates ELF shared libraries instead of AOT snapshots.
* `flutter build apk/appbundle` supports multiple `--target-platform` and defaults to `android-arm` and `android-arm64`.
* `flutter build apk` now has a flag called `--split-per-abi`.
2019-06-10 13:16:09 -07:00
Jonah Williams
4c0145d97b
Fix web size test for new world (#34063) 2019-06-07 13:52:04 -07:00
Jonah Williams
227ffb58d7
fix devicelab test for chrome reload worklow (#34056) 2019-06-07 12:44:17 -07:00
Jonah Williams
50ea6f32d3
add benchmarks to track web size (#33892) 2019-06-05 12:40:21 -07:00
Jonah Williams
83986acbdb
Add a real-er web restart, doctor, workflow (#33786) 2019-06-03 23:19:42 -07:00
Chris Bracken
98ea501995
Revert "Add real-er restart for web using webkit inspection protocol (#33629)" (#33703)
Revert "fix devicelab manfiest (#33698)"

This reverts commit 5a6a00dc54.
This reverts commit 0d79f0fc79.
2019-05-31 21:53:30 -07:00
Jonah Williams
5a6a00dc54
Add real-er restart for web using webkit inspection protocol (#33629) 2019-05-31 17:37:01 -07:00
Jason Simmons
311cde98a6
Reland "Clean up some flutter_tools tests and roll dependencies" (#33225)
This relands flutter/flutter#33163 with a fix to the integration_ui test.
2019-05-29 19:04:35 -07:00
Jonah Williams
925f5f1c53
Revert "Wire up hot restart and incremental rebuilds for web (#33197)" (#33529)
This reverts commit 52ae99682e.
2019-05-29 12:09:01 -07:00
Jonah Williams
52ae99682e
Wire up hot restart and incremental rebuilds for web (#33197) 2019-05-29 09:49:16 -07:00
Chris Bracken
156b4220b4
Americanise spellings (#33323)
Updates documentation and non-public API to use American spellings for
consistency with the rest of the codebase.

No changes to behaviour... other than how it's spelt.
2019-05-24 19:13:02 -07:00
Chris Bracken
4d9923201b
Correct typos (#33322)
Corects a bnuch of typeos throuhgout teh Fluter codebsae.

Made use of the `misspell` tool:
https://github.com/client9/misspell
2019-05-24 19:12:45 -07:00
Jonah Williams
a30ffb60ad
Revert "Clean up some flutter_tools tests and roll dependencies (#33163)" (#33206) 2019-05-22 12:20:02 -07:00
Ian Hickson
e5f81e1048
Clean up some flutter_tools tests and roll dependencies (#33163)
* Clean up some flutter_tools tests

* Remove arbitrary retry that happens even for fundamental errors, and generally clean up _DevFSHttpWriter.

* Update dependencies (requires fixes; see next commit)

* Fixes for new dependencies.
2019-05-22 09:31:37 -07:00
Dan Field
99bd2795b0 Add rrect contains microbenchmark (#30985) 2019-04-15 09:01:40 -07:00
Tim Sneath
529189791c
Replace flutter.io with flutter.dev (#30562) 2019-04-05 11:39:30 -07:00
Jonah Williams
1bfa2f2311
Intercept errors thrown by synchronous Completers in image resolution. (#30139) 2019-03-29 10:05:18 -07:00
Dan Field
ecfdd7e1ea
Detect and cleanup leaky processes (#29196)
* Detect and cleanup leaky processes
* Add flaky tests for detecting leaked processes
2019-03-22 14:32:36 -07:00
Alexandre Ardhuin
387f885481
Add missing trailing commas (#28673)
* add trailing commas on list/map/parameters

* add trailing commas on Invocation with nb of arg>1

* add commas for widget containing widgets

* add trailing commas if instantiation contains trailing comma

* revert bad change
2019-03-01 08:17:55 +01:00
liyuqian
19a6a6f431
Reland "Shader warm up (#27660)" (#28537)
This reverts commit adc8e159a5.

This should be safe to land once https://github.com/flutter/flutter/pull/28530 gets merged

Merge on yellow doc test because the doc test is actually green.
2019-02-27 09:33:08 -08:00
liyuqian
adc8e159a5
Revert "Shader warm up (#27660)" (#28376)
This reverts commit a44f174efc.

Reason: start_up tests become flaky.

See https://github.com/flutter/flutter/issues/28374

TBR: xster

Merge on red to fix the tree
2019-02-22 17:32:37 -08:00
liyuqian
a44f174efc
Shader warm up (#27660)
This patch adds a default shader warm up process which moves shader compilation from the animation time to the startup time. This also provides an extension for `runApp` so developers can customize the warm up process.

This should reduce our worst_frame_rasterizer_time_millis from ~100ms to ~20-30ms for both flutter_gallery and complex_layout benchmarks. Besides, this should also have a significant improvement on 90th and 99th percentile time (50%-100% speedup in some cases, but I haven't tested them thoroughly; I'll let our device lab collect the data afterwards).

The tradeoff the is the startup time (time to first frame). Our `flutter run --profile --trace-startup` seems to be a little noisy and I see about 100ms-200ms increase in that measurement for complex_layout and flutter_gallery. Note that this only happens on the first run after install or data wipe. Later the Skia persistent cache will remove the overhead.

This also adds a cubic_bezier benchmark to test the custom shader warm up process.

This should fix https://github.com/flutter/flutter/issues/813 (either by `defaultShaderWarmUp`, or a `customShaderWarmUp`).
2019-02-22 15:37:02 -08:00
Dan Field
04b7a5b366
Reduce memory benchmark perf tests outer loop to 10. (#28281)
* reduce iterations to avoid test timing out
2019-02-21 14:46:24 -08:00
Jonah Williams
67cf21577f
Add basic codegen app to be used for integration testing and benchmarks (#27257) 2019-02-14 23:17:16 -08:00
liyuqian
de321b9cc1
Add scroll performance test for flutter_gallery (#27513)
Previously, we had a scroll_perf drive test but we were not exercising
it on our device lab. This helps us tracking issues like:
https://github.com/flutter/flutter/issues/24782
2019-02-04 22:54:11 -08:00
Jason Simmons
b5ca595d61
Remove icudtl.dat from APK asset size checks (#27505) 2019-02-04 12:40:43 -08:00
Michael Goderbauer
6d20ff2446
Remove all obsolete "// ignore:" (#27271) 2019-01-30 08:56:12 -08:00
liyuqian
8ad4cbe06e
Add cull opacity perf test to device lab (#25381)
For https://github.com/flutter/flutter/issues/24712

This test verifies that https://github.com/flutter/engine/pull/6923 will speedup the average rasterize time of this test from ~150ms to ~10ms

Please see non-auto-generated files in 37b21d9fb4
2018-12-17 22:01:07 -08:00
Jonah Williams
938dd5a4aa
Revert "Update driver script to execute test through test_core" (#24401)
* Revert "Add dashing config file for generating docset from flutter docs (#24374)"

This reverts commit ec8ca8606c.

* Revert "Update driver script to execute test through test_core (#24168)"

This reverts commit 6c62cf337f.
2018-11-15 11:17:00 -08:00
Jonah Williams
6c62cf337f
Update driver script to execute test through test_core (#24168) 2018-11-15 10:50:25 -08:00
Michael Goderbauer
b8c29a1490
Add benchmark for LICENSE file size (#23948) 2018-11-05 13:57:40 -08:00
Michael Klimushyn
43c99404c5
Test flutter create --offline (#23278)
Verifies that flutter tools isn't rebuilt. Doesn't truly check that the
created output is runnable.
2018-10-22 16:39:28 -07:00
Greg Spencer
abb1758edf
Fix places in devicelab tests where it expects the app template. (#22696)
I missed some create template conversions in the devicelab directory.
2018-10-04 15:30:08 -07:00
Greg Spencer
3246a66374
Make devicelab tests use 'app' template (#22694)
Fixes the build after my module rename broke the Windows tests.
2018-10-04 14:30:59 -07:00
Alexandre Ardhuin
d340e2f229
apply lint prefer_void_to_null in dev/ (#22661) 2018-10-04 18:44:23 +02:00
Alexandre Ardhuin
2ea1d81cdc
sort_constructors_first (#22575) 2018-10-04 07:28:07 +02:00
Alexandre Ardhuin
f62afdcf57
add missing type parameter on methods (#22096) 2018-10-01 21:29:08 +02:00
liyuqian
872eba2d4c
Track 90th, 99th percentile instead of missed_frame_build_budget_count (#21754)
See https://github.com/flutter/flutter/pull/19121#issuecomment-419520765
2018-09-14 13:24:13 -07:00
Alexandre Ardhuin
d927c93310
Unnecessary new (#20138)
* enable lint unnecessary_new

* fix tests

* fix tests

* fix tests
2018-09-12 08:29:29 +02:00
Jonah Williams
c6e8a513f0
Add android semantics integration test to device lab (#20971) 2018-09-06 12:31:18 -07:00
Chris Bracken
9f9676c4c6
Eliminate Dart 1 support in devicelab tests (#21240)
Flutter commands all default to --preview-dart-2. Eliminate support for
the use of this option and the previewDart2 parameter in dev/devicelab
tests.
2018-08-30 17:17:37 -07:00
Ian Hickson
53b635817c
Re-enable missing_return (#20844) 2018-08-28 11:22:57 -07:00
amirh
1f31c3b354
Run the embedded Android views integration test on the device lab. (#20841) 2018-08-23 10:26:00 -07:00
Ian Hickson
1993a67381
Performance test cleanup (#20652)
* Fix TODO syntax.
* Clarify messages for some timeouts, to aid debugging.
* Increase some other timeouts that were a needlessly short, to reduce sources of flakes.
* Remove some more timeouts that were mostly redundant, to remove complexity.
* Minor style cleanup.
* Remove some dangerous traps (specifically, hide the explicit start/end times in TimedEvent since they shouldn't matter).
2018-08-17 13:19:28 -07:00
Ian Hickson
3dec6a6930
Clean up usage of temporary directories (#20682)
All temporary directory start with `flutter_` and have their random component separated from the name by a period, as in `flutter_test_bundle.YFYQMY`.

I've tried to find some of the places where we didn't cleanly delete temporary directories, too. This greatly reduces, though it does not entirely eliminate, the directories we leave behind when running tests, especially `flutter_tools` tests.

While I was at it I standardized on `tempDir` as the variable name for temporary directories, since it was the most common, removing occurrences of `temp` and `tmp`, among others.

Also I factored out some common code that used to catch exceptions that happen on Windows, and made more places use that pattern.
2018-08-17 13:17:23 -07:00
Ian Hickson
9e9ac1a16c
Refactor analysis benchmark and collect more data (#20282)
* Refactor analysis benchmark and collect more data (#20169)

This does some code cleanup to avoid duplication, improves the logs,
and records min and max values as well as the mean.

* Fix typo that led to infinite loop
2018-08-06 18:15:44 -07:00
Ian Hickson
b81e361192
Revert "Refactor analysis benchmark and collect more data" (#20280)
* Revert "increase size of user account drawer headers to 48 by 48 (#20266)"

This reverts commit 4a7b4a4dde.

* Revert "EditableText Cursor can be set to not blink for testing (#20004)"

This reverts commit d041b319e8.

* Revert "Refactor analysis benchmark and collect more data (#20169)"

This reverts commit 5ea0a13598.
2018-08-06 15:34:08 -07:00
Ian Hickson
5ea0a13598
Refactor analysis benchmark and collect more data (#20169)
This does some code cleanup to avoid duplication, improves the logs,
and records min and max values as well as the mean.
2018-08-06 12:47:09 -07:00
Ian Hickson
872e88cec7
Reduce noise on the memory benchmarks (#19630) (#20163)
- Check memory usage in release builds, not profile.
- Use multiple runs and average the results.
2018-08-06 12:46:51 -07:00
Devon Carew
3ead9c0f0c
reduce the number of runs on the analyze benchmakr (#20155) 2018-08-02 16:13:05 -07:00