Commit Graph

46 Commits

Author SHA1 Message Date
Ian Hickson
11119cff38
More helpful test.dart output (#140175)
Each error section is numbered, so you can all be sure you're talking about the same one.
A message is printed at the very end telling you how to find the error blocks in the verbose logs.
2023-12-28 23:16:12 +00:00
godofredoc
fe7299d318
Migrate customer_testing to sharded tests. (#138659)
This will allow Dart Team to run customer tests as part of monorepo and will be a step forward to remove ad_hoc tests.

Bug: https://github.com/dart-lang/sdk/issues/51042
Bug: https://github.com/flutter/flutter/issues/115476
2023-11-28 00:41:53 +00:00
Michael Goderbauer
fda9ecfef7
Remove 1745 decorative breaks (#123259)
Remove 1745 decorative breaks
2023-03-22 21:12:22 +00:00
Ian Hickson
4c99da6c56
Avoid printing blank lines between "Another exception was thrown:" messages. (#119587) 2023-02-01 23:57:54 +00:00
Ian Hickson
97901da149
Cleaner test.dart output. (#109206) 2022-08-14 08:53:54 +00:00
Ian Hickson
03aa059c3c
Do not abort at first error when tests fail. (#108936) 2022-08-05 16:18:54 -07:00
Greg Spencer
738ce43d97
Randomize tests, exclude tests that fail with randomization. (#86793)
* Randomize tests, exclude tests that fail with randomization.

* Disable some more tool tests
2021-07-26 23:40:49 -07:00
Michael Goderbauer
045ba2b661
Migrate dev/bots to null safety (#86522) 2021-07-19 13:59:15 -07:00
Ian Hickson
d3c17acf9d
Run "pub global activate devtools" before overall_experience_test (#85812) 2021-07-12 14:36:05 -07:00
Greg Spencer
6adea05276
Revert "Randomize Framework tests, opt out some tests that currently fail. (#85159)" (#85673)
This reverts commit b5f9612 because it is taking about 40% longer to run the tests, which is causing timeouts.
2021-06-30 19:38:09 -07:00
Greg Spencer
b5f9612cac
Randomize Framework tests, opt out some tests that currently fail. (#85159)
This turns on order shuffling for all tests that don't fail with it on, marking those tests that do fail with a tag so that they will be run without shuffling on.

To determine which tests fail with it on, I ran all the tests 100 times with different random shuffle seeds, and then also ran it with the date seeds from today until the end of July, and tagged all of the test suites (files) that fail, with a seed that caused them to fail.
2021-06-29 13:46:13 -07:00
Yegor
2fa03438de
add web_long_running_tests shard containing long-running web tests (#67324) 2020-10-29 14:23:02 -07:00
Yegor
ff9082cf6d
Refactor command utilities for tests (#68324)
* refactor command running utilities for tests

Co-authored-by: Jonah Williams <jonahwilliams@google.com>
2020-10-19 20:56:25 -07:00
Michael R Fairhurst
7f3c9b6bda
Remove unused 'dart:async' imports. (#65568) 2020-09-16 14:14:06 -07:00
Dan Field
22239f4183
Test engine version hash, but skip for Dart HHH bot (#55223) 2020-05-04 11:19:02 -07:00
Dan Field
6a490ed497
Revert "validate engine hash (#55057)" (#55220)
This reverts commit 98651c0252.
2020-04-20 10:56:39 -07:00
Dan Field
98651c0252
validate engine hash (#55057) 2020-04-17 16:30:26 -07:00
Yegor
b34046903b
Fix stack trace parsing on non-debug builds; add e2e tests (#50652)
* Fix stack trace parsing on non-debug builds; add e2e tests
2020-02-13 18:34:08 -08:00
Alexandre Ardhuin
1c79347ef6
use yield* (#48307) 2020-01-08 07:34:36 +01:00
Alexandre Ardhuin
4f9b6cf017
enable lint prefer_final_in_for_each (#47724) 2020-01-07 16:32:04 +01:00
Dan Field
24f39d40a4 Revert "Revert "Add many more global analyses. (#47875)" (#48080)" (#48081)
This reverts commit 04ea3183ce.
2020-01-02 11:47:28 -08:00
Francisco Magdaleno
04ea3183ce
Revert "Add many more global analyses. (#47875)" (#48080)
This reverts commit e768c92fbc.
2020-01-02 09:25:59 -08:00
Ian Hickson
e768c92fbc
Add many more global analyses. (#47875)
* Update packages.

* Add many more global analyses.

* Catch trailing spaces and trailing newlines in all text files.
  Before we were only checking newly added files, but that means we
  missed some.

* Port the trailing spaces logic to work on Windows too.

* Correct all the files with trailing spaces and newlines.

* Refactor some of the dev/bots logic into a utils.dart library.
  Notably, the "exit" and "print" shims for testing are now usable
  from test.dart, analyze.dart, and run_command.dart.

* Add an "exitWithError" function that prints the red lines and
  then exits. This is the preferred way to exit from test.dart,
  analyze.dart, and run_command.dart.

* More consistency in the output of analyze.dart.

* Refactor analyze.dart to use the _allFiles file enumerating logic
  more widely.

* Add some double-checking logic to the _allFiles logic to catch
  cases where changes to that logic end up catching fewer files
  than expected (helps prevent future false positives).

* Add a check to prevent new binary files from being added to
  the repository. Grandfather in the binaries that we've already
  added.

* Update all the dependencies (needed because we now import crypto in
  dev/bots/analyze.dart).
2019-12-30 17:12:19 -08:00
Ian Hickson
449f4a6673
License update (#45373)
* Update project.pbxproj files to say Flutter rather than Chromium

Also, the templates now have an empty organization so that we don't cause people to give their apps a Flutter copyright.

* Update the copyright notice checker to require a standard notice on all files

* Update copyrights on Dart files. (This was a mechanical commit.)

* Fix weird license headers on Dart files that deviate from our conventions; relicense Shrine.

Some were already marked "The Flutter Authors", not clear why. Their
dates have been normalized. Some were missing the blank line after the
license. Some were randomly different in trivial ways for no apparent
reason (e.g. missing the trailing period).

* Clean up the copyrights in non-Dart files. (Manual edits.)

Also, make sure templates don't have copyrights.

* Fix some more ORGANIZATIONNAMEs
2019-11-27 15:04:02 -08:00
Ian Hickson
124dc6617f
Clean up test infrastructure (#43030)
See #41880 for history.
2019-10-18 16:35:39 -07:00
Jonah Williams
1982a5c3ec
Revert "Clean up test infrastructure (#41880)" (#42982)
This reverts commit 1781d5c9bb.
2019-10-17 19:47:16 -07:00
Ian Hickson
1781d5c9bb
Clean up test infrastructure (#41880) 2019-10-17 19:22:47 -07:00
Ian Hickson
e0a31de61b
Clean up bots output (#36864) 2019-08-20 14:53:39 -07:00
James Lin
9823b3dbd4 Update with more review feedback from jonahwilliams
* Remove the identity() function.
* Make _flattenMap private.
* Don't bother with ungrowable lists.
2019-08-15 16:47:20 -07:00
James Lin
4f3b70671c Add assert messages 2019-08-15 16:47:20 -07:00
James Lin
e3ffa7689a Update with review feedback from dnfield and jonahwilliams 2019-08-15 16:47:20 -07:00
James Lin
c02b805cdc Give _runFlutterTest the ability to validate command output
In another change (#37646), I want to test that a test fails and
prints expected output.  I didn't see an existing way to do that, so
I modified `_runFlutterTest` and `runCommand` to allow capturing the
output.  Currently capturing and printing output are mutually
exclusive since we don't need both.

Some awkward bits:
* There already exists a `runAndGetStdout` function that is very
  similar to `runCommand`, and this change makes the conceptual
  distinction more confusing.

* `runFlutterTest` has multiple code paths for different
  configurations.  I don't understand what the different paths are
  for, and I added output checking only along one of them.
2019-08-15 16:47:20 -07:00
Jonah Williams
fb9ff9296e
fix linesplitter (#36006) 2019-07-11 16:52:14 -07:00
Jonah Williams
2b20345bb8
use pub run for create test and remove [INFO] logs (#35839) 2019-07-10 08:48:01 -07:00
Jonah Williams
c1a2e44c78
Allow flaky tests to pass or fail and mark web tests as flaky (#34456) 2019-06-13 17:52:04 -07:00
Dan Field
dcc965a38e
Revert "Speculative fix for #29262 (#29273)" (#29319)
This reverts commit 53d45258dc.
2019-03-13 12:58:10 -07:00
Dan Field
53d45258dc
Speculative fix for #29262 (#29273)
* Avoid writing to stderr before process has finished streaming
2019-03-13 12:27:05 -07:00
Dan Field
b9f013c044
Make sure test reporter prints out stderr, and disables Bigquery for non-contributors (#29073)
* print stderr to stderr, no bq if not contributor

* let test continue of bigquery fails
2019-03-10 11:26:17 -07:00
Dan Field
f67a529e6d
Fix coverage shard and print summary after test run (#28970)
* Fix coverage shard

* clarify

* Make sure we print test results

* Actually report test results before exiting

* revert unintended changes
2019-03-07 11:31:35 -08:00
Dan Field
20e0f13cc9
Test reporter (#28297)
* Wrap test.main with a custom processor
* Report test results to bigquery table
2019-03-06 13:13:45 -08:00
Alexandre Ardhuin
387f885481
Add missing trailing commas (#28673)
* add trailing commas on list/map/parameters

* add trailing commas on Invocation with nb of arg>1

* add commas for widget containing widgets

* add trailing commas if instantiation contains trailing comma

* revert bad change
2019-03-01 08:17:55 +01:00
Alexandre Ardhuin
8b0de38ef8
fix upcoming lint avoid_returning_null_for_void (#23190) 2018-10-17 11:01:37 +02:00
Alexandre Ardhuin
d340e2f229
apply lint prefer_void_to_null in dev/ (#22661) 2018-10-04 18:44:23 +02:00
Alexandre Ardhuin
f62afdcf57
add missing type parameter on methods (#22096) 2018-10-01 21:29:08 +02:00
Alexandre Ardhuin
d927c93310
Unnecessary new (#20138)
* enable lint unnecessary_new

* fix tests

* fix tests

* fix tests
2018-09-12 08:29:29 +02:00
Alexander Aprelev
391e91c996
Split analysis steps out of dev/bots/test.dart into dev/bots/analyze.dart (#21174)
* Split analysis steps out of dev/bots/test.dart into dev/bots/analyze.dart.

This allows to run analysis step with command line arguments that are only applicable to flutter analyze(like --dart-sdk, needed for dart-flutter-engine head-head-head bot).

* Add forgotten dev/bots/analyze.dart

* Refactor common code from analyze.dart and test.dart into run_command.dart

* Remove comments, add header
2018-08-30 07:30:25 -07:00