* Make it possible to override the FLUTTER_TEST env variable without unsetting it.
* Switch to using platform instead of Platform.
* Document the bindings, and introduce tests that initialize multiple WidgetsBindings with different environments.
* Add tests for the flutter platform test.
* Add license headers
* Fix lints
* Remove trailing whitespace
* Respond to Jonahs comments
* Respond to Ians comments
* Mock out the HttpServer in flutter_platform_test
* Mock out the HttpServer in flutter_platform_test
* Explain why we mock out the HttpServer in flutter_platform_test
In runTests, we previously now default the optional boolean `web`
parameter to false to ensure (or at least improve the odds) that the
conditional on line 70 evaluates to true or false.
* Added support for authentication codes for the VM service.
Previously, a valid web socket connection would use the following URI:
`ws://127.0.0.1/ws`
Now, by default, the VM service requires a connection to be made with a
URI similar to the following:
`ws://127.0.0.1:8181/Ug_U0QVsqFs=/ws`
where `Ug_U0QVsqFs` is an authentication code generated and shared by
the
service.
This behavior can be disabled with the `--disable-service-auth-codes`
flag.
* Use source list from the compiler to track invalidated files.
* Revert accidental change
* Fix first-time-seen-the-file logic
* Fix/simplify invalidate logic now that we can rely on compiler to let us know what is the cut-off point for invalidation.
* Update devfs mock to accommodate for new fields
* Fix deleted files case
* Analyzer found missing final
* some space formattings
* always use blocks in if-else if a block is used
* format spaces in for and while
* allow multiline if conditions
* fix missing space
* add trailing commas on list/map/parameters
* add trailing commas on Invocation with nb of arg>1
* add commas for widget containing widgets
* add trailing commas if instantiation contains trailing comma
* revert bad change
These are essentially self-inflicted race conditions. Instead of timeouts we're going to try a more verbose logging mechanism that points out when things are taking a long time.
* Remove many timeouts.
These are essentially self-inflicted race conditions. Instead of timeouts we're going to try a more verbose logging mechanism that points out when things are taking a long time.
* Get the attach tests to pass.
* Apply review comments from Todd
* More review comment fixes
* Put back the extended timeouts here now that I know why we have them...
* Revert "Add dashing config file for generating docset from flutter docs (#24374)"
This reverts commit ec8ca8606c.
* Revert "Update driver script to execute test through test_core (#24168)"
This reverts commit 6c62cf337f.
Ensure that cached dill files for builds with --track-widget-creation
always have .track. in the file name to avoid mixing transformed and
untransformed kernel files.
* Improve documentation and clean up code.
* Remove "Note that".
The phrase "note that" is basically meaningless as a prefix to an
otherwise fine sentence.
This changes the compiler output for gradle to be less verbose and more easily read.
This only applies to compilation error messages: other gradle messages will continue to print as before.
It also fixes a small problem with the performance measurement printing (see that "7.1s" on it's own line in the original?) so that if something is expected to have multiple lines of output, it prints an initial line, and a "Done" line with the elapsed time, so that it's possible to know what the time applies to.
It also updates the spinner to be fancier, at least on platforms other than Windows (which is missing a lot of symbols in its console font).
Addresses #17307
This adds support to AnsiTerminal for colored output, and makes all tool output written to stderr (with the printError function) colored red.
No color codes are sent if the terminal doesn't support color (or isn't a terminal).
Also makes "progress" output print the elapsed time when not connected to a terminal, so that redirected output and terminal output match (redirected output doesn't print the spinner, however).
Addresses #17307
This patch eliminates the --preview-dart-2/--no-preview-dart-2 flag,
hardcoding all uses to true. It also defaults all previewDart2 method
parameters to true, where they hadn't yet been.
A series of subsequent patches will eliminate all previewDart2
parameters and the associated code from within the codebase.
* Fix coverage collection crash
Based on Jason's patch in https://github.com/flutter/flutter/pull/19546/
This is more or less the same but I tried to avoid using `dynamic`.
* Improve argument and variable names in flutter_platform
* Don't bother with reduce, since the order is guaranteed.
All temporary directory start with `flutter_` and have their random component separated from the name by a period, as in `flutter_test_bundle.YFYQMY`.
I've tried to find some of the places where we didn't cleanly delete temporary directories, too. This greatly reduces, though it does not entirely eliminate, the directories we leave behind when running tests, especially `flutter_tools` tests.
While I was at it I standardized on `tempDir` as the variable name for temporary directories, since it was the most common, removing occurrences of `temp` and `tmp`, among others.
Also I factored out some common code that used to catch exceptions that happen on Windows, and made more places use that pattern.
* Try to resolve an intermitted crash during coverage collection
The only theory I can come up with is that maybe the test completes
before we finish processing the standard input, so I made the test
harness wait for the observatory URL before considering whether the
test has finished or not.
Also, some code cleanup while I'm at it, e.g. avoiding using "onFoo"
for the names of methods, avoiding back-to-back switch statements with
the same values, avoiding `_` argument names, and using `?.` instead
of `if (foo != null) foo.`.
* Revert back the signature of _pipeStandardStreamsToConsole
* Also remove the other additions to this method.
* Reland "Roll engine to version b148e628ec86b3a9a0382e0bcfae73f0390a8232 (#20427)"
This is a re-land with downgraded `package:flutter_gallery_assets`
version.
* Downgrade package:flutter_gallery_assets to 0.1.4
* Change engine.version to 81baff97c29bb08cbf8453a3f9042c5813f84ad3 (which contains an additional fix)
* Change engine.version to e3687f70c7ece72000b32ee1b3c02755ba5361ac (since mac tarballs are corrupted on earlier commit)
Reason for revert: The package:flutter_gallery_assets has removed some images which are required for the examples/flutter_gallery, so the gallery build is failing (only discovered after landing, since gallery doesn't seem to get built during github PR presubmit checks)
This CL
* rolls `engine.version` to flutter/engine@b148e628 (which includes dart sdk 2.1.0-dev)
* rolls `goldens.version` to flutter/goldens@6c45fafdf (which includes updates due to skia changes in engine)
* changes `platform.dill` to `platform_strong.dill` in various places due to flutter/engine@a84b210b
* adds explicit `environment: sdk: ">=2.0.0-dev.68 < 3.0.0"` constraints to `pubspec.yaml` and `pubspec.yaml.tmpl` files (since pub defaults to `<2.0.0` if omitted)
* upgrades to newer versions of various 3rd party packages (to ensure transitive dependencies have `<3.0.0` sdk constraint)
In certain cases, the test would fail before creating the (lazily created) compiler object, and then we'd
try to call shutdown() on null in those cases.
Fixes#18610
We don't actually care if we're on linux, we care only if the lcov
tool is available. We check for that explicitly just below.
Additionally there is code below which indicates that lcov is available
for macOS. Before this change that code would never execute.
This will let us re-use that code from fuchsia_tester.dart.
Tested by running the stocks example tests with coverage collection before and after.
.lcov files are identical.
fuchsia_tester.dart still assumes Dart 1. Previously, it ran tests directly
from source, flutter_platform.dart automatically runs a kernel compile when
operating in Dart 2 mode, but this assumes a functional Dart SDK is available
in the artifacts directly, and fuchsia_tester.dart mocks out the artifacts
directory with an empty temp dir.
Remaining work is:
1. Get the frontend server building as a dependency on Fuchsia.
2. Patch fuchsia_tester.dart to use a valid Dart SDK and frontend server.
This also reverts migration to Dart 2 typedef syntax.
This reverts commit 6c56bb2. (#18362)
This reverts commit 3daebd0. (#18316)
Now that Dart 1 is turned off, reapplying my change to turn on the prefer_generic_function_type_aliases analysis option, and fix all the typedefs to Dart 2 preferred syntax.
Also eliminated the unused analysis_options_repo.yaml file and turned on public_member_api_docs in analysys_options.yaml.
No logic changes, just changing the typedef syntax for all typedefs, and updating analysis options.
* It's time to #deleteDart1 (#18293)
Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.
Eliminates --preview-dart-2 / --no-preview-dart-2 support.
* Fix indentation, remove no longer necessary .toList()
* Only push udpated kernel if >0 invalidated srcs
Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.
Eliminates --preview-dart-2 / --no-preview-dart-2 support.
Now that we have thousands of tests, it doesn't make sense to display a separate line for each test. The result is just megabytes of logs that you have to scrub through to find error messages.
This will allow external tools that wrap our test harness to share the
code that generates the test bootstrap.
This change exposed an issue whereby the LocalGoldenFileComparator
was being too strict in its URI handling, so this changes relaxes
that constraint as well (and adds associated tests).
In order to avoid checking binaries into flutter/flutter,
this comparator can be used to retrieve golden files from
a sibling flutter/goldens repository.
https://github.com/flutter/flutter/issues/16859
This enables support for a `flutter_test_config.dart` configuration file,
which will be discovered and handed the responsibility of running the
test file (thus allowing it to run pre-test setup on a project level).
https://github.com/flutter/flutter/issues/16859
* Add a `matchesGoldenFile()` async matcher that will match
a finder's widget's rasterized image against a golden file.
* Add support for pluggable image comparison backends
* Add a default backend that does simplistic PNG byte
comparison on locally stored golden files.
* Add support for `flutter test --update-goldens`, which will
treat the rasterized image bytes produced during the test
as the new golden bytes and update the golden file accordingly
Still TODO:
* Add support for the `flutter_test_config.dart` test config hook
* Utilize `flutter_test_config.dart` in `packages/flutter/test`
to install a backend that retrieves golden files from a dedicated
`flutter/goldens` repo
https://github.com/flutter/flutter/issues/16859
This avoids notifying the coverage collector that we
completed the test when in fact we timed out, which in
turn avoids a downstream exception caused by the fact
that coverage collector assumes it has a valid observatory
port by which it can gather coverage data.
Fixes https://github.com/flutter/flutter/issues/16839
* Handle error count reported by frontend.
Extend compilation result from single string to a structure(string filename and integer error count).
* Use ?.
* Include engine roll with dart sdk roll.
* parse(onError) -> tryParse
* Make '?? throw' more readable and avoid issue with analyzer
* Fix test so it mocks compiler output including errors count
* Update dependencies with forced upgrade.
Now that https://github.com/flutter/flutter/issues/15261 is fixed, do the update with upgrade.
* Update test package use following upgrade
* Replace addOption(allowMultiple) with addMultiOption
Make flutter test support the --track-widget-creation flag.
Add widget creation location tests. Tests are skipped when
--track-widget-creation flag is not passed.
* Create frontend compiler in `flutter test` lazily.
This is needed to avoid unwarranted runtime dependencies on Artifacts,
for example when in non-preview-dart-2 mode.
Refactor the compilation-related code into _Compiler class.
* Fix `flutter test --preview-dart-2` following last dart roll.
Incremental compiler maintains single incremental dill file, so for every test compilation that dill file has to be copied away for execution.
* Fix lints
* Add to Artifacts, and add optional arguments to entry points for flut… (#15185)
Add to Artifacts, and add optional arguments to entry points for flutter run and test to allow for wiring up the same with preview-dart-2 internally
* Changing packages uri here is not necessary
* Run flutter tests in preview-dart-2 mode on travis flutter builds.
* Run dart2 tests on osx. Run dart tests in dart2.
* Fix name camelCase
* Default options to empty array, rather than null
* Troubleshoot failures
* More logging
* Troubleshoot: run single test
* Troubleshoot: run 15 tests
* Troubleshoot: run 15 tests with fix
* Try limit concurrency to 1
* Limit concurrency for preview-dart-2 tests to 4
* Move dart2 tests to allow_failures section
* Reinstate tests_dart_2 shard
* Raise concurrency to 8
* Reuse compiler across multiple test runs
* Allow to switch entry points when recompiling.
Actually use single compiler to incrementally recompile all tests executed by 'flutter test'.
* Remove leftover commented code
* Fix comment
* Lints
If compiler failed to produce Kernel binary then compile(...) returns
null to the caller. If we don't check for null we end up trying to
run file called "null" which causes a very confusing crash of the
flutter_tester.
* Default to strong for 'flutter test --preview-dart-2' mode
* Remove empty line
* Dont init to null
* Break up line to make it more readable
* Add missing type annotation
The main purpose of this PR is to make it so that when you set the
initial route and it's a hierarchical route (e.g. `/a/b/c`), it
implies multiple pushes, one for each step of the route (so in that
case, `/`, `/a`, `/a/b`, and `/a/b/c`, in that order). If any of those
routes don't exist, it falls back to '/'.
As part of doing that, I:
* Changed the default for MaterialApp.initialRoute to honor the
actual initial route.
* Added a MaterialApp.onUnknownRoute for handling bad routes.
* Added a feature to flutter_driver that allows the host test script
and the device test app to communicate.
* Added a test to make sure `flutter drive --route` works.
(Hopefully that will also prove `flutter run --route` works, though
this isn't testing the `flutter` tool's side of that. My main
concern is over whether the engine side works.)
* Fixed `flutter drive` to output the right target file name.
* Changed how the stocks app represents its data, so that we can
show a page for a stock before we know if it exists.
* Made it possible to show a stock page that doesn't exist. It shows
a progress indicator if we're loading the data, or else shows a
message saying it doesn't exist.
* Changed the pathing structure of routes in stocks to work more
sanely.
* Made search in the stocks app actually work (before it only worked
if we happened to accidentally trigger a rebuild). Added a test.
* Replaced some custom code in the stocks app with a BackButton.
* Added a "color" feature to BackButton to support the stocks use case.
* Spaced out the ErrorWidget text a bit more.
* Added `RouteSettings.copyWith`, which I ended up not using.
* Improved the error messages around routing.
While I was in some files I made a few formatting fixes, fixed some
code health issues, and also removed `flaky: true` from some devicelab
tests that have been stable for a while. Also added some documentation
here and there.
* Bump to Dart SDK 1.23.0-dev.10
* allows us to understand flutter usage via telemetry
* brings in `@immutable`
Fixes: #9042
* completer fix
* Update to platform 1.1.1.
* Small Flutter strong mode cleanup fixes.
These are cases where strong mode down cast composite errors
generally indicated cases that would performance or correctness
issues if Flutter code was run in a strong mode VM.
* Fix Command API so that it is always in terms of Map<String,String>.
* Fix typedef
Also, defer to test package for throttling (this will require a test
package update as well).
Also, add a lot more instrumentation to --verbose mode for tests, and
fix minor trivial things here and there, and add error handling in
more places.
Also, refactor how coverage works to be simpler and not use statics.
When a machine is heavily loaded, it can severely delay the time it
takes for the OS to start a process once it's asked to do so. Our
5 second timeout that we were giving the test process to connect
to the test harness seemed like plenty of time, were it not for the
fact that the test process itself was not being started in time when
CPU throttling was in effect.
This change updates the test timeout to begin counting only once the
test process has been started. We keep the original timeout in play
in the event that the test process *never* starts up for some reason,
but we up that timeout value to 5 minutes.
Previously, it was possible for the test harness to bail
and the test runner to complete before the platform plugin
triggered the collection of coverage data. This fixes the
race condition such that the pending coverage collection
task is recorded immediately after starting the process.
We now produce a more reasonable error message when we're missing the
flutter_test dependency in a test. Also, remove the flutter_tools stack traces
when the engine dies.
Fixes#6187
Fixes https://github.com/flutter/flutter/issues/7351
When a test runs to completion, the test harness closes the stream side of the
StreamChannel, causing the sink side to be closed as well. So by the time we
receive a test result of completed/harnessBailed, the controller sink has been
closed.
This removes direct file access from within flutter_tools
in favor of using `package:file` via a `FileSystem` that's
accessed via the `ApplicationContext`.
This lays the groundwork for us to be able to easily swap
out the underlying file system when running Flutter tools,
which will be used to provide a record/replay file system,
analogous to what we have for process invocations.
It is not enabled by default in the VM because applications not launched via the tools may try to connect with the debugger. This causes the debugger and the IDE to hang on Mac.
With this change, they're run via instance methods on an object
obtained through the context. This will allow us to substitute
that object in tests with replay/record versions to allow us to
mock out the os-layer in tests.
If the input test script contains a bad import, sky_shell will fail to
execute main(), in which case a connection to /runner is never
established and the _ServerInfo.socket never completes.
This change works around this by issuing a request on /shutdown when
sky_shell exits.
* refactor the --resident run option into a separate file
* update daemon to run --resident apps
* re-plumbing daemon start
* send app logs
* update tests
* review changes
* fix test runner
* remove PackageMap.createGlobalInstance; rely on the ctor
* review comments
This makes it possible to substitute 'flutter run' for 'flutter test'
and actually watch a test run on a device.
For any test that depends on flutter_test:
1. Remove any import of 'package:test/test.dart'.
2. Replace `testWidgets('...', (WidgetTester tester) {`
with `testWidgets('...', (WidgetTester tester) async {`
3. Add an "await" in front of calls to any of the following:
* tap()
* tapAt()
* fling()
* flingFrom()
* scroll()
* scrollAt()
* pump()
* pumpWidget()
4. Replace any calls to `tester.flushMicrotasks()` with calls to
`await tester.idle()`.
There's a guarding API that you can use, if you have particularly
complicated tests, to get better error messages. Search for
TestAsyncUtils.
