* Allow FlutterTester to be provided with the working directory for execution
Previously this test set fs.currentDirectory which prevents running tests concurrently. This allows setting the working directory for a FlutterTester in the cosntructor (optionally) and passes it through from the test (without setting fs.currentDirectory).
* Remove trailing whitespace
* Add a detach command to detach without terminating
Fixes#21154.
* Bump protocol version for app.detach
* Tweak to detach/quit text
* Change logPrefix to named param
* Unskip integration tests on Windows that now pass
The referenced issue is closed and these tests pass for me locally on Windows now.
* Remove import that's no longer used
* Add another issue that is afffecting these tests on Windows
* Add a hot restart test and mark skip on Windows
Skipped due to https://github.com/flutter/flutter/issues/21348.
This is a bit of a stab in the dark for a flake sometimes seen on bots (but I can't repro locally) #20822. Detaching from a Flutter app currently seems to terminate it, so it may be the cause of the "app not found" when trying to stop the app subsequently.
This change means we stop the app from the main process first, before terminating the attach process.
* Don't time out if process terminates when told to stop but response was not processed
While trying to reproduce #20822 I found a condition where the app may quit before the `app.stop` response is received but this code (which is called in integration test teardown) would sit around waiting and time out. With this change, the process exit is considered a valid response to the stop request.
* Add process exit to log for better errors in failures
This log can be turned on for debugging but is also dumped when a test fails (in certain conditions). With this in the log, it'll be clear if the timeout is because we're waiting for an event but the process quit.
This test is sometimes flaky (#20822) because the two processes try to stop the same app. This fix changes to just gracefully terminate the attach process without explicitly trying to stop and then uses the original spawning process to stop the app.
I can't repro the flake locally to be certain, but I've verified only one stop command is sent now so it *should* be good.
* Don't keep parsing results once we've had the event we expected
* Report errors if an app.stop event is received while waiting for another event
* Don't throw if we see app.stop event when we've just sent an app.stop request
* Improve debug print to include --start-paused if being used
* Improve wrapping
All temporary directory start with `flutter_` and have their random component separated from the name by a period, as in `flutter_test_bundle.YFYQMY`.
I've tried to find some of the places where we didn't cleanly delete temporary directories, too. This greatly reduces, though it does not entirely eliminate, the directories we leave behind when running tests, especially `flutter_tools` tests.
While I was at it I standardized on `tempDir` as the variable name for temporary directories, since it was the most common, removing occurrences of `temp` and `tmp`, among others.
Also I factored out some common code that used to catch exceptions that happen on Windows, and made more places use that pattern.
* Reland "Roll engine to version b148e628ec86b3a9a0382e0bcfae73f0390a8232 (#20427)"
This is a re-land with downgraded `package:flutter_gallery_assets`
version.
* Downgrade package:flutter_gallery_assets to 0.1.4
* Change engine.version to 81baff97c29bb08cbf8453a3f9042c5813f84ad3 (which contains an additional fix)
* Change engine.version to e3687f70c7ece72000b32ee1b3c02755ba5361ac (since mac tarballs are corrupted on earlier commit)
Reason for revert: The package:flutter_gallery_assets has removed some images which are required for the examples/flutter_gallery, so the gallery build is failing (only discovered after landing, since gallery doesn't seem to get built during github PR presubmit checks)
This CL
* rolls `engine.version` to flutter/engine@b148e628 (which includes dart sdk 2.1.0-dev)
* rolls `goldens.version` to flutter/goldens@6c45fafdf (which includes updates due to skia changes in engine)
* changes `platform.dill` to `platform_strong.dill` in various places due to flutter/engine@a84b210b
* adds explicit `environment: sdk: ">=2.0.0-dev.68 < 3.0.0"` constraints to `pubspec.yaml` and `pubspec.yaml.tmpl` files (since pub defaults to `<2.0.0` if omitted)
* upgrades to newer versions of various 3rd party packages (to ensure transitive dependencies have `<3.0.0` sdk constraint)
* Upgrade everything except matcher.
* Roll matcher (and test)
* Adjust tests that depend on flutter:test directly to depend on a shim
* Require use of package:test shim and remove other references to package:test
* Skip tests instead of expecting failure, because expected failure = timeout which is slow
Some of these tests are known to fail and were coded to expect it; however because the failure is a timeout it's just dragging out test runs (shortening timeouts will make the tests flakier).
* Make it easier to find skipped tests
* Change SKIP to TODO
* Skip integration tests that use flutter-tester on Windows
This is known to be broken. I will un-skip all of these once a fix for flutter-tester lands on Windows.
* Save with added import!
* Add tests to ensure flutter run does not quit at startup
These tests require https://github.com/dart-lang/vm_service_client/pull/38 to be merged and currently the third test (pause-on-exceptions) fails because of a bug.
* Remove TODO
Probably makes most sense this way for now.
* Address PR comments
* Extract some of startApp into a reusable method
* Get basic attach --machine working
* Attach --machine tweaks
Move validation to validate method and create daemon early so we get the startup event before trying to get a connection.
* Bump daemon version so we know whether it's valid to flutter attach
* Tweak output text
* Swap package imports for relative
* Review tweaks (naming, formatting, typedefs)
* Separate arguments from process spawning
This will make calling attach easier
* Add a basic test for flutter attach --machine
* Fix crash if port unforward modifies the list of forwarded ports
* Add a no-op port forwarder for flutter-tester
* Switch to using BasicProject instead of our own inline code
* Fix expectation in test now we have a portForwarder
* Remove stale TODO (this is done)
* Tweak formatting
* Change some Completers to void to fix Dart 2 issues
* Make cleanup of flutter processes in tests more reliable
* Fix quit signals
I confused SigInt&SigTerm for SigTerm&SigKill when I first did this. SigTerm can be blocked and doesn't guarantee the process will be terminated.
* Don't use deprecated constants
* Fix typo
* Add some additional info to debug buffer
* Fix return types on Futures
* Allow a longer timeout for app launches
* Include timings in errors when waiting for events
* Extract a helper that adds a timeout and includes message logging
* Ensure all timeouts include messages
* Ensure logs are set up before we invoke actions
* Ensure all messages are logged during timeout periods
This adds a new stream _allMessages that collects all output (as routed through debugPrint) and updates _timeoutWithMessages to use that instead of stdin/out of only the flutter run process (since that was missing stdin - the sends, and both input/output to the VM service).
* Revert "Revert "Roll engine to fed2ea458ed49088d33eddabc546ba56d600c717 (includes dart roll) (#19044)" (#19276)"
This reverts commit cf932490b7 as it also
includes fix for type error that broke tests.
* Add type cast for dart2 type checks.
* Move up to latest goldens
* Make inDirectory() type-parameterized.
* Add typecasting to transitions_perf_test.dart and microbenchmarks.
* Add boolean flag initialization in save_catalog_screenshots.dart
* Add type conversion to gallery transition test
* Roll engine to rolled dart
Add connectionTimeout property to class that extends HttpOverrides.
Upgrade packages.
Add more type fixes.
Update goldens hash.
Add more type fixes.
Upgrade packages.
Fix tests.
Remove strong-mode from analysis options yaml file.
Increase dev/devicelab timeout from 2 to 10 seconds.
Added getVM to waitForViews. Fix type errors in compileExpression.
* Pick up engine revision with fix for microbenchmarks regression.
Increase default timeout from 20/30 to 60s.
* Rename util -> test_utils
* Rename flutter_test_driver -> test_driver
* Switch testWithContext -> test
* Remove unused import
* Move test project into a class to make it easier to have multiple of these
Each "TestProject" class can contain its files and things like named breakpoint locations.
* Split expression evaluation tests into own file
* Include last response in error messages
* Update expectations based on current bugs
* Fix async-ness in tests
* Fix incorrect expectation in test
* Fix incorrect evaluations
* Remove skips for tests that are now passing on master
* Expect pass on Linux
🤷♂️
* Call the code
* Skip expression evaluation tests on Windows
* Skip whole group, not just one test
* Remove duplicated method from merge
* Fix misplaced close of group
* Remove code that was duplicated from test we copied
Not sure how this ended up in here?
* Re-fix typo
This test was designed to ensure flutter_tester keeps running (previously it would quit immediately). However it's turned out ot be rather flaky and we have new tests on the way that supersede this by actually testing real things (debug stepping, reloading, expression evaluation).
After landing the un-skip this test failed with a timeout. It then passed on the next build (!). I think it's too flaky to leave running until we can better diagnose what's happening.
This test is failing on mac_bot (but passing elsewhere) because flutter-tester is apparently quitting earlier than expected. Locally it fails with an even weirder error and almost all tests are failing with "Compilation failed" (this isn't happening on the builds, so something is bad on my MacBook). Marking as skip to fix build while investigating; there's no real impact of this test not running; it's testing a tool that its itself used for testing (and not currently in any way that should be affected by this failure).
