As of flutter.dev/go/engine-cpu-profiling, we collect the CPU and
Memory usage. With work being done to collect GPU usage on iOS as well.
This adds them to the timeline summary.
Fixes: https://github.com/flutter/flutter/issues/58803
Work-around large timeline data killing devicelab machines by requesting data in 1 second intervals and combining at the end. Non-breaking change to the driver API.
* Support Flutter Driver test for Flutter Web application.
* Support Flutter Driver test for Flutter Web application.
* Fix documentation issues.
* Support Flutter Driver test for Flutter Web application.
* Fix documentation.
* Remove unused file from dartdoc check.
* Sync to date.
* Revert change to dartdoc.
* Address comments.
* Apply suggestions from code review
Co-Authored-By: Jonah Williams <jonahwilliams@google.com>
* Update copyrights.
* Update allowed list for browsers.
* Verify command line arguments for Drive command is correctly parsed.
* Make waitUntilFirstFrameRasterized throw unimplementedError for Flutter Web Driver.
* Add comment for why sync WebDriver is used.
* Update documentations.
* Add more unit tests and update documentation.
* Configure test.dart so that web_extension_test will be executed with --platform=chrome.
* Revert unnecessary changes.
* Add new file path for Windows to blacklist.
* Reconstruct the structure of flutter_driver/test/src folder to remove filtering logic in dev/bots/test.dart/
* Fix path to web_extension_test.dart.
* Add instructions for how to use WebFlutterDriver.
* Update getLayerTree to use sendCommand instead of _sendCommand.
* Update pubspec files.
* Simplify logging code in flutter_driver
It appears nobody is benefitting from this logic, so we may as well not pay the cost.
* Make all of the driver interface public.
This is the second attempt at this; it should be better than before because the Logger interface, which conflicted in various places, no longer exists.
See #45733 for original attempt.
This reverts commit 0ce9206ef4 (PR #45924).
We keep finding things that we've accidentally not made public. The
pattern we use elsewhere in the platform is to just export everything,
to avoid encouraging people from importing `src/` files directly.
* Update project.pbxproj files to say Flutter rather than Chromium
Also, the templates now have an empty organization so that we don't cause people to give their apps a Flutter copyright.
* Update the copyright notice checker to require a standard notice on all files
* Update copyrights on Dart files. (This was a mechanical commit.)
* Fix weird license headers on Dart files that deviate from our conventions; relicense Shrine.
Some were already marked "The Flutter Authors", not clear why. Their
dates have been normalized. Some were missing the blank line after the
license. Some were randomly different in trivial ways for no apparent
reason (e.g. missing the trailing period).
* Clean up the copyrights in non-Dart files. (Manual edits.)
Also, make sure templates don't have copyrights.
* Fix some more ORGANIZATIONNAMEs
This is to account for the flutter_runner migration.
fxb/fl-250 has more context. This change specifically uses
the build rules that were added as a part of: fxr/333681
* WIP on web plugin registry
* WIP on registering plugins
* WIP on web plugin registration
* Only generate `package:flutter_web_plugins` imports if plugins are
defined
* Add parsing test
* Add documentation
* Fix analyzer warnings
* add license headers
* Add tests for package:flutter_web_plugins
* Run `flutter update-packages --force-upgrade`
* Fix analyzer errors
* Fix analyzer error in test
* Update copyright and remove flutter SDK constraints
* Enable tests since engine has rolled
* add flutter_web_plugins tests to bots
* Create an empty .packages file for WebFs test
* Added a composable waitForCondition Driver/extension API.
This reverts commit df2db4e50f.
* Added SerializableWaitCondition for communicating between driver script on host and extension on device.
This relands https://github.com/flutter/flutter/pull/35297
The followings have been done to fix the broken tests:
1. Add `didSendFirstFrameRasterizedEvent` extension and its tests
2. Wait for `didSendFirstFrameRasterizedEvent` instead of
`didSendFirstFrameEvent` during start up tests
3. Mark missed (probably newly added) start up tests as flaky
Fixes https://github.com/flutter/flutter/issues/36091.
Previously, a sendRequest will be sent even if the peer is closed during
a driver test. That will cause a time out without any error information.
Such issue is unreproducible on my Mac Book Pro, but 100% reproducible
on our new Mac mini (2018).
The closing issue is tracked in https://github.com/flutter/flutter/issues/36268
Additional to this fix, we should also patch the jason_rpc_2 so the peer
will throw exception if sendRequest is attempted while the connection is
closed.
**Test**:
tiles_scroll_perf_iphonexs__timeline_summary failed without this patch.
It will pass after this patch.
I'm not sure how to add a unit test for this. Please let me know if you have
some ideas.
This patch will generate the following warning log on the new Mac Mini
```
flutter: Observatory listening on http://127.0.0.1:50192/cZPDF4sW7MM=/
Installing and launching... 10.2s
00:00 +0: scrolling performance test (setUpAll)
[info ] FlutterDriver: Connecting to Flutter application at http://127.0.0.1:1069/cZPDF4sW7MM=/
[trace] FlutterDriver: Isolate found with number: 3684677742843303
[trace] FlutterDriver: Isolate is paused at start.
[trace] FlutterDriver: Attempting to resume isolate
[trace] FlutterDriver: Waiting for service extension
[info ] FlutterDriver: Connected to Flutter application.
00:00 +0: scrolling performance test complex_layout_scroll_perf
[warning] FlutterDriver: Instance of '_WebSocketImpl' is closed with an unexpected code 1005
[warning] FlutterDriver: Peer connection is closed! Trying to restore the connection...
00:10 +1: scrolling performance test tiles_scroll_perf
[warning] FlutterDriver: Instance of '_WebSocketImpl' is closed with an unexpected code 1005
[warning] FlutterDriver: Peer connection is closed! Trying to restore the connection...
00:20 +2: scrolling performance test (tearDownAll)
00:20 +2: All tests passed!
Stopping application instance.
```
* Clean up some flutter_tools tests
* Remove arbitrary retry that happens even for fundamental errors, and generally clean up _DevFSHttpWriter.
* Update dependencies (requires fixes; see next commit)
* Fixes for new dependencies.
## Description
Print actionable warnings if `flutter drive` (where most of our performance benchmarks come from) is run in debug mode and it tries to gather benchmarks using `traceAction`.
## Related Issues
https://github.com/flutter/flutter/issues/30625
## Tests
I added the following tests:
* drive_perf_debug_warning devicelab test
* some space formattings
* always use blocks in if-else if a block is used
* format spaces in for and while
* allow multiline if conditions
* fix missing space
* add trailing commas on list/map/parameters
* add trailing commas on Invocation with nb of arg>1
* add commas for widget containing widgets
* add trailing commas if instantiation contains trailing comma
* revert bad change
For `fuchsia_compat.dart` Instead of using `runSync`, use `run` to avoid
deadlock when attempting to access specific resources like the Hub in Fuchsia.
The specific example is that in Fuchsia, the `find` command is
attempting to explore `out` which hasn't yet been serviced, as `find` is
blocking on it, causing a deadlock.
These are essentially self-inflicted race conditions. Instead of timeouts we're going to try a more verbose logging mechanism that points out when things are taking a long time.
When setting the Fuchsia logging function, it should happen before any
initialization code, as init can still cause warning/error/info messages
to get printed to logs. Since the default stderr/stdout fd's aren't
correct, this can cause a program to crash for unclear reasons.
* Remove many timeouts.
These are essentially self-inflicted race conditions. Instead of timeouts we're going to try a more verbose logging mechanism that points out when things are taking a long time.
* Get the attach tests to pass.
* Apply review comments from Todd
* More review comment fixes
* Put back the extended timeouts here now that I know why we have them...
Adds a Fuchsia compatibility function that allows for connecting to a
specific Isolate by name when running Flutter Driver on a Fuchsia
device. This will check over multiple Dart VM's in an attempt to find
an Isolate by its name.
* Fix TODO syntax.
* Clarify messages for some timeouts, to aid debugging.
* Increase some other timeouts that were a needlessly short, to reduce sources of flakes.
* Remove some more timeouts that were mostly redundant, to remove complexity.
* Minor style cleanup.
* Remove some dangerous traps (specifically, hide the explicit start/end times in TimedEvent since they shouldn't matter).
All temporary directory start with `flutter_` and have their random component separated from the name by a period, as in `flutter_test_bundle.YFYQMY`.
I've tried to find some of the places where we didn't cleanly delete temporary directories, too. This greatly reduces, though it does not entirely eliminate, the directories we leave behind when running tests, especially `flutter_tools` tests.
While I was at it I standardized on `tempDir` as the variable name for temporary directories, since it was the most common, removing occurrences of `temp` and `tmp`, among others.
Also I factored out some common code that used to catch exceptions that happen on Windows, and made more places use that pattern.
* Reland "Roll engine to version b148e628ec86b3a9a0382e0bcfae73f0390a8232 (#20427)"
This is a re-land with downgraded `package:flutter_gallery_assets`
version.
* Downgrade package:flutter_gallery_assets to 0.1.4
* Change engine.version to 81baff97c29bb08cbf8453a3f9042c5813f84ad3 (which contains an additional fix)
* Change engine.version to e3687f70c7ece72000b32ee1b3c02755ba5361ac (since mac tarballs are corrupted on earlier commit)
Reason for revert: The package:flutter_gallery_assets has removed some images which are required for the examples/flutter_gallery, so the gallery build is failing (only discovered after landing, since gallery doesn't seem to get built during github PR presubmit checks)
This CL
* rolls `engine.version` to flutter/engine@b148e628 (which includes dart sdk 2.1.0-dev)
* rolls `goldens.version` to flutter/goldens@6c45fafdf (which includes updates due to skia changes in engine)
* changes `platform.dill` to `platform_strong.dill` in various places due to flutter/engine@a84b210b
* adds explicit `environment: sdk: ">=2.0.0-dev.68 < 3.0.0"` constraints to `pubspec.yaml` and `pubspec.yaml.tmpl` files (since pub defaults to `<2.0.0` if omitted)
* upgrades to newer versions of various 3rd party packages (to ensure transitive dependencies have `<3.0.0` sdk constraint)
* Upgrade everything except matcher.
* Roll matcher (and test)
* Adjust tests that depend on flutter:test directly to depend on a shim
* Require use of package:test shim and remove other references to package:test
Our Cirrus Windows tests were not really working: they reported success even when failing.
This should fix several problems with the Cirrus Windows tests.
This uses @kevmoo's completion package to do command line completion for flutter, and a new command "bash-completion" (with alias "zsh-completion") that will output the necessary shell script setup code, and adds the hidden command "completion" that does the actual completion.
Because it adds a dependency, I also had to do flutter update-packages --force-upgrade.
Fixes#18988.
* Revert "Revert "Roll engine to fed2ea458ed49088d33eddabc546ba56d600c717 (includes dart roll) (#19044)" (#19276)"
This reverts commit cf932490b7 as it also
includes fix for type error that broke tests.
* Add type cast for dart2 type checks.
* Move up to latest goldens
* Make inDirectory() type-parameterized.
* Add typecasting to transitions_perf_test.dart and microbenchmarks.
* Add boolean flag initialization in save_catalog_screenshots.dart
* Add type conversion to gallery transition test
* Roll engine to rolled dart
Add connectionTimeout property to class that extends HttpOverrides.
Upgrade packages.
Add more type fixes.
Update goldens hash.
Add more type fixes.
Upgrade packages.
Fix tests.
Remove strong-mode from analysis options yaml file.
Increase dev/devicelab timeout from 2 to 10 seconds.
Added getVM to waitForViews. Fix type errors in compileExpression.
* Pick up engine revision with fix for microbenchmarks regression.
Increase default timeout from 20/30 to 60s.
fuchsia_tester.dart still assumes Dart 1. Previously, it ran tests directly
from source, flutter_platform.dart automatically runs a kernel compile when
operating in Dart 2 mode, but this assumes a functional Dart SDK is available
in the artifacts directly, and fuchsia_tester.dart mocks out the artifacts
directory with an empty temp dir.
Remaining work is:
1. Get the frontend server building as a dependency on Fuchsia.
2. Patch fuchsia_tester.dart to use a valid Dart SDK and frontend server.
This also reverts migration to Dart 2 typedef syntax.
This reverts commit 6c56bb2. (#18362)
This reverts commit 3daebd0. (#18316)
Now that Dart 1 is turned off, reapplying my change to turn on the prefer_generic_function_type_aliases analysis option, and fix all the typedefs to Dart 2 preferred syntax.
Also eliminated the unused analysis_options_repo.yaml file and turned on public_member_api_docs in analysys_options.yaml.
No logic changes, just changing the typedef syntax for all typedefs, and updating analysis options.
Our style guide says the k's are not necessary, and it seems like a good idea to make all the code be consistent on this.
Only naming changes to private vars: no logic changes.
In FlutterDriver.scrollUntilVisible, pause momentarily after we kick off
the waitFor() call, in order to give that call a chance to complete
before we start scrolling if the element is on-screen.
This change adds Dart VM event support (listening for when a VM starts/stops by using a periodic heartbeat).
This also adds support to connect to a specific `IsolateRef` through the flutter driver, so that when an application spawns, it can immediately be driven (as shown in included example code).
We're seeing occasional test timeouts trying to call `FlutterDriver.connect()`.
Unfortunately, when the test is timed out at the test runner level, you don't
get a meaningful test failure or stack trace of where the timeout occurrred.
Your test harness also doesn't get to clean up, which can include things like
saving the device logs to see what was going on in the device.
Thus, this change adds timeouts in the places where we've most
commonly observed hangs during tests.
* Revert "Revert "Move mockito to 3.0.0-alpha. (#15949)" (#15979)"
This reverts commit e59651f925.
* More thenReturn to thenAnswer when mocking Futures
* Revert "More thenReturn to thenAnswer when mocking Futures"
This reverts commit 194d2cf417 as we are reverting engine roll.
* Revert "Revert "More thenReturn to thenAnswer when mocking Futures""
This reverts commit 52c9e96b30.
* Add dependency override
* Fix issue
* Move mockito to 3.0.0-alpha.
This is needed so it matches mockito version used internally.
* Pick up +2 for the fixes
* Fix whenReturn in fuchsia_remote_debug_protocol
* more thenReturn becomes thenAnswer
* more thenReturn becomes thenAnswer
* more thenReturn becomes thenAnswer
Adds a package that will support connecting to and debugging devices with multiple instances of the Dart VM, and operate on multiple Isolates and Flutter Views.
Currently supports connecting to all existing Dart VMs and extracting information about all Flutter Views running across each VM.
* traverse dependencies and dev dependencies separately
* separate dev and reg deps
* update pubspecs
* add checksum string and update transitive dep string. Add additional verify command to check checksum
* cleanup comments
* add missing space, quote, run verify-onlu from bot script
* rerun tool with fixed string
* ensure correct working directory and bump versions
* flutter not flutterRoot
* Update dependencies with forced upgrade.
Now that https://github.com/flutter/flutter/issues/15261 is fixed, do the update with upgrade.
* Update test package use following upgrade
* Replace addOption(allowMultiple) with addMultiOption
30+4 has made some changes to the remoting protocol that are incompatbile
with a test harness running 30+3 (e.g. adding a type='initial' field to the
first message)
* All lists produced by JSON parsing are List<dynamic>. If more
speficic type is required then they need to be explicitly cast, e.g.
using castFrom helper;
* Function of type (ByText) -> Finder is not a subtype of
(SerializableFinder) -> Finder because ByText is in the contravariant
position;
* In Dart 2 typed(any) should be used instead of any in mockito based
tests.
* Use engine-built dart sdk
* Download dart-sdk from engine
* Move up deps to fix dart sdk constraint problem
* Update update_dart_sdk.ps1 for Windows
* Fix tests so they pass analysis
* More types for tests
* Roll engine
* Update dart sdk stamp location in flutter.bat
* Add newline
There were some problems I introduced with the last PR for this. It passed the test, but failed in practice.
This adds tests for those failure cases, adds a "--help" and fixes the test so that it doesn't try and actually download MinGit as part of the Windows test.
I added package:platform as a dependency, so I did a force upgrade on the packages.
Also, re-enabling 'create package' in the cache warming code, now that #14448 is fixed.
* Unpin package:test and upgrade packages
* Update packages/flutter/test/foundation/stack_trace_test.dart
* Also add packages/flutter_tools/test/data/asset_test/font/.dartignore to ensure that update-packages --force-upgrade does not crash.
Generate the "version" file from git tags.
Remove the old VERSION file and mentions of versions in pubspec.yaml files.
Replace the old update_versions.dart script with a new roll_dev.dart script.
Update "flutter channel".
Update "flutter upgrade", including making it transition from alpha to dev.
Update "flutter --version" and "flutter doctor".
* Revert "Reverting package changes until I can figure out how to fix Windows. (#14007)"
This reverts commit 6fda8ee821.
* Make prepare_package run on Windows
* Revert "Fixed output validation. (#14005)"
This reverts commit d84398db72.
* Revert "Update package prep script to do async process execution and emit output as it happens. (#13918)"
This reverts commit b7169c1d95.
- Switches to async process execution, which now shows output as it happens instead of in chunks when the process completes
- Now uses ProcessManager so that it may be mocked for the test.
- Adds in the download and install of mingit on Windows.
- Updated package dependencies because of added dependency on process package.
This simply updates the package dependencies by running flutter update-packages --force-upgrade.
I'm doing this with no other changes, because the last time I tried that, redness occurred. I want to isolate the problem to a "clean" update of the packages.
It looks like the plugins device_info, connectivity, and url_launcher haven't yet had their gradle configurations updated, so they fail when trying to build with the new gradle. I did not upgrade for those three packages only (in flutter_gallery) until we are ready to fix them (fixing them for master will break them for alpha users, so we need to do an alpha roll to do that).
This adds our self-compiled copy of the MinGit executable (built from the flutter/git repo) to the archive when building an archive for Windows.
I also tweaked the internal API for prepare_package.dart so that there's a single entry point to build an archive.
* Change async stubbing to use thenAnswer.
Mockito now prohibits calling thenReturn with Futures and Streams. dart-lang/mockito#79
* Update all Mockito deps to 3.0.0.
* Revert "Update all Mockito deps to 3.0.0."
This reverts commit e8ab9d37c3.
I did not correctly update the mockito dep, and there's no easy way to update to 3.0 alpha right now.
* Change thenAnswer((_) => to thenAnswer((invocation) =>
* Add Invocation type to thenAnswer lambdas
This allows the scheduler library to depend on the services library
and the painting library to depend on the scheduler library without
the services library having to depend on the scheduler library.
While I was at it I also cleaned up some of the binding logic: the
licenses logic can now be overridden (and the test library does so),
and the image cache can now be overridden as well.
This patch reorganises flutter_driver's lib/src directory into three
subdirectories:
* driver: sources transitively imported/exported by flutter_driver.dart,
which defines the API used by driver tests executed on the host machine.
* extension: sources transitively imported/exported by
driver_extension.dart, which defines the API used to instrument a
Flutter app running on the target device for use with a driver test.
* common: sources common to both driver and extension code.
This makes rolling Flutter into repositories that use Bazel (or
Bazel-like build systems) significantly more robust, due to the simpler
glob patterns involved.
1. We want to measure wall-clock duration for the benchmarks, as opposed
to thread duration (e.g., waiting on a mutex should accrue time) and
'dur' is the metric to use for that.
2. On Darwin-based systems (macOS and iOS) 'tdur' is the result of a
mach syscall lookup to thread_info. This call returns unreliable data
on iOS. Chromium, for example, disables thread time support entirely
for iOS.
I got tired of drive-by spelling fixes, so I figured I'd just take care of them all at once.
This only corrects errors in the dartdocs, not regular comments, and I skipped any sample code in the dartdocs. It doesn't touch any identifiers in the dartdocs either. No code changes, just comments.
Creates a new (hidden) flutter command 'ide-config' that will create and/or update
existing .iml files and some files under the .idea directory, as well as
removing existing *.iml files and the .idea directory.
It also:
* Adds *.iml to the .gitignore
* Removes existing .iml files from the repo, and moves them to the
packages/flutter_tools/ide_templates/intellij directory.
* Adds a flag to ide-config ('--update-templates') that will take any new .iml
files in the flutter tree and add them to the existing templates.
- If --overwrite is also specified, then all existing templates will also
be overwritten with the contents from the flutter tree, and any that have
been deleted from the flutter tree will also be removed from the
templates.
* Added new run configurations for all existing app targets that will now also
be automatically added to IntelliJ.
* Setting up the environment also includes setting the coding style guidelines
and the git VCS.
* Note that after this PR lands, Flutter developers will need to run it once to
re-create the .iml files and configuration files that have been removed.
After this PR lands, .iml files will no longer appear in the untracked files
section for git.
* fix updrade script; upgrade to the latest package versions
* exclude special dependencies from transitive closure
* fix stack trace handling in flutter_test due to stack_trace change
* change type on _emptyStackTrace
* Pin all dependencies ONCE AND FOR ALL
This replaces the secret `flutter update-packages --upgrade` with a destructive `flutter update-packages --force-upgrade` that actually goes and pins every dependency and transitive dependency in every flutter package to the same version.
* Add comments.
* only tap on widgets reachable by hit testing
* use FractionalOffset
* added tests
* check finder finds correct widget
* undo unintentional changes
* address comments
* style fix
* add Directionality in test
* fix analysis warning
The flakiness of commands_test is generally visible via:
```
drive:stdout: Expected: 'log: paint'
drive:stdout: Actual: ''
drive:stdout: Which: is different. Both strings start the same, but the actual value is missing the following trailing characters: log: paint ...
```
By returning the name of the test we will be able to understand if
another test is running by reading the `Actual` value
- Wait for full Hot Reload
- Wait for full Restart
- Fallback if checkHealth throws METHOD_NOT_FOUND.
We try to wait for the service extensions to be registered and retry.
The most recent Flutter IntelliJ plugin replaces FLUTTER_MODULE with
WEB_MODULE and eliminates the exclusion of packages/ directories.
Use of the packages/ directory was turned off by default months ago, and
is replaced by the .packages file.
* Revert "Make plugins add their repos to projects in the consuming app (#11447)"
This reverts commit abe1e2520b.
* Revert "Support for custom build types on Android (#11354)"
This reverts commit 87eec719e2.
* Revert "add a profile() method (#11443)"
This reverts commit 561d17a876.
* Revert "Fix documentation based on dartdoc's warnings (#11428)"
This reverts commit 6655074b37.
* Revert "Improve some docs around WillPopScope. (#11429)"
This reverts commit 58a28a2965.
* Revert "temporarily disable broken driver test in integration_ui (#11440)"
This reverts commit 764515ec78.
* Revert "style fix"
This reverts commit 00bfc86630.
* Revert "tests for waitFor/waitForAbsent"
This reverts commit 31d2ee9c0c.
* Revert "Always evaluate the finder in `driver.waitFor()` and `driver.waitForAbsent()`"
This reverts commit 11d7c79b99.
Mainly, this adds documentation to members that were previously
lacking documentation.
It also adds a big block of documentation about improving performance
of widgets.
This also removes some references to package:collection and adds
global setEquals and listEquals methods in foundation that we can use.
(setEquals in particular should be much faster than the
package:collection equivalent, though both should be faster as they
avoid allocating new objects.) All remaining references now qualify
the import so we know what our remaining dependencies are.
Also lots of code reordering in Flutter driver to make the code
consistent and apply the style guide more thoroughly.
The main purpose of this PR is to make it so that when you set the
initial route and it's a hierarchical route (e.g. `/a/b/c`), it
implies multiple pushes, one for each step of the route (so in that
case, `/`, `/a`, `/a/b`, and `/a/b/c`, in that order). If any of those
routes don't exist, it falls back to '/'.
As part of doing that, I:
* Changed the default for MaterialApp.initialRoute to honor the
actual initial route.
* Added a MaterialApp.onUnknownRoute for handling bad routes.
* Added a feature to flutter_driver that allows the host test script
and the device test app to communicate.
* Added a test to make sure `flutter drive --route` works.
(Hopefully that will also prove `flutter run --route` works, though
this isn't testing the `flutter` tool's side of that. My main
concern is over whether the engine side works.)
* Fixed `flutter drive` to output the right target file name.
* Changed how the stocks app represents its data, so that we can
show a page for a stock before we know if it exists.
* Made it possible to show a stock page that doesn't exist. It shows
a progress indicator if we're loading the data, or else shows a
message saying it doesn't exist.
* Changed the pathing structure of routes in stocks to work more
sanely.
* Made search in the stocks app actually work (before it only worked
if we happened to accidentally trigger a rebuild). Added a test.
* Replaced some custom code in the stocks app with a BackButton.
* Added a "color" feature to BackButton to support the stocks use case.
* Spaced out the ErrorWidget text a bit more.
* Added `RouteSettings.copyWith`, which I ended up not using.
* Improved the error messages around routing.
While I was in some files I made a few formatting fixes, fixed some
code health issues, and also removed `flaky: true` from some devicelab
tests that have been stable for a while. Also added some documentation
here and there.
