mirror of
https://github.com/flutter/flutter.git
synced 2025-06-03 00:51:18 +00:00
e65d28319a
📜 [Doc on the Google testing Queue](https://docs.google.com/document/d/15gMCfOIUWlsRuLRMhMge-7odVHTzU8zibiVFDLdn3Pk/edit?usp=sharing) Project for tracking: https://github.com/orgs/flutter/projects/200 This updates the docs on Google testing and incorporates the proposed queue for managing blocked PRs better. Fixes https://github.com/flutter/flutter/issues/162832 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [ ] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
152 lines
6.5 KiB
Markdown
152 lines
6.5 KiB
Markdown
|
|
|
|
# How to fix a PR's failing checks
|
|
|
|
<br>
|
|
|
|
### tree-status
|
|
|
|
|
|
|
|
Unlike other checks, **tree-status** isn't tied to the pull request:
|
|
instead, it shows whether checks are passing in the main branch.\
|
|
Failures can happen for a variety of reasons and should be addressed
|
|
before anything else is merged in.
|
|
|
|
**What to do:** Once [review requirements](../Tree-hygiene.md#getting-a-code-review)
|
|
are met and all other checks are passing, a reviewer will add the
|
|
[**`autosubmit`**](../../infra/Landing-Changes-With-Autosubmit.md) label,
|
|
and then a bot will merge the PR once **tree-status** succeeds.
|
|
|
|
<br>
|
|
|
|
### Google testing
|
|
|
|
|
|
|
|
A Google testing failure could be a flake ([see below](#flaking)), or it
|
|
might be due to changes in the PR (See
|
|
[Understanding Google Testing](../../infra/Understanding-Google-Testing.md)
|
|
for more info).
|
|
Google employees can view the test output and give feedback accordingly.
|
|
|
|
The correct course of action in this case will likely be further changes in the pull request to make
|
|
it non-breaking. The Google employee assisting will be able to advise. Breaking changes create churn
|
|
for users and decay the ecosystem, so most often updating to a non-breaking change is
|
|
the correct course of action to unblock the PR.
|
|
|
|
If the breaking change _is_ approved, the [breaking change policy](../Tree-hygiene.md#handling-breaking-changes)
|
|
must be followed. This means these changes should include a dart fix when they can, or a migration guide.
|
|
|
|
Pull requests that are blocked by a Google Testing failure are tracked in the
|
|
[Google testing queue project](https://github.com/orgs/flutter/projects/200/views/1)
|
|
for resolution. Ping your reviewer if the PR has not been added to the project
|
|
for tracking.
|
|
|
|
**What to do:** If 2 weeks have gone by and nobody's looked into it,
|
|
feel free to [reach out on Discord](../Chat.md).
|
|
|
|
<br>
|
|
|
|
### ci.yaml validation
|
|
|
|
|
|
|
|
In order for checks to run correctly, the [.ci.yaml](../../../.ci.yaml)
|
|
file needs to stay in sync with the base branch.
|
|
|
|
**What to do:** This check failure can be fixed by applying the latest changes
|
|
from master.\
|
|
(The [Tree hygiene](../Tree-hygiene.md#using-git) page recommends updating
|
|
via rebase, rather than a merge commit.)
|
|
|
|
|
|
|
|
<br>
|
|
|
|
## A bug in the PR
|
|
|
|
Oftentimes, a change inadvertently breaks expected behavior.\
|
|
When this happens, usually the best way to find out what's wrong is to
|
|
[**view the test output**](#view-the-test-output).
|
|
|
|
If a **customer_testing** check is unsuccessful, it's a signal that something in the
|
|
[Flutter customer test registry](https://github.com/flutter/tests/) has failed.
|
|
This includes [package tests](../../ecosystem/testing/Understanding-Packages-tests.md)
|
|
along with other tests from open-source Flutter projects.\
|
|
If a pull request requires an update to those external tests, it qualifies as a
|
|
[**breaking change**](../Tree-hygiene.md#handling-breaking-changes);
|
|
please avoid those when possible.
|
|
|
|
If **Linux Analyze** fails, it's likely that one or more changes in the PR
|
|
violated a [linter rule](https://dart.dev/lints/).\
|
|
Consider reviewing the steps outlined in
|
|
[setting up the framework dev environment](../../Setting-up-the-Framework-development-environment.md)
|
|
so that most of these problems get caught in static analysis right away.
|
|
|
|
> [!NOTE]
|
|
> All Dart code is run through static analysis:
|
|
> this includes markdown code snippets in doc comments!
|
|
>
|
|
> See [Hixie's Natural Log](https://ln.hixie.ch/?start=1660174115) for more details.
|
|
|
|
<br>
|
|
|
|
### View the test output
|
|
|
|
Click on **Details** for the failing test, and then click
|
|
**View more details on flutter-dashboard**.
|
|
|
|
|
|
|
|
The full test output is linked at the bottom of the page.
|
|
|
|
|
|
|
|
<br>
|
|
|
|
Often, there will be a message that resembles the one below:
|
|
|
|
```
|
|
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
|
|
The following TestFailure was thrown running a test:
|
|
Expected: exactly one matching candidate
|
|
Actual: _TextWidgetFinder:<Found 0 widgets with text
|
|
"AsyncSnapshot<String>(ConnectionState.waiting, null, null, null)": []>
|
|
Which: means none were found but one was expected
|
|
|
|
When the exception was thrown, this was the stack:
|
|
#4 main.<anonymous closure>.<anonymous closure> (…/packages/flutter/test/widgets/async_test.dart:115:7)
|
|
<asynchronous suspension>
|
|
#5 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:189:15)
|
|
<asynchronous suspension>
|
|
#6 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1032:5)
|
|
<asynchronous suspension>
|
|
<asynchronous suspension>
|
|
(elided one frame from package:stack_trace)
|
|
|
|
This was caught by the test expectation on the following line:
|
|
file:///b/s/w/ir/x/w/flutter/packages/flutter/test/widgets/async_test.dart line 115
|
|
The test description was:
|
|
gracefully handles transition from null future
|
|
════════════════════════════════════════════════════════════════════════════════════════════════════
|
|
```
|
|
|
|
From there, it's just a matter of finding the failing test,
|
|
[running it locally](./Running-and-writing-tests.md),
|
|
and figuring out how to fix it!
|
|
|
|
<br>
|
|
|
|
### Flaking
|
|
|
|
A check might "flake", or randomly fail, for a variety of reasons.
|
|
|
|
Sometimes a flake resolves itself after changes are pushed to re-trigger
|
|
the checks. Consider [performing a rebase](#ciyaml-validation) to include
|
|
the latest changes from the main branch.
|
|
|
|
Flakes often happen due to **infra errors**.
|
|
For information on how to view and report infrastructure bugs, see the
|
|
[infra failure overview](../../infra/Understanding-a-LUCI-build-failure.md#overview-of-an-infra-failure-build).
|