A few changes in this PR (could have been split up, but that would take
an extra 4 hours - 2 days in CI time):
- Removed the old `flutter_driver_android_test` shards and references
- Override `AndroidManifest.xml` per backend, forcing that backend to be
used (no fallbacks) or it fails
- Bumps the Android emulator to 35 due to the use of magic strings, I
think
The check for the Impeller backend actually being used is the honor
system right now, as a bug in `flutter drive`
(https://github.com/flutter/flutter/issues/162087) prevents me from
seeing error output, meaning that it's impossible to be sure a
particular Impeller backend is used (or understand why a crash happens),
so I guess I'll work on that next.
Due to https://github.com/flutter/flutter/issues/162088, we were
accidentally running our Vulkan tests as OpenGLES 🤦🏼 .
Towards https://github.com/flutter/flutter/issues/161333.
This adds the initial `.ci.yaml` configuration (marked `bringup: true`),
without changing the existing shard to avoid the problems we ran into
last time when I renamed the shard atomically, as well as piping for
providing context in which the test will run (Vulkan or OpenGLES), which
can't be introspected in the _running_ app.
In a follow-up PR I'll:
- Edit `AndroidManifest.xml` before the suite, setting `ImpellerBackend`
appropriately
- Move the bringup nodes to presubmit
- Delete the existing (legacy) node
The goal here is to have a great standalone `android_engine_test` suite
that [replaces
`scenario_app/android`](https://github.com/flutter/flutter/pull/160992).
No test is _functionally_ changed in this PR, but overview of changes:
- Finished renaming the suite `android_engine_tests` instead of
`flutter_driver_android`
- Added instructions and an environment variable for local generation of
golden-files (`UPDATE_GOLDENS=1`)
- Added explanations of the individual tests, where they live, and how
to run them locally
- Added a hybrid-composition (HC, not TLHC, which is already tested)
test
- Renamed "other_smiley" to "surface_texture_smiley" (and renamed the
original to "surface_producer_smiley")
- Removed unnecessary ".android" suffix (we will not run this on
anything but Android)
- Added a `tool/deflake.dart` to run a test suite 10x (or custom) times
locally to try and determine flakiness
After this PR, I'll add flags to let you control variants and name the
screenshots accordingly, i.e.:
- API v34 or v35
- OpenGLES or Vulkan (will require an `AndroidManifest.xml` edit during
the test instrumentation)
Towards https://github.com/flutter/flutter/issues/161261 (before I add
more tests).
This is a 1:1 move, with massaging of
`run_flutter_driver_android_tests.dart` to be more automated as tests
are added.
/cc @reidbaker
