9cc132d4c2
A few changes in this PR (could have been split up, but that would take an extra 4 hours - 2 days in CI time): - Removed the old `flutter_driver_android_test` shards and references - Override `AndroidManifest.xml` per backend, forcing that backend to be used (no fallbacks) or it fails - Bumps the Android emulator to 35 due to the use of magic strings, I think The check for the Impeller backend actually being used is the honor system right now, as a bug in `flutter drive` (https://github.com/flutter/flutter/issues/162087) prevents me from seeing error output, meaning that it's impossible to be sure a particular Impeller backend is used (or understand why a crash happens), so I guess I'll work on that next. Due to https://github.com/flutter/flutter/issues/162088, we were accidentally running our Vulkan tests as OpenGLES 🤦🏼 .
5.5 KiB
android_engine_test
This directory contains a sample app and tests that demonstrate how to use the (experimental) native Flutter Driver API to drive Flutter apps that run on Android devices or emulators, interact with and capture screenshots of the app, and compare the screenshots against golden images.
Caution
This test suite is a very end-to-end suite that is testing a combination of the graphics backend, the Android embedder, the Flutter framework, and Flutter tools, and only useful when the documentation and naming stays up to date and is clearly actionable.
Please take extra care when updating the test suite to also update the REAMDE.
How it runs on CI (LUCI)
See dev/bots/suite_runners/run_android_engine_tests.dart, but tl;dr:
# TIP: If golden-files do not exist locally, this command will fail locally.
SHARD=android_engine_vulkan_tests bin/cache/dart-sdk/bin/dart dev/bots/test.dart
SHARD=android_engine_opengles_tests bin/cache/dart-sdk/bin/dart dev/bots/test.dart
Running the apps and tests
Each lib/{prefix}_main.dart file is a standalone Flutter app that you can run
on an Android device or emulator.
flutter_rendered_blue_rectangleexternal_texture/surface_producer_smiley_faceexternal_texture/surface_texture_smiley_faceplatform_view/hybrid_composition_platform_viewplatform_view/texture_layer_hybrid_composition_platform_viewplatform_view/virtual_display_platform_viewplatform_view_tap_color_change
flutter_rendered_blue_rectangle
This app displays a full screen blue rectangle. It mostly serves as a test that Flutter can run at all on the target device, and that the Flutter (native) driver can take a screenshot and compare it to a golden image. If this app or test fails, it's likely none of the other apps or tests will work either.
# Run the app
$ flutter run lib/flutter_rendered_blue_rectangle_main.dart
# Run the test
$ flutter drive lib/flutter_rendered_blue_rectangle_main.dart
external_texture/surface_producer_smiley_face
This app displays a full screen rectangular deformed smiley face with a yellow
background. It tests the SurfaceProducer API end-to-end, including historic regression cases around
backgrounding the app, trimming memory, and resuming the app.
# Run the app
$ flutter run lib/external_texture/surface_producer_smiley_face_main.dart
# Run the test
$ flutter drive lib/external_texture/surface_producer_smiley_face_main.dart
external_texture/surface_texture_smiley_face
This app displays a full screen rectangular deformed smiley face with a yellow
background. It tests the SurfaceTexture API end-to-end.
# Run the app
$ flutter run lib/external_texture/surface_texture_smiley_face_main.dart
# Run the test
$ flutter drive lib/external_texture/surface_texture_smiley_face_main.dart
platform_view/hybrid_composition_platform_view
This app displays a blue orange gradient, the app is backgrounded, and then resumed. It tests the Hybrid Composition implementation.
# Run the app
$ flutter run lib/platform_view/hybrid_composition_platform_view_main.dart
# Run the test
$ flutter drive lib/platform_view/hybrid_composition_platform_view_main.dart
platform_view/texture_layer_hybrid_composition_platform_view
This app displays a blue orange gradient, the app is backgrounded, and then resumed. It tests the Texture Layer Hybrid Composition implementation.
# Run the app
$ flutter run lib/platform_view/texture_layer_hybrid_composition_platform_view_main.dart
# Run the test
$ flutter drive lib/platform_view/texture_layer_hybrid_composition_platform_view_main.dart
platform_view/virtual_display_platform_view
This app displays a blue orange gradient, the app is backgrounded, and then resumed. It tests the Virtual Display implementation.
# Run the app
$ flutter run lib/platform_view/virtual_display_platform_view_main.dart
# Run the test
$ flutter drive lib/platform_view/virtual_display_platform_view_main.dart
platform_view_tap_color_change
This app displays a blue rectangle, using platform views, which upon being tapped (natively, not by Flutter), changes from blue to red.
# Run the app
$ flutter run lib/platform_view_tap_color_change_main.dart
# Run the test
$ flutter drive lib/platform_view_tap_color_change_main_test.dart
Deflaking
Use tool/deflake.dart <path/to/lib/main.dart> to, in 1-command:
- Build an APK.
- Establish a baseline set of golden-files locally.
- Run N tests (by default, 10) in the same state, asserting the same output.
For example:
dart tool/deflake.dart lib/flutter_rendered_blue_rectangle_main.dart
For more options, see dart tool/deflake.dart --help.