Increased the memory size of multiple_flutters to help identify regressions in performance
This is not a regression and will affect benchmark's readings.
Makes the android_semantics_testing test not fail for newer versions of Talkback.
Also, since the test now depends on pub_semver, upgraded the pubspec files.
The only substantive changes in the PR are in dev/integration_tests/android_semantics_testing/test_driver/main_test.dart
This change updates the platform runners in the benchmarks subdir.
I've attempted to replace the entire group with new code as if the project were recreated with flutter create so that they don't become a Frankenstein of minor changes to make it work again, but just plain old generated code that can be replaced and updated in place.
This fixes an issue with the Dart Head-Head-Head CI bot, where it was seeing failures because of concurrent execution of the snippets tool.
It adds a single run of the snippets tool right after activation to force the compiling of the package executable, which is what has trouble with concurrent invocation.
This is a temporary measure until dart pub global run is fixed.
This does a cleanup of the examples, removing all of the "section" markers and extra comments that we don't need anymore now that the samples are no longer in the source code. It also removes the --template arguments from the {@tool dartpad} and {@tool sample} directives, since those are no longer used. It converts two examples that I discovered were still embedded into linked examples in the examples folder.
I didn't delete the templates from the snippets config folder yet, because there are still embedded samples in the dart:ui package from the engine that use them. Once dart:ui no longer uses the templates, they can be removed.
I bumped the version of the snippets package to pick up a change that allows removal of the --template argument.
This adds a smoke test for every single API example. It also fixes 17 tests that had bugs in them, or were otherwise broken, and even fixes one actual bug in the framework, and one limitation in the framework.
The bug in the framework is that NetworkImage's _loadAsync method had await response.drain<List<int>>();, but if the response is null, it will throw a cryptic exception saying that Null can't be assigned to List<int>. The fix was just to use await response.drain<void>(); instead.
The limitation is that RelativePositionedTransition takes an Animation<Rect> rect parameter, and if you want to use a RectTween with it, the value emitted there is Rect?, and one of the examples was just casting from Animation<Rect> to Animation<Rect?>, which is invalid, so I modified RelativePositionedTransition to take a Rect? and just use Rect.zero if the rect is null.
This extracts the sample code out from the API doc comments, and places them in separate files on disk, allowing running of the examples locally, testing them, and building of slightly larger examples.
Switch document generation to use the snippets package instead of the snippets code in the Flutter repo. In the process, some bugs in sample code analysis have been fixed, and I've fixed some more errors in the samples.
This will allow the snippets package to be developed separately from the Flutter repo, and reduce the code in the Flutter repo.
The snippets code is deleted in this PR.
I also converted some comments in the snippet templates to be regular comments instead of doc comments, because having a doc comment block before the imports causes the Dart import sorter to lose the comment. They should have been regular comments in the first place.
The snippets package resides in the assets-for-api-docs repo.
The sample analysis has also been converted to be run in parallel, and I've bumped the Dartdoc version to 1.0.2.
Added a check to the analyzer script to detect skipped tests that aren't commented.
The comment following the `skip` parameter should include either a link to a
github issue tracking the reenabling of the test, or a '[intended]' tag with
a brief description of why the test should never be enabled for the given
condition.
Python 2 has been end-of-life'd since January 2020, all Python scripts
on which Flutter depends have been migrated to Python 3. This changes
documentation and remaining invocations to explicitly invoke python3.
Issue: https://github.com/flutter/flutter/issues/83043
This fixes how the sample analyzer handles missing_identifier errors. It was looking at the wrong line, and missing an else clause, so it was silently allowing missing_identifier errors to pass.
In addition, this fixes the sample generation so that it uses the correct filename for the output files: it previously was looking for the first line that had a filename, which was meant to indicate a non-generated line. This change adds a new Line.generated constructor for generated lines, so that they can also have the correct filename associated with them.
The (new, not yet used) code gen for iOS was setting up a std::map from key codes to logical and physical key codes, but it was using uint32_t, which isn't big enough to hold the Flutter key codes.
Also, iOS needs to be able to filter out function keys, so I added a function key set.
Remove `//dev/bots/run_fuchsia_tests.sh` as there are no longer any references to it.
Migrating tests to cfv2 will be easier if we don't have to account for this script.
See https://fxbug.dev/79691.
This includes some major internal changes that should improve performance (the AOT template compiler) and the new lookup code. The big changes noticeable for Flutter will be resolution of field formal parameters, extension method support, and more consistent disambiguation in comment reference lookups.
While a vast net improvement, this PR will change a few links to point to the wrong place. #85484 will address that after this lands, as there was no good way to specify what the user wanted unambiguously before dartdoc 1.0.0 in a few cases. That PR includes more details on the introduced regressions and link changes.
This turns on order shuffling for all tests that don't fail with it on, marking those tests that do fail with a tag so that they will be run without shuffling on.
To determine which tests fail with it on, I ran all the tests 100 times with different random shuffle seeds, and then also ran it with the date seeds from today until the end of July, and tagged all of the test suites (files) that fail, with a seed that caused them to fail.
This adds avoid_dynamic_calls to the list of lints, and fixes all instances where it was violated.
Importantly, this lint is NOT turned on for flutter/packages/test, because those changes are happening in another PR: #84478
This switches the sample analysis code to use package:flutter_lints instead of the flutter repo analysis options, so that they are compatible with a similar change to DartPad.
This is bassically reapplying #71721, but only enables it on web tests.
There are known issues that several tests under the `integration.shard`
depend on a specific platform, and as a result they require some
additional flexiblity (bots need to build more than one engine, and the
test flags should allow for secondary engines to be picked by such
tests).
By enabling this on the web-test shard, we will reduce the false
positives in the dart-flutter-HHH-web bot.
Fixing the more general problem is tracked by #72368.
This turns off the "directives_ordering" lint when analyzing samples, since it's indeterminate what the ordering will be, given that samples derive their imports both from the template and the example code.
This is temporary to avoid broken builds, but the correct solution is to reorder the output so that the imports are ordered properly so that we give the proper example for our users.
This reverts commit 4300226a50.
Reason for revert: The original PR correctly passes down the local
engine flags to subprocesses of the tests. Though this causes issues
because some tests assume a specific target but do not modify the
value for `--local-engine` (which is set to `host_debug`) to reflect
that.
See more discussion here:
https://github.com/flutter/flutter/pull/71721#issuecomment-743218994
Adds a new Luci test shard that runs flutter/plugins tests.
For now only the analysis phase of the flutter/plugins test suite is executed.
Manual invocation on Luci completed successfully: 3210bb2dac/+/annotations
* Move web integration tool tests to web.shard
Web integration tool tests depend on DDC changes in SDK. This change
moves them to a separate shard and subshard so CI bot configurations
can run them separately.
In particular, with will allow running those tests on dart CI flutter
HHH web bot instead of a non-web one, allowing early detection and easy
classification of issues caused by SDK changes as VM- or Web related.
* Enabled verbose mode for flaky web_tool_tests
* Split out the test changes to be commited first
This reverts ae0a9cb560 and therefore relands #69629.
Additionally, `flutter update-packages --force-upgrade` has been run to update `pubspec.yaml` to resolve the analyze test failures.
For the ease of code reviews, this only includes minimal code from
MetricPoint, GoogleBenchmarksParser, and their unit tests.
See go/flutter-metrics-center-migration for the overall plan.
Enables build_test for Linux. Currently only flutter_gallery has platform directories for the desktop platforms, so this will run only that build, but this will provide an end-to-end build test for Linux.
Other example/test projects can be brought online for Linux in the future just by adding the linux/ directory to the project.
Currently we only have a test that measures if flutter drive can fail. Unfortunately it has been failing for multiple years based on the fact that the error is due to a missing Gradle config.
Add a smoke success test case, and update both tests to use the test API and the tester-device.
Rolls in several fixes to web tooling, including better handling of absolute file imports. Updates to latest vm service to unblock null safety mode query
Co-authored-by: Gary Roumanis <grouma@google.com>
Co-authored-by: Anna Gringauze <annagrin@google.com>
Enables build_test for Windows. Currently only flutter_gallery has platform directories for the desktop platforms, so this will run only that build, but this will provide an end-to-end build test for Windows.
Other example/test projects can be brought online for Windows in the future just by adding the windows/ directory to the project.
Enables build_test for Windows, macOS, and Linux. Currently only
flutter_gallery has platform directories for the desktop platforms, so
this will run only that build, but this will provide an end-to-end build
test for all three desktop platforms.
Once this lands, other example/test projects can be brought online for
desktop platforms in the future just by adding the relevant platform
directories to the project.
* Fix docset generation.
* Remove checks to be able to test.
* More logging
* More logging
* More logging.
* more logging.
* More logging.
* Add flags to firebase command.
* Re-enable docset generation.
* Disable firebase step.
* Comment docset to speed up validations.
* Re-enable docset.
* Add back condition to prevent uploading docs from PRs.
The train command does nothing and was originally added to provide a no-output default for generating app-jit snapshots. The inject-plugins command is only for a repo-only analysis check, which is not necessary since we regenerate during pub get.
#29805
* adding tests that uses integration_test (e2e) package to flutter
* change the package name for the import
* fix licenses. fix README commands. add links
* adding dependency change auto generated by the tool
* more analyzer error fixes
This converts the packages/flutter/test/services directory to NNBD, now that the services package is converted.
I changed the signature of checkMessageHandler and checkMockMessageHandler on BinaryMessenger to take a nullable handler, since the tests wanted to check to make sure a handler wasn't set, and that functionality no longer works if the handler is non-nullable.
Add integration tests to verify that ddc and dart2js can be built and run in sound mode. Updates dart2js compilation to insert a language version comment into the generated entrypoint if necessary.
dart-lang/sdk#42253
Check in linux and windows platform code now that they are stable, so that we could use in devicelab in the future. Removed the ICO from the windows example to avoid analysis check, and since it won't be important for benchmarking or UI tests
Updates all null safe dependencies to versions that allow 2.10 stable and 2.11 dev releases.
Also updates flutter_goldens and flutter_goldens_client to allow 2.11 dev.
Currently an invocation of flutter/dart will always attempt to acquire a lock. This can pose problems for tools that attempt to run multiple dart/flutter instances.
Instead update the lock logic (on Linux/macOS) so that we only attempt to acquire it if an update/snapshot needs to be performed. To avoid repeatedly performing downloads/snapshots if multiple flutter/dart invocations are fired off concurrently when an update needs to be performed, do a second check of the download/snapshot condition after the lock is released.
Additionally, moves all of the building/debug output to stderr on both the bash and batch scripts. This allows machine mode consumption of the tool to ignore needing to parse/handle the rebuild messages.
Port the deprecated settings devicelab test to tool integration shard. Tests that apps can be built using the deprecated android/settings.gradle file.
Part of #65790
Currently these tests are taking CI time without providing any value, because there is no verification that network connectivity is avoided. This is better covered by a unit test that verifies that --offline is passed to pub.
To allow tests to begin migrating to null safety, sound null safety needs to be disabled. Also removes the experiment flag that is unnecessary since the framework is on the SDK allowlist.
Like Android/iOS, only enable --null-assertions if asked. Previously this was enabled by default for web, but in general this has proved to be too breaking to enable by default.
#61042
This reverts #62710, since it has a noSuchMethodError in some circumstances.
This isn't just a straight git revert, because I had to resolve the conflicts in the reverted pubspec.yaml files and re-run flutter update-packages --force-upgrade to make the checksums match.
There are some occasional difficult to repro flakes on the tool integration tests. Try running only a single isolate to simplify the environment a bit.
Enable null safety asserts for web debug mode. This induces runtime asserts at the boundaries between null safe and non-null safe libraries. Adds integration test that validates assertion error is thrown.
#61042
Use tester (disclaimer, written by me) to unit test the flutter tools general shard. Since these tests are designed to be hermetic, the single compilation/single isolate model offers a tremendous speedup over the default pub run test workflow. On previous linux shard, test execution alone took 10:30, while now it takes only 1:30.
Update all versions, specifically file which fixes some bugs that caused tests to fail if run in different orders:
- systemTemp directories created by MemoryFileSystem will allot names based on the file system instance instead of globally.
- MemoryFile.readAsLines()/readAsLinesSync() no longer treat a final newline in the file as the start of a new, empty line.
- RecordingFile.readAsLine()/readAsLinesSync() now always record a final newline.
Ensure that the language version of the test/web generated entrypoint matches the language version of the test file to run, or the overall package language version if no annotation is provided.
Also combines experiments into extraGenSnapshot/ExtraFrontEndOptions. Allows providing --no-sound-null-safety to allow out of order migration and running.
Fixes#58003
The .dart_tool directory can contain build assets, as well as the package_config.json file - the replacement for .packages. Remove it for the same reason we remove .packages
An upcoming Dart SDK change
(fa2fd41166)
changes the precise text of the exception generated by `utf8.decode`
if a non-UTF8 file is found. This is causing a breakage in the Dart
team's `flutter-analyze` bot (and will presumably cause a breakage in
the corresponding Flutter bot as soon as this change is rolled into
Flutter). To avoid this breakage, the bot shouldn't rely on the exact
exception text; it is sufficient to simply catch a FormatException.
Ensure iOS and android builds can be correctly cached. Use the performance-measurement-file to verify that all targets were skipped on the second invocation. This is only run on the flutter_gallery build.
This moves the app template more toward being a more generic starting
point for any Flutter application, eliminating some hard-code
assumptions about there being a single window/engine pair that is
directly bound to the life of the application:
- Moves the runloop into its own class, making it capable of servicing
any number of engine instances.
- Moves the logic for setting up a window containing only a Flutter view
into a window subclass for ease of re-use.
- Makes quit-on-window-close an optional property. (Long term this
should be even more generic, like a quit-when-last-window-closes
option, but this is a short-term improvement that removes the binding
between the runloop and the window).
- Allows for multiple instances of Win32Window to exist without issues
relating to the window class registration.
Since there are getting to be a non-trivial number of files associated
with the runner, this moves the source into a runner/ directory, as is
already done on some other platforms.
Note that creating multiple Flutter windows at the same time still
doesn't work correctly even with this change, but this addresses some of
the known issues, and makes it easier to test in the future (e.g., for
debugging engine-level issues with multiple instances).
Fixes#45397
Adds initial support for flutter create of apps and plugins. This is derived from the current FDE example app and sample plugin, adding template values where relevant.
Since the APIs/tooling/template aren't stable yet, the app template includes a version marker, which will be updated each time there's a breaking change. The build now checks that the template version matches the version known by that version of the tool, and gives a specific error message when there's a mismatch, which improves over the current breaking change experience of hitting whatever build failure the breaking change causes and having to figure out that the problem is that the runner is out of date. It also adds a warning to the create output about the fact that it won't be stable.
Plugins don't currently have a version marker since in practice this is not a significant problem for plugins yet the way it is for runners; we can add it later if that changes.
Fixes#30704
* Fix web test flakiness; enable web golden tests
The fix is three-part:
- Only allow one test to load _and_ test at any point in time.
- Use a fresh Chrome instance for each test file.
- Increase Cirrus resources.
The first two changes only fix the "Unknown error loading" error, but not hanging tests. The resource increase also prevents hanging tests.
Other minor changes:
- Remove test batching (it's no longer necessary)
- Fix the Chrome class, which was using the wrong Completer.
Move Flutter Gallery to dev/integration_tests/ as it is an older copy used only for testing. The current version of the Flutter Gallery now lives in https://github.com/flutter/gallery.
The stocks example app is outdated and deprecated, but we still use it for some benchmark tests. Moving it into the benchmarks directory to indicate its status.
dev_finder name is deprecated. device-finder is preferred.
Functionality remains identical, this is an aesthetic change.
This change was generated via:
$ find . -not -path '*/\.*' -type f -exec sed -i 's/(^import.*)!dev_finder/device-finder/g' {} \;
* [fuchsia] Create an entry-point script to run driver tests on Fuchsia
This script currently only paves the device. This will soon run a
driver test on a NUC in LUCI.
* reference the engine script and fix comment
* Update packages.
* Add many more global analyses.
* Catch trailing spaces and trailing newlines in all text files.
Before we were only checking newly added files, but that means we
missed some.
* Port the trailing spaces logic to work on Windows too.
* Correct all the files with trailing spaces and newlines.
* Refactor some of the dev/bots logic into a utils.dart library.
Notably, the "exit" and "print" shims for testing are now usable
from test.dart, analyze.dart, and run_command.dart.
* Add an "exitWithError" function that prints the red lines and
then exits. This is the preferred way to exit from test.dart,
analyze.dart, and run_command.dart.
* More consistency in the output of analyze.dart.
* Refactor analyze.dart to use the _allFiles file enumerating logic
more widely.
* Add some double-checking logic to the _allFiles logic to catch
cases where changes to that logic end up catching fewer files
than expected (helps prevent future false positives).
* Add a check to prevent new binary files from being added to
the repository. Grandfather in the binaries that we've already
added.
* Update all the dependencies (needed because we now import crypto in
dev/bots/analyze.dart).
* Support Flutter Driver test for Flutter Web application.
* Support Flutter Driver test for Flutter Web application.
* Fix documentation issues.
* Support Flutter Driver test for Flutter Web application.
* Fix documentation.
* Remove unused file from dartdoc check.
* Sync to date.
* Revert change to dartdoc.
* Address comments.
* Apply suggestions from code review
Co-Authored-By: Jonah Williams <jonahwilliams@google.com>
* Update copyrights.
* Update allowed list for browsers.
* Verify command line arguments for Drive command is correctly parsed.
* Make waitUntilFirstFrameRasterized throw unimplementedError for Flutter Web Driver.
* Add comment for why sync WebDriver is used.
* Update documentations.
* Add more unit tests and update documentation.
* Configure test.dart so that web_extension_test will be executed with --platform=chrome.
* Revert unnecessary changes.
* Add new file path for Windows to blacklist.
* Reconstruct the structure of flutter_driver/test/src folder to remove filtering logic in dev/bots/test.dart/
* Fix path to web_extension_test.dart.
* Add instructions for how to use WebFlutterDriver.
* Update getLayerTree to use sendCommand instead of _sendCommand.
* Update pubspec files.
* Add tests to gen_l10n.dart tool
* Separate out LocalizationsGenerator class to improve testability of code
* Add testing dependencies to dev/tools
* Integrate dev/tools testing to flutter CI
* Restructure dev/tools folder for testing
* Fix license headers
* Relicense Shrine demo to match rest of repository
The Shrine demo was Apache-licensed. The code was mostly
Google-written, with contributions from:
- Michelle Dudley (@michdud)
- Abhijeeth Padarthi <rkinabhi@gmail.com> (@rkinabhi)
- @a14n
I contacted all three, and they confirmed their approval for this
change, as described below.
Abhijeeth Padarthi said by e-mail on Thu, Nov 21, 2019 at 5:48 PM:
> hi Ian,
>
> sure :)
>
> let me know if I need to do anything on my end..
Michelle Dudley wrote by e-mail on Sun, Nov 24, 2019 at 2:07 PM:
> Hi Ian,
>
> That would be ok with me.
>
> Thanks,
>
> Michelle
@a14n said on Discord's Flutter server in the #hackers channel at 10:44PM on Thursday, November 21, 2019:
> @Hixie no problem I agree with this relicensing
* Remove shrine loophole from license checker.
* Update project.pbxproj files to say Flutter rather than Chromium
Also, the templates now have an empty organization so that we don't cause people to give their apps a Flutter copyright.
* Update the copyright notice checker to require a standard notice on all files
* Update copyrights on Dart files. (This was a mechanical commit.)
* Fix weird license headers on Dart files that deviate from our conventions; relicense Shrine.
Some were already marked "The Flutter Authors", not clear why. Their
dates have been normalized. Some were missing the blank line after the
license. Some were randomly different in trivial ways for no apparent
reason (e.g. missing the trailing period).
* Clean up the copyrights in non-Dart files. (Manual edits.)
Also, make sure templates don't have copyrights.
* Fix some more ORGANIZATIONNAMEs
This fixes the sample code analysis to treat dartpad snippets in the same way as snippet snippets, which it wasn't until now (the snippet generator was treating them as "samples"), and some errors crept in. This PR also fixes those errors.
Also, added a --verbose option to the sample analyzer.
