https://github.com/dart-lang/webdev/issues/2516https://github.com/dart-lang/webdev/issues/2561
- Adds logic to emit newly compiled sources to a file that can be read
by the bootstrapper.
- Adds bootstrapping logic to reload scripts as needed. This involves
implementing the necessary hot restart callback, fetching and processing
the emitted file of newly compiled sources, cache busting, and loading
the scripts onto the page. A lot of this logic is similar or identical
to what we have for internal hot restart support.
- Runs existing hot restart tests with the new bundle format.
- Adds meta tag to run with utf-8 like in
https://github.com/flutter/flutter/pull/161493.
- Uses DWDS 24.3.3.
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [X] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.
This gets us closer to enabling [hot
reload](https://github.com/dart-lang/sdk/issues/54934) on the web as
this format is a prerequisite. Historically, we added support for the
DDC module format only to enable hot reload, but that format is not
feasible for the goal, so we added the DDC library bundle format. The
DDC library bundle format is currently represented as the combination of
the `ddc` module format and `canary`. We no longer need to support the
old DDC module format.
- Adds build artifacts to build the SDKs for this format (but only in
sound mode as unsound is unsupported), and removes said artifacts for
the DDC module format.
- Update artifact maps and constants to add the new format and remove
the old format.
- Adds handling of the `canaryFeatures` flag.
- Update dwds to 24.3.0 and use the new
`FrontendServerDdcLibraryBundleStrategyProvider`.
- Add bootstrap code for the new format. Kept DDC module format
bootstrap code as it's used internally.
- Updates tests.
I ran `spinning_square` with the new module format to verify that it can
run.
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [X] All existing and new tests are passing.
This auto-formats all *.dart files in the repository outside of the
`engine` subdirectory and enforces that these files stay formatted with
a presubmit check.
**Reviewers:** Please carefully review all the commits except for the
one titled "formatted". The "formatted" commit was auto-generated by
running `dev/tools/format.sh -a -f`. The other commits were hand-crafted
to prepare the repo for the formatting change. I recommend reviewing the
commits one-by-one via the "Commits" tab and avoiding Github's "Files
changed" tab as it will likely slow down your browser because of the
size of this PR.
---------
Co-authored-by: Kate Lovett <katelovett@google.com>
Co-authored-by: LongCatIsLooong <31859944+LongCatIsLooong@users.noreply.github.com>
Part of https://github.com/flutter/flutter/issues/160043, makes it
easier to add https://github.com/flutter/flutter/pull/160131.
This PR has no functional changes to any of the code, but does refactor
both the code and tests:
- Makes a number of always non-null but not migrated to non-null
properties, well, not-null
- Creates two concrete methods (`update` and `compare` versus a
positional nullable boolean)
- Uses type signatures instead of `String?` to explain the possible
results of the methods
- Renames the mysterious `shellPath` variable to `flutterTesterBinPath`
- Expands and rewrites internally-facing doc comments
- Moves `WebRenderer` environment variable setting to
`flutter_web_platform.dart`
- Makes the tests have less duplication, and check for update/compare
cases
After this PR, I can use it in the non-web branch of the Flutter tool
without any hacks or TODOS :)
/cc @eyebrowsoffire (trivial web refactoring), @camsim99 (changes being
made to tool).
Closes https://github.com/flutter/flutter/issues/158012.
This is (effectively) a user-facing NOP, which is exchanging an
on-by-default command-line argument (`--implicit-pubspec-resolution`)
for an off-by-default global feature flag
(`explicit-package-dependencies`). It matches the mental model better,
is less painstaking to maintain and feed throughout, and will be easier
to globally flip on/off in a future PR.
---------
Co-authored-by: Andrew Kolos <andrewrkolos@gmail.com>
Work towards https://github.com/flutter/flutter/issues/56591.
I explicitly want an LGTM from @andrewkolos @jmagman @jonahwilliams before merging.
---
After this PR, `<Plugin>.isDevDependency` is resolved based on the following logic, IFF:
- The plugin comes from a package _A_ listed in the app's package's `dev_dependencies: ...`
- The package _A_ is not a normal dependency of any transitive non-dev dependency of the app
See [`compute_dev_dependencies_test.dart`](51676093a3/packages/flutter_tools/test/general.shard/compute_dev_dependencies_test.dart) for probably the best specification of this behavior.
We (still) do not write the property to disk (i.e. it never makes it to `.flutter-plugins-dependencies`), so there is no impact to build artifacts at this time; that would come in a follow-up PR (and then follow-up follow-up PRs for the various build systems in both Gradle and Xcode to actually use that value to omit dependencies).
Some tests had to be updated; for the most part it was updating the default `ProcessManager` because a call to `dart pub deps --json` is now made in code that computes what plugins are available, but there should be no change in behavior.
_/cc @jonasfj @sigurdm for FYI only (we talked on an internal thread about this; see https://github.com/dart-lang/sdk/issues/56968)._
_/cc @camsim99 @cbracken @johnmccutchan for visibility on the change._
Work towards https://github.com/flutter/flutter/issues/157819. **No behavior changes as a result of this PR**.
Based on a proof of concept by @jonahwilliams (https://github.com/flutter/flutter/pull/157818).
The existence of this flag (which for the time being, defaults to `true`) implies the following:
1. The (legacy, deprecated) `.flutter-plugins` file is not generated:
https://docs.flutter.dev/release/breaking-changes/flutter-plugins-configuration
2. The (legacy, deprecated) `package:flutter_gen` is not synthetically generated:
https://github.com/flutter/website/pull/11343
(awaiting website approvers, but owners approve this change)
This change creates `useImplicitPubspecResolution` and plumbs it through as a required variable, parsing it from a `FlutterCommand.globalResults` where able. In tests, I've defaulted the value to `true` 100% of the time - except for places where the value itself is acted on directly, in which case there are true and false test-cases (e.g. localization and i10n based classes and functions).
I'm not extremely happy this needed to change 50+ files, but is sort of a result of how inter-connected many of the elements of the tools are. I believe keeping this as an explicit (flagged) argument will be our best way to ensure the default behavior changes consistently and that tests are running as expected.
There have been various requests for this for a while. See https://github.com/flutter/flutter/issues/96283 for an example. This has become more important with dart2wasm builds in the mix, as the profiling versions of the dart2wasm builds are a lot less debuggable than the debug versions. Most of this is already wired up so it just requires taking out a few explicit checks for it and making sure that we compile with the right optimization levels when compiling debug.
Fix JS compilation to use the command 'compile js' instead of using
snapshot names to invoke dart2js
The Dart SDK is switching all tools to use AOT snapshots instead of JIT
snapshots and using snapshot names directly causes things to break.
This pull request adds a local function `runInTestbed()` to **devfs_web_ddc_modules_test.dart**, which wraps the `testbed.run()` method. Several whitespace adjustments have been made as well.
<br>
It's much easier to read after clicking "hide whitespace".
<br>
Relevant style guidelines:
- Prefer avoiding line breaks after assignment operators.
- If you have a newline after some opening punctuation, match it on the closing punctuation.
- Only use `=>` when everything, including the function declaration, fits on a single line.
If the user specifies the `--no-web-resources-cdn` or `--local-web-sdk`, we should use the local version of CanvasKit. `flutter.js` now has a flag that can be specified in the build configuration that tells it to load locally instead.
Also, added a link to the relevant docs in the web template warnings.
This addresses https://github.com/flutter/flutter/issues/148713
Also fixes https://github.com/flutter/flutter/issues/145559
- When `--web-renderer` is omitted, keep the value `null` until it later materializes to either `canvaskit` or `skwasm`.
- No more hardcoded defaults anywhere. We use `WebRendererMode.defaultForJs/defaultForWasm` instead.
- When in `--wasm` mode, the JS fallback is now `canvaskit` instead of `auto`.
- Add test for defaulting to `skwasm` when `--wasm` is enabled.
Fixes https://github.com/flutter/flutter/issues/149826
This adds support for adding the `--wasm` flag to `flutter run` and `flutter drive`
* Emits errors if you attempt to use the skwasm renderer without the `--wasm` flag
* Emits errors if you try to use `--wasm` when not using a web device
* Uses the skwasm renderer by default if you pass `--wasm` and no `--web-renderer`
* Adds support for `flutter test --wasm`.
* The test compilation flow is a bit different now, so that it supports compilers other than DDC. Specifically, when we run a set of unit tests, we generate a "switchboard" main function that imports each unit test and runs the main function for a specific one based off of a value set by the JS bootstrapping code. This way, there is one compile step and the same compile output is invoked for each unit test file.
* Also, removes all references to `dart:html` from flutter/flutter.
* Adds CI steps for running the framework unit tests with dart2wasm+skwasm
* These steps are marked as `bringup: true`, so we don't know what kind of failures they will result in. Any failures they have will not block the tree at all yet while we're still in `bringup: true`. Once this PR is merged, I plan on looking at any failures and either fixing them or disabling them so we can get these CI steps running on presubmit.
This fixes https://github.com/flutter/flutter/issues/126692
This makes several changes to flutter web app bootstrapping.
* The build now produces a `flutter_bootstrap.js` file.
* By default, this file does the basic streamlined startup of a flutter app with the service worker settings and no user configuration.
* The user can also put a `flutter_bootstrap.js` file in the `web` subdirectory in the project directory which can have whatever custom bootstrapping logic they'd like to write instead. This file is also templated, and can use any of the tokens that can be used with the `index.html` (with the exception of `{{flutter_bootstrap_js}}`, see below).
* Introduced a few new templating tokens for `index.html`:
* `{{flutter_js}}` => inlines the entirety of `flutter.js`
* `{{flutter_service_worker_version}}` => replaced directly by the service worker version. This can be used instead of the script that sets the `serviceWorkerVersion` local variable that we used to have by default.
* `{{flutter_bootstrap_js}}` => inlines the entirety of `flutter_bootstrap.js` (this token obviously doesn't apply to `flutter_bootstrap.js` itself).
* Changed `IndexHtml` to be called `WebTemplate` instead, since it is used for more than just the index.html now.
* We now emit warnings at build time for certain deprecated flows:
* Warn on the old service worker version pattern (i.e.`(const|var) serviceWorkerVersion = null`) and recommends using `{{flutter_service_worker_version}}` token instead
* Warn on use of `FlutterLoader.loadEntrypoint` and recommend using `FlutterLoader.load` instead
* Warn on manual loading of `flutter_service_worker.js`.
* The default `index.html` on `flutter create` now uses an async script tag with `flutter_bootstrap.js`.
Partial implementation of https://github.com/flutter/flutter/issues/143348
This enables asset transformation during hot reload (except for web, because that has its own implementation of `DevFS` ð). Asset transformers will be reapplied after changing any asset and performing a hot reload during `flutter run`.
### Context:
DDC modules are abstractions over how libraries are loaded/updated. The entirety of google3 uses the DDC/legacy module system due to its flexibility extensibility over the other two (ES6 and AMD/RequireJS). Unifying DDC's module system saves us from duplicating work and will allow us to have finer grained control over how JS modules are loaded. This is a a prerequisite to features such as hot reload.
### Overview:
This change plumbs a boolean flag through flutter_tools that switches between DDC (new) and AMD (current) modules. This mode is automatically applied when `--extra-front-end-options=--dartdevc-module-format=ddc` is specified alongside `flutter run`. Other important additions include:
* Splitting Flutter artifacts between DDC and AMD modules
* Adding unit tests for the DDC module system
* Additional bootstrapper logic for the DDC module system
We don't expect to see any user-visible behavior or performance differences.
This is dependent on [incoming module system support in DWDS](https://github.com/dart-lang/webdev/pull/2295) and [additional artifacts in the engine](https://github.com/flutter/engine/pull/47783).
This is part of a greater effort to deprecate the AMD module system: https://github.com/dart-lang/sdk/issues/52361
Update: Accidentally use `--O4` instead of `-O4` in `dev/devicelab/lib/tasks/web_benchmarks.dart` update.
Original description:
* Make `flutter build web` have one option that determins the
optimization level: `-O<level>` / `--optimization-level=<level>` =>
Defaulting to -O4 => Will apply to both dart2js and dart2wasm
* Deprecate `--dart2js-optimization=O<level>`
* Disentagle concept of optimization from concept of static symbols =>
Add a `--strip-wasm` / `--no-strip-wasm` flag that determins whether
static symbols are kept in the resulting wasm file.
* Remove copy&past'ed code in the tests for wasm build tests
* Cleanup some artifacts code, now that we no longer use `wasm-opt`
inside flutter tools
Reverts flutter/flutter#143517
Initiated by: dnfield
Reason for reverting: broke CI, see https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_benchmarks_skwasm/3446/overview
Original PR Author: mkustermann
Reviewed By: {eyebrowsoffire}
This change reverts the following previous change:
Original Description:
* Make `flutter build web` have one option that determins the optimization level: `-O<level>` / `--optimization-level=<level>` => Defaulting to -O4 => Will apply to both dart2js and dart2wasm
* Deprecate `--dart2js-optimization=O<level>`
* Disentagle concept of optimization from concept of static symbols => Add a `--strip-wasm` / `--no-strip-wasm` flag that determins whether static symbols are kept in the resulting wasm file.
* Remove copy&past'ed code in the tests for wasm build tests
* Cleanup some artifacts code, now that we no longer use `wasm-opt` inside flutter tools
* Make `flutter build web` have one option that determins the
optimization level: `-O<level>` / `--optimization-level=<level>` =>
Defaulting to -O4 => Will apply to both dart2js and dart2wasm
* Deprecate `--dart2js-optimization=O<level>`
* Disentagle concept of optimization from concept of static symbols =>
Add a `--strip-wasm` / `--no-strip-wasm` flag that determins whether
static symbols are kept in the resulting wasm file.
* Remove copy&past'ed code in the tests for wasm build tests
* Cleanup some artifacts code, now that we no longer use `wasm-opt`
inside flutter tools
This is an attempt at a reland of https://github.com/flutter/flutter/pull/141396
The main changes here that are different than the original PR is fixes to wire up the `flutter test` command properly with the web renderer.
Dual Web Compile has had some issues where `flutter test` is not respecting the `--web-renderer` flag for some reason. I haven't gotten entirely to the bottom of the issue, but for now we need to rever these changes while I investigate. This reverts the following PRs:
https://github.com/flutter/flutter/pull/143128https://github.com/flutter/flutter/pull/141396
While doing this revert, I had a few merge conflicts with https://github.com/flutter/flutter/pull/142760, and I tried to resolve the merge conflicts within the spirit of that PR's change, but @chingjun I might need your input on whether the imports I have modified are okay with regards to the change you were making.
Add a new `BuildTargets` class that provides commonly used build targets. And avoid importing files from `build_system/targets` except from the top level entrypoints or from top level commands.
Also move `scene_importer.dart` and `shader_compiler.dart` into `build_system/tools` because they are not `Target` classes, but wrapper for certain tools.
With this change, we can ignore all files in `build_system/targets` internally and make PR #142709 easier to land internally. See cl/603434066 for the corresponding internal change.
Related to:
https://github.com/flutter/flutter/pull/142709https://github.com/flutter/flutter/issues/142041
Also note that I have opted to add a new variable in `globals.dart` for `BuildTargets` in this PR, but I know that we are trying to get rid of globals. Several alternatives that I was considering:
1. Add a new field in `BuildSystem` that returns a `BuildTargets` instance. Since `BuildSystem` is already in `globals`, we can access build targets using `globals.buildSystem.buildTargets` without adding a new global variable.
2. Properly inject the `BuildTargetsImpl` instance from the top level `executable.dart` and top level commands.
Let me know if you want me to do one of the above instead. Thanks!
This implements dual compile via the newly available flutter.js bootstrapping APIs for intelligent build fallback.
* Users can now use the `FlutterLoader.load` API from flutter.js
* Flutter tool injects build info into the `index.html` of the user so that the bootstrapper knows which build variants are available to bootstrap
* The semantics of the `--wasm` flag for `flutter build web` have changed:
- Instead of producing a separate `build/web_wasm` directory, the output goes to the `build/web` directory like a normal web build
- Produces a dual build that contains two build variants: dart2wasm+skwasm and dart2js+CanvasKit. The dart2wasm+skwasm will only work on Chrome in a cross-origin isolated context, all other environments will fall back to dart2js+CanvasKit.
- `--wasm` and `--web-renderer` are now mutually exclusive. Since there are multiple build variants with `--wasm`, the web renderer cannot be expressed via a single command-line flag. For now, we are hard coding what build variants are produced with the `--wasm` flag, but I plan on making this more customizable in the future.
* Build targets now can optionally provide a "build key" which can uniquely identify any specific parameterization of that build target. This way, the build target can invalidate itself by changing its build key. This works a bit better than just stuffing everything into the environment defines because (a) it doesn't invalidate the entire build, just the targets which are affected and (b) settings for multiple build variants don't translate well to the flat map of environment defines.
Refactors `ShaderTarget` to make it opaque as to whether it's using Impeller or SkSL and instead has it focus on the target platform it's generating for.
ImpellerC includes SkSL right now whether you ask for it or not.
The tester target also might need SkSL or Vulkan depending on whether `--enable-impeller` is passed.
Fixes https://github.com/flutter/flutter/issues/130277
This PR does two things:
1. introduce a hidden `flutter build _preview` command, that will build a debug windows desktop app and copy it into the SDK's binary cache. This command is only intended to be run during packaging.
2. introduce a new device type, called `PreviewDevice`, which relies on the prebuilt desktop debug app from step 1, copies it into the target app's assets build folder, and then hot reloads their dart code into it.
Adds support for a new --web-header option to flutter run.
Creates a workaround for https://github.com/flutter/flutter/issues/127902
This PR allows adding additional headers for the flutter run web server. This is useful to add headers like Cross-Origin-Embedder-Policy and Cross-Origin-Opener-Policy without the use of a proxy server. These headers are required enable advanced web features. This approach provides flexibility to the developer to make use of the feature as they see fit and is backward-compatible. One tradeoff is that it increases the surface area to support for future changes to the flutter web server.
https://github.com/flutter/flutter/issues/127902 is not fully addressed by this change. The solution for that task will be more opinionated. This PR creates a general-purpose workaround for anyone who needs a solution sooner while the bigger solution is developed.
Support for FFI calls with `@Native external` functions through Native assets on MacOS and iOS. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for MacOS and iOS.
Dylibs are bundled by (1) making them fat binaries if multiple architectures are targeted, (2) code signing these, and (3) copying them to the frameworks folder. These steps are done manual rather than via CocoaPods. CocoaPods would have done the same steps, but (a) needs the dylibs to be there before the `xcodebuild` invocation (we could trick it, by having a minimal dylib in the place and replace it during the build process, that works), and (b) can't deal with having no dylibs to be bundled (we'd have to bundle a dummy dylib or include some dummy C code in the build file).
The dylibs are build as a new target inside flutter assemble, as that is the moment we know what build-mode and architecture to target.
The mapping from asset id to dylib-path is passed in to every kernel compilation path. The interesting case is hot-restart where the initial kernel file is compiled by the "inner" flutter assemble, while after hot restart the "outer" flutter run compiled kernel file is pushed to the device. Both kernel files need to contain the mapping. The "inner" flutter assemble gets its mapping from the NativeAssets target which builds the native assets. The "outer" flutter run get its mapping from a dry-run invocation. Since this hot restart can be used for multiple target devices (`flutter run -d all`) it contains the mapping for all known targets.
### Example vs template
The PR includes a new template that uses the new native assets in a package and has an app importing that. Separate discussion in: https://github.com/flutter/flutter/issues/131209.
### Tests
This PR adds new tests to cover the various use cases.
* dev/devicelab/bin/tasks/native_assets_ios.dart
* Runs an example app with native assets in all build modes, doing hot reload and hot restart in debug mode.
* dev/devicelab/bin/tasks/native_assets_ios_simulator.dart
* Runs an example app with native assets, doing hot reload and hot restart.
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
* Runs (incl hot reload/hot restart), builds, builds frameworks for iOS, MacOS and flutter-tester.
* packages/flutter_tools/test/general.shard/build_system/targets/native_assets_test.dart
* Unit tests the new Target in the backend.
* packages/flutter_tools/test/general.shard/ios/native_assets_test.dart
* packages/flutter_tools/test/general.shard/macos/native_assets_test.dart
* Unit tests the native assets being packaged on a iOS/MacOS build.
It also extends various existing tests:
* dev/devicelab/bin/tasks/module_test_ios.dart
* Exercises the add2app scenario.
* packages/flutter_tools/test/general.shard/features_test.dart
* Unit test the new feature flag.
Flutter web uses requireJS in `debug` mode to assemble a DDC-compiled app from a bunch of small files ("modules").
This caused that `canvaskit.js` (and all other modules that used a browserify-like loading header) didn't work because they attempted to use the `define` function provided by Flutter's instance of `requireJS` (which kept the defined modules private, rather than as globals on the page, as the users of the JS expected).
A [fix](https://github.com/flutter/engine/pull/27342) was added to `flutter/engine` to trick loaders into *not* using the `requireJS` module loader, but a recent change in the fix's js-interop layer *subtly* changed its JS output on the page (objects went from `undefined` to `null`), causing this:
* https://github.com/flutter/flutter/issues/126131 (and others)
This PR hides a bit of code that is commonly used by module loaders to decide that they may use the `define` function provided by requireJS (so the engine workaround can be removed).
## Next steps
* https://github.com/flutter/engine/pull/42941
## Issues
Partially addresses: https://github.com/flutter/flutter/issues/126131 (and others)
## Tests
* Added a unit test to ensure the `delete` stays
* Manually tested with the Gallery app in `debug` mode with a bunch of user-supplied scripts that currently fail to load.
* Also tested hot restart as suggested by @nshahan
