Currently the `NativeAsset` target in flutter tools is responsible for
two things:
* performing the dart build (in the app as well as all transitive pub
dependencies)
* taking output (shared libraries) from this build and copying them
around
This intermingling of responsibilities leads to more complex code and
potentially unnecessary work: If the source code changed (e.g. `.c`
files change) we have to run the dart build again. But doing so may
result in the same shared libraries (e.g. adding comments to the `.c`
code). Currently we're going to copy the shared libraries despite them
having not changed, which then may cause upstream things to be dirtied
(if it's based on timestamp of files) and re-built.
Instead this PR splits this `NativeAsset` into the two orthogonal pieces
* `DartBuild` target that is responsible for the dart build
* `InstallCodeAssets` that is responsible for copying shared libraries
to the right place and producing a `native_assets.yaml`.
This decoupling is also preparation for a future where a dart build can
produce other kinds of assets (e.g. data assets) and is used in the web
build as well. (The web build would use `DartBuild` but not
`InstalCodeAssets`).
After the dart build is done, the flutter tool has to bundle the
produced shared libraries, which it does that by copying them around.
Though the code assumed that all code assets are shared libraries to be
bundled, whereas in fact one can have code assets without any actual
code (ones that are installed on the target system already or artificial
code assets whose symbols get resolved from executable / process).
=> Using non-bundled code assets currently results in null pointer
exceptions and/or cast errors.
=> We update the copy code to only operate on code assets that have a
shared library to bundle.
We also update the copy routines by removing copy&past'ed - but slightly
different - printing code into the shared caller function.
In release builds linking of native assets is enabled. The build step is
only a temprary step, it's output is given to the link step which then
returns all final assets (effectively a map-reduce system). Assets that
aren't sent to a specific linker could be conceptually viewed as sent to
a linker that will emit it's input as-is.
=> The code currently took output of build & link step and therefore
accumulated assets that aren't explicitly sent to a linker twice.
=> This led to performing work twice for those (e.g. copying them twice)
This PR changes this such that if linking mode is enabled, we only rely
on the output of the link phase.
That in return means many tests that mock the native asset builds need
to be updated to mock the output of the link phase.
Almost all of the code is just adopting to changes to the APIs of
`package:native_assets_builder`, `package:native_assets_cli` and
`package:native_toolchain_c`
There's only two semantic changes
* Removes a test that checks for a verification error if a build hook
produces a static library if the preferred linking mode is dynamic:
=> The test is written in a very hacky way. By monkey patching the build
config.json that flutter build actually made. This monkey patching
relies on package:cli_config which is now no longer used.
=> The actual code that checks for this mismatch lives in
dart-lang/native repository and is tested there. So there's really no
need to duplicate that.
* The `package:native_assets_builder` no longer knows about code assets.
This is something a user of that package (e.g. flutter tools) adds. Now
the dry-run functionality will invoke build hooks who produce code
assets without an architecture.
=> The `package:native_assets_builder` used to expand such a code asset
to N different code assets (one for each supported architecture)
=> This logic was now moved to flutter tools. => In the near future
we're going to this dry-run complexity, which will then also get rid of
this uglyness (of expanding to all archs of an OS).
Changes to original CL: The code that issues an error on unsupported
operating system in the dry-run case was missing a case for iOS and
Android
Original CL description
tl;dr Removes 50% (>1650 locs) of native asset related code in
`packages/flutter_tools`
Before this PR the invocation of dart build/link/dry-run was implemented
per OS. This lead to very large code duplication of almost identical,
but slightly different code. It also led to similarly duplicated test
code.
Almost the entire dart build/link/dry-run implementation is identical
across OSes. There's small variations:
- configuration of the build (e.g. android/macos/ios version, ios sdk,
...)
- determining target locations & copying the final shared libraries
This PR unifies the implementation by reducing the code to basically two
main functions:
* `runFlutterSpecificDartBuild` which is responsible for
- obtain flutter configuration
- perform dart build (& link)
- determine target location & install binaries
* `runFlutterSpecificDartDryRunOnPlatforms` which is responsible for a
similar (but not same):
- obtain flutter configuration
- perform dart dry run
- determine target location
these two functions will call out to helpers for the OS specific
functionality:
* `_assetTargetLocationsForOS` for determining the location of the code
assets
* `_copyNativeCodeAssetsForOS` for copying the code assets (and possibly
overriting the install name, etc)
=> Since we get rid of the code duplication across OSes and have only a
single code path for the build/link/dry-run, we can also remove the
duplicated tests that were pretty much identical across OSes.
We also harden the building code by adding asserts, e.g.
* the dry fun functionality should never be used by `flutter test`
* the `build/native_assets/<os>/native_assets.yaml` should only be used
by `flutter test` and the dry-run of `flutter run`
=> We change the tests to also comply with these invariants (so the
tests are not testing things that cannot happen in reality)
We also rename `{,Flutter}NativeAssetsBuildRunner` to disambiguate it
from the `package:native_asset_builder`'s `NativeAssetsBuildRunner`.
Reverts: flutter/flutter#155430
Initiated by: eyebrowsoffire
Reason for reverting: Postsubmit failures closing the tree. See the following examples:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_ios%20native_assets_ios/5738/overviewhttps://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_mokey%20native_assets_android/583/overviewhttps://ci.chromium.org/ui/p/flutter/builders/prod/Linux_pixel_7pro%20native_assets_android/4075/overviewhttps://ci.chromium.org/u
Original PR Author: mkustermann
Reviewed By: {bkonyi, dcharkes}
This change reverts the following previous change:
tl;dr Removes 50% (>1650 locs) of native asset related code in `packages/flutter_tools`
Before this PR the invocation of dart build/link/dry-run was implemented per OS. This lead to very large code duplication of almost identical, but sligthly different code. It also led to similarly duplicated test code.
Almost the entire dart build/link/dry-run implementation is identical across OSes. There's small variations:
- configuration of the build (e.g. android/macos/ios version, ios sdk, ...)
- determining target locations & copying the final shared libraries
This PR unifies the implementation by reducing the code to basically two main functions:
* `runFlutterSpecificDartBuild` which is responsible for
- obtain flutter configuration
- perform dart build (& link)
- determine target location & install binaries
* `runFlutterSpecificDartDryRunOnPlatforms` which is responsible for a similar (but not same):
- obtain flutter configuration
- perform dart dry run
- determine target location
these two functions will call out to helpers for the OS specific functionality:
* `_assetTargetLocationsForOS` for determining the location of the code assets
* `_copyNativeCodeAssetsForOS` for copying the code assets (and possibly overriting the install name, etc)
=> Since we get rid of the code duplication across OSes and have only a single code path for the build/link/dry-run, we can also remove the duplicated tests that were pretty much identical across OSes.
We also harden the building code by adding asserts, e.g.
* the dry fun functionality should never be used by `flutter test`
* the `build/native_assets/<os>/native_assets.yaml` should only be used by `flutter test` and the dry-run of `flutter run`
=> We change the tests to also comply with these invariants (so the tests are not testing things that cannot happen in reality)
We also rename `{,Flutter}NativeAssetsBuildRunner` to disambiguate it from the `package:native_asset_builder`'s `NativeAssetsBuildRunner`.
We also reorganize the main code to make it readable from top-down and make members private where they can be.
tl;dr Removes 50% (>1650 locs) of native asset related code in
`packages/flutter_tools`
Before this PR the invocation of dart build/link/dry-run was implemented
per OS. This lead to very large code duplication of almost identical,
but slightly different code. It also led to similarly duplicated test
code.
Almost the entire dart build/link/dry-run implementation is identical
across OSes. There's small variations:
- configuration of the build (e.g. android/macos/ios version, ios sdk, ...)
- determining target locations & copying the final shared libraries
This PR unifies the implementation by reducing the code to basically two
main functions:
* `runFlutterSpecificDartBuild` which is responsible for
- obtain flutter configuration
- perform dart build (& link)
- determine target location & install binaries
* `runFlutterSpecificDartDryRunOnPlatforms` which is responsible for a
similar (but not same):
- obtain flutter configuration
- perform dart dry run
- determine target location
these two functions will call out to helpers for the OS specific
functionality:
* `_assetTargetLocationsForOS` for determining the location of the code
assets
* `_copyNativeCodeAssetsForOS` for copying the code assets (and possibly
overriting the install name, etc)
=> Since we get rid of the code duplication across OSes and have only a
single code path for the build/link/dry-run, we can also remove the
duplicated tests that were pretty much identical across OSes.
We also harden the building code by adding asserts, e.g.
* the dry fun functionality should never be used by `flutter test`
* the `build/native_assets/<os>/native_assets.yaml` should only be used
by `flutter test` and the dry-run of `flutter run`
=> We change the tests to also comply with these invariants (so the
tests are not testing things that cannot happen in reality)
We also rename `{,Flutter}NativeAssetsBuildRunner` to disambiguate it
from the `package:native_asset_builder`'s `NativeAssetsBuildRunner`.
