flutter/packages/flutter_tools/test/general.shard/isolated/native_assets_test.dart
Daco Harkes 4aa2caef20
[native assets] Create NativeAssetsManifest.json instead of kernel embedding (#159322)
This PR introduces a `NativeAssetsManifest.json` next to the
`AssetManifest.bin` and `FontManifest.json`. This removes the need for
embedding the native assets mapping inside the kernel file and enables
decoupling native assets building and bundling from the kernel
compilation in flutter tools. This means `flutter run` no longer does a
dry run of `hook/build.dart` hooks.

(It also means all isolate groups will have the same native assets.
However, since Flutter does not support `Isolate.spawnUri` from kernel
files, this is not a regression.)

Implementation details:

* g3 is still using kernel embedding.
https://github.com/flutter/flutter/pull/142016 introduced an argument to
embed a `native_assets.yaml` inside `flutter attach` and `flutter run`
(the outer flutter process), but it is not used in `flutter assemble`
(the inner process when doing `flutter run`). So, those arguments need
to still be respected. However, all other logic related to embedding a
yaml encoding in the kernel file has been removed.
* All dry-run logic has been removed. 🎉 
* The `KernelSnapshot` target no longer depends on the
`InstallCodeAssets` target. Instead, the various OS-specific
"BundleAsset" targets now depend on the `InstallCodeAssets` target. The
`InstallCodeAssets` invokes the build hooks and produces the
`NativeAssetsManifest.json`. The various "BundleAsset" commands
synchronize the `NativeAssetsManifest.json` to the app bundle.
* `InstallCodeAssets` produces a `native_assets.json`, which is renamed
to `NativeAssetsManifest.json` in the various "Bundle" targets. This
means that all unit tests of the "Bundle" targets now need to create
this file. (Similar to how `app.dill` is expected to exist because
`KernelSnapshot` is a dependency of the "Bundle" targets.)
* Because dynamic libraries need to be code signed (at least on iOS and
MacOS), the bundling of the dylibs is _not_ migrated to reuse
`_updateDevFS` (which is used for ordinary assets). Only the 2nd and 3rd
invocation of `flutter assemble` from `xcodebuild` has access to the
code signing identity.

Relevant tests:

* test/integration.shard/isolated/native_assets_test.dart - runs
`flutter run` with native assets including hot restart and hot reload.

TODO:

* Undo engine-roll in this PR after engine has rolled in.

Issue:

* https://github.com/flutter/flutter/issues/154425

Related PRs:

* https://dart-review.googlesource.com/c/sdk/+/388161
* https://github.com/flutter/engine/pull/56727

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
2024-11-27 11:03:19 +00:00

285 lines
11 KiB
Dart

// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'package:file/file.dart';
import 'package:file/memory.dart';
import 'package:file_testing/file_testing.dart';
import 'package:flutter_tools/src/artifacts.dart';
import 'package:flutter_tools/src/base/file_system.dart';
import 'package:flutter_tools/src/base/logger.dart';
import 'package:flutter_tools/src/build_info.dart';
import 'package:flutter_tools/src/build_system/build_system.dart';
import 'package:flutter_tools/src/build_system/targets/native_assets.dart';
import 'package:flutter_tools/src/features.dart';
import 'package:flutter_tools/src/globals.dart' as globals;
import 'package:flutter_tools/src/isolated/native_assets/native_assets.dart';
import 'package:native_assets_cli/code_assets_builder.dart' hide BuildMode;
import 'package:package_config/package_config_types.dart';
import '../../src/common.dart';
import '../../src/context.dart';
import '../../src/fakes.dart';
import 'fake_native_assets_build_runner.dart';
void main() {
late FakeProcessManager processManager;
late Environment environment;
late Artifacts artifacts;
late FileSystem fileSystem;
late BufferLogger logger;
late Uri projectUri;
setUp(() {
processManager = FakeProcessManager.empty();
logger = BufferLogger.test();
artifacts = Artifacts.test();
fileSystem = MemoryFileSystem.test();
environment = Environment.test(
fileSystem.currentDirectory,
inputs: <String, String>{},
artifacts: artifacts,
processManager: processManager,
fileSystem: fileSystem,
logger: logger,
);
environment.buildDir.createSync(recursive: true);
projectUri = environment.projectDir.uri;
});
testUsingContext('build with no package config', overrides: <Type, Generator>{
ProcessManager: () => FakeProcessManager.empty(),
}, () async {
await runFlutterSpecificDartBuild(
environmentDefines: <String, String>{
kBuildMode: BuildMode.debug.cliName,
},
targetPlatform: TargetPlatform.windows_x64,
projectUri: projectUri,
fileSystem: fileSystem,
buildRunner: FakeFlutterNativeAssetsBuildRunner(
hasPackageConfigResult: false,
),
);
expect(
(globals.logger as BufferLogger).traceText,
contains('No package config found. Skipping native assets compilation.'),
);
});
testUsingContext('Native assets: non-bundled libraries require no copying', overrides: <Type, Generator>{
FeatureFlags: () => TestFeatureFlags(isNativeAssetsEnabled: true),
ProcessManager: () => FakeProcessManager.empty(),
}, () async {
final File packageConfig =
environment.projectDir.childFile('.dart_tool/package_config.json');
final Uri nonFlutterTesterAssetUri = environment.buildDir.childFile('native_assets.json').uri;
await packageConfig.parent.create();
await packageConfig.create();
final File directSoFile = environment.projectDir.childFile('direct.so');
directSoFile.writeAsBytesSync(<int>[]);
CodeAsset makeCodeAsset(String name, LinkMode linkMode, [Uri? file])
=> CodeAsset(
package: 'bar',
name: name,
linkMode: linkMode,
os: OS.linux,
architecture: Architecture.x64,
file: file,
);
final Map<String, String> environmentDefines = <String, String>{
kBuildMode: BuildMode.release.cliName,
};
final List<CodeAsset> codeAssets = <CodeAsset>[
makeCodeAsset('malloc', LookupInProcess()),
makeCodeAsset('free', LookupInExecutable()),
makeCodeAsset('draw', DynamicLoadingSystem(Uri.file('/usr/lib/skia.so'))),
];
final DartBuildResult dartBuildResult = await runFlutterSpecificDartBuild(
environmentDefines: environmentDefines,
targetPlatform: TargetPlatform.linux_x64,
projectUri: projectUri,
fileSystem: fileSystem,
buildRunner: FakeFlutterNativeAssetsBuildRunner(
packagesWithNativeAssetsResult: <Package>[
Package('bar', projectUri),
],
buildResult: FakeFlutterNativeAssetsBuilderResult.fromAssets(codeAssets: codeAssets),
linkResult: FakeFlutterNativeAssetsBuilderResult.fromAssets(codeAssets: codeAssets),
),
);
await installCodeAssets(
dartBuildResult: dartBuildResult,
environmentDefines: environmentDefines,
targetPlatform: TargetPlatform.windows_x64,
projectUri: projectUri,
fileSystem: fileSystem,
nativeAssetsFileUri: nonFlutterTesterAssetUri,
);
expect(testLogger.traceText, isNot(contains('Copying native assets to')));
});
testUsingContext('build with assets but not enabled', overrides: <Type, Generator>{
ProcessManager: () => FakeProcessManager.empty(),
}, () async {
final File packageConfig = environment.projectDir.childFile('.dart_tool/package_config.json');
await packageConfig.parent.create();
await packageConfig.create();
expect(
() => runFlutterSpecificDartBuild(
environmentDefines: <String, String>{
kBuildMode: BuildMode.debug.cliName,
},
targetPlatform: TargetPlatform.windows_x64,
projectUri: projectUri,
fileSystem: fileSystem,
buildRunner: FakeFlutterNativeAssetsBuildRunner(
packagesWithNativeAssetsResult: <Package>[
Package('bar', projectUri),
],
),
),
throwsToolExit(
message: 'Package(s) bar require the native assets feature to be enabled. '
'Enable using `flutter config --enable-native-assets`.',
),
);
});
testUsingContext('build no assets', overrides: <Type, Generator>{
FeatureFlags: () => TestFeatureFlags(isNativeAssetsEnabled: true),
ProcessManager: () => FakeProcessManager.empty(),
}, () async {
final File packageConfig = environment.projectDir.childFile('.dart_tool/package_config.json');
final Uri nonFlutterTesterAssetUri = environment.buildDir.childFile(InstallCodeAssets.nativeAssetsFilename).uri;
await packageConfig.parent.create();
await packageConfig.create();
final Map<String, String> environmentDefines = <String, String>{
kBuildMode: BuildMode.debug.cliName,
};
final DartBuildResult dartBuildResult = await runFlutterSpecificDartBuild(
environmentDefines: environmentDefines,
targetPlatform: TargetPlatform.windows_x64,
projectUri: projectUri,
fileSystem: fileSystem,
buildRunner: FakeFlutterNativeAssetsBuildRunner(
packagesWithNativeAssetsResult: <Package>[
Package('bar', projectUri),
],
),
);
await installCodeAssets(
dartBuildResult: dartBuildResult,
environmentDefines: environmentDefines,
targetPlatform: TargetPlatform.windows_x64,
projectUri: projectUri,
fileSystem: fileSystem,
nativeAssetsFileUri: nonFlutterTesterAssetUri,
);
expect(
await fileSystem.file(nonFlutterTesterAssetUri).readAsString(),
isNot(contains('package:bar/bar.dart')),
);
expect(
environment.projectDir.childDirectory('build').childDirectory('native_assets').childDirectory('windows'),
exists,
);
});
testUsingContext('Native assets build error', overrides: <Type, Generator>{
FeatureFlags: () => TestFeatureFlags(isNativeAssetsEnabled: true),
ProcessManager: () => FakeProcessManager.empty(),
}, () async {
final File packageConfig =
environment.projectDir.childFile('.dart_tool/package_config.json');
await packageConfig.parent.create();
await packageConfig.create();
expect(
() => runFlutterSpecificDartBuild(
environmentDefines: <String, String>{
kBuildMode: BuildMode.debug.cliName,
},
targetPlatform: TargetPlatform.linux_x64,
projectUri: projectUri,
fileSystem: fileSystem,
buildRunner: FakeFlutterNativeAssetsBuildRunner(
packagesWithNativeAssetsResult: <Package>[
Package('bar', projectUri),
],
buildResult: null,
),
),
throwsToolExit(
message:
'Building native assets failed. See the logs for more details.',
),
);
});
testUsingContext('Native assets: no duplicate assets with linking', overrides: <Type, Generator>{
FeatureFlags: () => TestFeatureFlags(isNativeAssetsEnabled: true),
ProcessManager: () => FakeProcessManager.empty(),
}, () async {
final File packageConfig =
environment.projectDir.childFile('.dart_tool/package_config.json');
await packageConfig.parent.create();
await packageConfig.create();
final File directSoFile = environment.projectDir.childFile('direct.so');
directSoFile.writeAsBytesSync(<int>[]);
final File linkableAFile = environment.projectDir.childFile('linkable.a');
linkableAFile.writeAsBytesSync(<int>[]);
final File linkedSoFile = environment.projectDir.childFile('linked.so');
linkedSoFile.writeAsBytesSync(<int>[]);
CodeAsset makeCodeAsset(String name, Uri file, LinkMode linkMode)
=> CodeAsset(
package: 'bar',
name: name,
linkMode: linkMode,
os: OS.linux,
architecture: Architecture.x64,
file: file,
);
final DartBuildResult result = await runFlutterSpecificDartBuild(
environmentDefines: <String, String>{
// Release mode means the dart build has linking enabled.
kBuildMode: BuildMode.release.cliName,
},
targetPlatform: TargetPlatform.linux_x64,
projectUri: projectUri,
fileSystem: fileSystem,
buildRunner: FakeFlutterNativeAssetsBuildRunner(
packagesWithNativeAssetsResult: <Package>[
Package('bar', projectUri),
],
buildResult: FakeFlutterNativeAssetsBuilderResult.fromAssets(
codeAssets: <CodeAsset>[
makeCodeAsset('direct', directSoFile.uri, DynamicLoadingBundled()),
],
codeAssetsForLinking: <String, List<CodeAsset>>{
'package:bar' : <CodeAsset>[
makeCodeAsset('linkable', linkableAFile.uri, StaticLinking()),
],
},
),
linkResult: FakeFlutterNativeAssetsBuilderResult.fromAssets(
codeAssets: <CodeAsset>[
makeCodeAsset('direct', directSoFile.uri, DynamicLoadingBundled()),
makeCodeAsset('linked', linkedSoFile.uri, DynamicLoadingBundled()),
],
),
),
);
expect(
result.codeAssets.map((CodeAsset c) => c.file!.toString()).toList()..sort(),
<String>[directSoFile.uri.toString(), linkedSoFile.uri.toString()]);
});
}