cline/scripts/generate-server-setup.mjs
Sarah Fortune 8c565b5a7c
Run the cline extension as a standalone process outside of vscode. (#3535)
* Add standalone cline server.

Add directory standalone/ with the scripts to generate
a cline instance that runs a gRPC service for the proto bus.

* Rm unused dependencies

* Build standalone extension

Build stubs for the whole vscode SDK.

Import extension.js instead of putting everything in one file.

Move all the files the extension needs at runtime in files/
  Use local packages for vscode and stub-utils instead of module alias.
  Move vscode-impls into the vscode module.
  Create separate package.json for the standalone extension in files/.

* Handlers for gRPC requests

Add code to the bottom of extension.js to export the gRPC handlers.
Add a wrapper to the handlers to catch and log extensions, otherwise the whole server process fails.
Fix use of open module.

* Standalone gRPC server

Export handers from the extension.
Add reflection and healthcheck to the server.
Add vscode launch file for standalone server.

* Fix formatting

* Better error handling in the server template.

Exit if the server could not bind to the port.
Use internal error code if exception is thrown.

* Formatting

* Stop using google-protobuf npm module to generate JS for protos

The code generated by google-protobuf cannot serialize protos from plain objects. It needs the protos to be class instances created with ProtoExample.create().
But, the protos created in the extension are just POJOs.
Use protoLoader instead which is fine with plain objects.
Protoloader is also the method used in the grpc JS documentation: https://grpc.io/docs/languages/node/basics/#loading-service-descriptors-from-proto-files

* Rm proto that was removed in cline/cline

* Rm old protos when building standalone extension.

* Log gRPC requests

* feat(standalone): implement TypeScript gRPC-based standalone extension

The major improvement is that the gRPC implementation is now written in TypeScript instead of JavaScript, and the standalone extension is compiled together with the original extension rather than using the compiled JS output. This provides full type safety throughout the codebase and prevents issues with the TypeScript compiler renaming handlers during compilation, making the system more robust and maintainable.

- Add new standalone implementation files in src/standalone/ directory using TypeScript
- Implement gRPC server setup in extension-standalone.ts with full type safety
- Generate server setup code with service registrations
- Update build script to support the new standalone architecture
- Reorganize runtime files from standalone/files/ to standalone/runtime-files/
- Replace template-based server generation with gRPC service registration

* Fix issues when doing clean build

Use correct build dir in esbuild.js
Remove undefined type.

* Add handler for gRPC methods with streaming response.

Add a handler-wrapper for rpc's with streaming responses.

Fix issue where grpc-js won't deserialize protos in camelcase. It is the default
for generated code for protos to use camelcase (keepCase: false), but I cannot find
where is being set for the proto serializations to keep the case. For now, just convert the
properties of the proto messages to snake case. This is not a good
solution, but trying to fix this is time sink.

* Formatting

* Add streaming response support to the script that generates setup-server.ts

Add types for the handlers.

* Formatting

* Fix case conversion for gRPC requset protos as well.

Convert snake case to camelcase for incoming request protos.

* formatting

* Improve build process / building for standalone extension

Add separate configs for the extension and the standalone in the esbuild config.
Modules that use __dirname to load files at runtime are marked as external in the build config.
Rename vscode-impls to vscode-context.
Remove unecessary files from the standalone runtime.

* Rename extension-standalone.js to standalone.js

* Move generate-server-setup script to protos dir.

Add the script the npm target `protos`, so it is run when the protos are regenerated.

* formatting

* Add a post build step for the npm run target `protos` to format the generated files.

* Move generate-server-setup to scripts directory

* Add a JS script to package the standalone build, replacing the shell script.

Add a post build step for the standalone target that:
    * copies the vscode module files into the output directory.
    * checks that native modules are not included in the output
    * creates a zip of the build.

* Rm files that were included from merge by mistake

* Move scripts from standalone in scripts directory

Remove unused package.json files from standalone/

* Update scripts and launch.json to use correct paths

* During build install external modules in the dist directory.

Add package.json for the distribution.
Set the node path for the vscode launch config.
Make the prettier silent during `npm run protos`

* Fix ellipsis suggestions
2025-05-15 12:04:46 -07:00

80 lines
2.6 KiB
JavaScript

import * as fs from "fs"
import * as grpc from "@grpc/grpc-js"
import * as protoLoader from "@grpc/proto-loader"
import * as health from "grpc-health-check"
import { fileURLToPath } from "url"
import path from "path"
const OUT_FILE = path.resolve("src/standalone/server-setup.ts")
const DESCRIPTOR_SET = path.resolve("dist-standalone/proto/descriptor_set.pb")
// Load service definitions.
const clineDef = protoLoader.loadFileDescriptorSetFromBuffer(fs.readFileSync(DESCRIPTOR_SET))
const healthDef = protoLoader.loadSync(health.protoPath)
const packageDefinition = { ...clineDef, ...healthDef }
const proto = grpc.loadPackageDefinition(packageDefinition)
/**
* Generate imports and function to add all the handlers to the server for all services defined in the proto files.
*/
function generateHandlersAndExports() {
let imports = []
let handlerSetup = []
for (const [name, def] of Object.entries(proto.cline)) {
if (!def || !("service" in def)) {
continue
}
const domain = name.replace(/Service$/, "")
const dir = domain.charAt(0).toLowerCase() + domain.slice(1)
imports.push(`// ${domain} Service`)
handlerSetup.push(` // ${domain} Service`)
handlerSetup.push(` server.addService(proto.cline.${name}.service, {`)
for (const [rpcName, rpc] of Object.entries(def.service)) {
imports.push(`import { ${rpcName} } from "../core/controller/${dir}/${rpcName}"`)
if (rpc.requestStream) {
throw new Error("Request streaming is not supported")
}
if (rpc.responseStream) {
handlerSetup.push(` ${rpcName}: wrapStreamingResponse(${rpcName}, controller),`)
} else {
handlerSetup.push(` ${rpcName}: wrapper(${rpcName}, controller),`)
}
}
handlerSetup.push(` });`)
imports.push("")
handlerSetup.push("")
}
return {
imports: imports.join("\n"),
handlerSetup: handlerSetup.join("\n"),
}
}
const { imports, handlerSetup } = generateHandlersAndExports()
const scriptName = path.basename(fileURLToPath(import.meta.url))
// Create output file
let output = `// GENERATED CODE -- DO NOT EDIT!
// Generated by ${scriptName}
import * as grpc from "@grpc/grpc-js"
import { Controller } from "../core/controller"
import { GrpcHandlerWrapper, GrpcStreamingResponseHandlerWrapper } from "./grpc-types"
${imports}
export function addServices(
server: grpc.Server,
proto: any,
controller: Controller,
wrapper: GrpcHandlerWrapper,
wrapStreamingResponse: GrpcStreamingResponseHandlerWrapper,
): void {
${handlerSetup}
}
`
// Write output file
fs.writeFileSync(OUT_FILE, output)
console.log(`Generated service handlers in ${OUT_FILE}.`)