diff --git a/docs/plan/target-architecture.md b/docs/plan/target-architecture.md index c8dcd2b9..ca04c0c3 100644 --- a/docs/plan/target-architecture.md +++ b/docs/plan/target-architecture.md @@ -1,332 +1,51 @@ -# XCEngine Target Architecture Plan +# XCEngine 重构计划 -## 1. Goal +## 1. 目标 -This document defines the target repository architecture for XCEngine. +这份计划用于定义 XCEngine 当前一轮重构的执行顺序和架构边界。 -The primary goals are: +本轮重构的目标是: -- Separate runtime and pipeline responsibilities at the source and build level. -- Stop pipeline-only code from leaking into runtime targets. -- Establish a stable shared contract layer for content identifiers and cooked payload formats. -- Separate project source assets, import cache, and final cooked output. -- Reshape the repository so future refactors are driven by dependency boundaries instead of directory names. +- 在代码和构建层面拆开 runtime、pipeline、shared 三类职责 +- 在拆分过程中保持 editor 日常迭代路径稳定 +- 先理顺依赖和 API,再做物理目录调整 +- 每个阶段结束后都执行一次 `XCEditor` 编译和 12 秒冒烟测试 -## 2. Core Design Rules +## 2. 本轮范围 -### 2.1 Stage Separation +这份计划描述的是当前引擎重构阶段,不是最终交付产物布局。 -The codebase is split by execution stage: +当前阶段的项目侧数据根目录是: -- `engine/runtime`: game runtime code -- `pipeline`: asset import, compilation, cook, package, catalog generation -- `editor`: current editor application tree, kept as-is in this phase -- `programs`: executable entry points +- `project/Assets` +- `project/Settings` +- `project/Library` -### 2.2 Dependency Separation +当前阶段的基本要求是: -The codebase is split by dependency direction: +- editor 日常迭代继续基于现有项目工作流 +- import、cache、rebuild 等职责逐步从 runtime 侧 API 中拆出去 +- 顶层 `editor` 目录结构在前几个阶段保持稳定 +- 物理目录调整放到依赖边界稳定之后再做 -- `foundation` is the lowest layer -- `shared` contains stable cross-stage data contracts -- `runtime` depends on `foundation` and `shared` -- `pipeline` depends on `foundation` and `shared`, but must not depend on `runtime` -- current top-level `editor` remains a consumer of runtime and pipeline, but its internal restructuring is out of scope for this phase +打包输出路径和产物布局放到单独的交付阶段再定义。 -### 2.3 Artifact Separation +## 3. 架构规则 -Project data is split by lifecycle: +### 3.1 阶段划分 -- `project/Assets`: source assets -- `project/Library`: import cache and intermediate artifacts -- `project/Content`: final cooked output +代码库按执行阶段拆分为: -This separation is mandatory. Runtime code must not treat `Library` artifacts as its main content source. +- `engine/foundation`:基础设施层 +- `engine/shared`:跨阶段稳定契约层 +- `engine/runtime`:运行时系统 +- `pipeline`:导入、构建、烘焙系统 +- `editor`:当前编辑器应用树 +- `programs`:各类可执行入口 -## 3. Target Repository Layout +### 3.2 依赖方向 -```text -XCEngine/ -├─ CMakeLists.txt -├─ cmake/ -│ ├─ presets/ -│ ├─ toolchains/ -│ └─ targets/ -├─ engine/ -│ ├─ foundation/ -│ │ ├─ base/ -│ │ ├─ containers/ -│ │ ├─ memory/ -│ │ ├─ threading/ -│ │ ├─ diagnostics/ -│ │ ├─ filesystem/ -│ │ ├─ math/ -│ │ └─ serialization/ -│ ├─ shared/ -│ │ ├─ content/ -│ │ │ ├─ AssetRef/ -│ │ │ ├─ LocalID/ -│ │ │ ├─ ResourceType/ -│ │ │ ├─ CatalogFormat/ -│ │ │ └─ PackageFormat/ -│ │ └─ cooked/ -│ │ ├─ CompiledTexture/ -│ │ ├─ CompiledMaterial/ -│ │ ├─ CompiledShader/ -│ │ ├─ CompiledModel/ -│ │ ├─ CompiledUIDocument/ -│ │ ├─ CompiledVolume/ -│ │ └─ CompiledGaussianSplat/ -│ ├─ runtime/ -│ │ ├─ platform/ -│ │ ├─ content/ -│ │ │ ├─ AssetManager/ -│ │ │ ├─ ResourceStore/ -│ │ │ ├─ AsyncLoad/ -│ │ │ ├─ Catalog/ -│ │ │ ├─ Package/ -│ │ │ └─ CookedReaders/ -│ │ ├─ resources/ -│ │ │ ├─ Texture/ -│ │ │ ├─ Mesh/ -│ │ │ ├─ Material/ -│ │ │ ├─ Shader/ -│ │ │ ├─ Model/ -│ │ │ ├─ UI/ -│ │ │ ├─ Volume/ -│ │ │ └─ GaussianSplat/ -│ │ ├─ scene/ -│ │ ├─ rendering/ -│ │ │ ├─ core/ -│ │ │ ├─ passes/ -│ │ │ ├─ pipelines/ -│ │ │ ├─ frame_graph/ -│ │ │ └─ gpu_cache/ -│ │ ├─ audio/ -│ │ ├─ physics/ -│ │ └─ scripting/ -│ └─ builtin_content/ -│ ├─ source/ -│ └─ cooked/ -├─ pipeline/ -│ ├─ core/ -│ │ ├─ SourceAssetDB/ -│ │ ├─ ArtifactDB/ -│ │ ├─ SourceIndex/ -│ │ ├─ DependencyGraph/ -│ │ ├─ BuildCache/ -│ │ ├─ BuildGraph/ -│ │ └─ ImportService/ -│ ├─ importers/ -│ │ ├─ TextureImporter/ -│ │ ├─ MaterialImporter/ -│ │ ├─ ShaderImporter/ -│ │ ├─ ModelImporter/ -│ │ ├─ UIDocumentImporter/ -│ │ ├─ VolumeImporter/ -│ │ └─ GaussianSplatImporter/ -│ ├─ compilers/ -│ │ ├─ ShaderCompiler/ -│ │ └─ UIDocumentCompiler/ -│ ├─ writers/ -│ │ ├─ ArtifactWriter/ -│ │ ├─ CatalogWriter/ -│ │ └─ PackageWriter/ -│ ├─ cook/ -│ └─ tools/ -├─ editor/ # existing editor tree retained in current phase -├─ programs/ -│ ├─ xcgame/ -│ ├─ xcassetbuild/ -│ ├─ xccook/ -│ └─ xcshadercook/ -├─ managed/ -│ ├─ runtime/ -│ └─ editor/ -├─ project/ -│ ├─ Assets/ -│ ├─ Settings/ -│ ├─ Library/ -│ │ ├─ SourceDB/ -│ │ ├─ ArtifactDB/ -│ │ ├─ Artifacts/ -│ │ ├─ ImportLogs/ -│ │ └─ ScriptAssemblies/ -│ └─ Content/ -│ ├─ Catalogs/ -│ └─ Packages/ -├─ tests/ -│ ├─ foundation/ -│ ├─ shared/ -│ ├─ runtime/ -│ ├─ pipeline/ -│ ├─ editor/ -│ └─ e2e/ -├─ docs/ -├─ tools/ -└─ third_party/ -``` - -## 4. Directory Responsibilities - -### 4.1 `engine/foundation` - -Lowest-level common code used by the whole repository. - -Typical contents: - -- primitive types -- logging and diagnostics -- string and container utilities -- math -- threading -- memory allocation -- binary/text serialization helpers -- low-level filesystem wrappers - -This layer must not know about assets, scene, rendering, editor, or pipeline stages. - -### 4.2 `engine/shared` - -Stable cross-stage contracts shared by runtime, pipeline, and editor. - -`shared/content` contains content addressing and package-level protocols: - -- `AssetRef` -- `LocalID` -- `ResourceType` -- catalog entry structures -- package entry structures -- versioning constants - -`shared/cooked` contains cooked payload definitions for each resource type: - -- compiled texture payload -- compiled shader payload -- compiled material payload -- compiled model payload -- compiled UI payload -- compiled volume payload -- compiled gaussian splat payload - -This layer must contain data contracts only. It must not contain managers, importers, loaders, caches, or build orchestration code. - -### 4.3 `engine/runtime` - -Game runtime code only. - -Key rules: - -- runtime reads cooked content -- runtime must not own reimport or build-cache logic -- runtime must not call importers or artifact builders -- runtime must not depend on `pipeline` - -Runtime content subsystem split: - -- `AssetManager`: runtime-facing load, unload, resolve, and async load entry point -- `ResourceStore`: CPU-side runtime object cache -- `AsyncLoad`: background load scheduling -- `Catalog`: cooked catalog resolution -- `Package`: cooked package access -- `CookedReaders`: parse cooked payloads into runtime objects - -Rendering split: - -- `core`: renderer base types and context -- `passes`: render passes -- `pipelines`: scene/game/render pipelines -- `frame_graph`: frame execution graph and scheduling -- `gpu_cache`: GPU-side residency and upload cache - -### 4.4 `pipeline` - -Authoring pipeline and cook system. - -`pipeline/core` responsibilities: - -- source asset database -- artifact database -- path-to-asset index -- dependency graph -- build cache -- build graph -- high-level import service - -`pipeline/importers` responsibilities: - -- source file ingestion -- asset-specific import logic -- import settings interpretation - -`pipeline/compilers` responsibilities: - -- shader compilation -- UI document compilation - -`pipeline/writers` responsibilities: - -- write Library artifacts -- write cooked catalogs -- write cooked packages - -This layer is the only place allowed to rebuild `Library` artifacts. - -### 4.5 top-level `editor` - -The editor codebase remains in its current structure for this phase. - -This plan does not prescribe a refactor of: - -- `editor/app` -- `editor/include/XCEditor` -- `editor/src/UI` -- `editor/src/Product` -- `editor/resources` -- editor-specific build target layout - -The editor remains a consumer of runtime and pipeline outputs, but its internal restructuring is not -part of the current architecture plan. - -### 4.6 `programs` - -Executable entry points. - -Examples: - -- runtime game executable -- editor executable -- asset build tool -- cook tool -- shader cook tool - -Tool `main()` functions belong here, not inside runtime or pipeline libraries. - -### 4.7 `project` - -Project-side content and generated outputs. - -- `Assets`: source assets under authoring -- `Settings`: project configuration -- `Library`: imported cache and intermediate state -- `Content`: final cooked output - -The repository root must not contain a global `generated/` directory for project outputs. - -### 4.8 `tests` - -Tests mirror architecture boundaries. - -- `foundation`: low-level base library tests -- `shared`: payload and protocol tests -- `runtime`: runtime subsystem tests -- `pipeline`: importer/build/cook tests -- `editor`: editor application and editor-engine bridge tests -- `e2e`: full flow tests across project, pipeline, and runtime - -## 5. Dependency Rules - -The dependency graph must converge to: +目标依赖关系收敛为: ```text foundation @@ -338,29 +57,252 @@ shared runtime pipeline ^ ^ | | - +--------editor (existing structure, unchanged in this phase) + +--------editor programs -> runtime / pipeline ``` -Mandatory constraints: +约束如下: -- `pipeline` must not depend on `runtime` -- `runtime` must not depend on `pipeline` -- top-level `editor` can orchestrate both runtime and pipeline -- shared code must remain data-contract-oriented +- `runtime` 只向下依赖 `foundation` 和 `shared` +- `pipeline` 只向下依赖 `foundation` 和 `shared` +- `editor` 负责编排 runtime 和 pipeline +- `shared` 只保留稳定数据契约 -## 6. Build Target Plan +### 3.3 项目数据职责 -Suggested target families: +当前阶段的项目侧数据职责为: + +- `project/Assets`:源资源 +- `project/Settings`:项目配置 +- `project/Library`:导入缓存、中间产物、索引、日志、脚本编译输出 + +`Library` 归 pipeline 侧职责管理。 +runtime 可以消费现有工作流准备好的资源数据,但 import 和 build 编排必须继续往 pipeline 侧收拢。 + +## 4. 目标目录结构 + +```text +XCEngine/ + CMakeLists.txt + cmake/ + engine/ + foundation/ + base/ + containers/ + memory/ + threading/ + diagnostics/ + filesystem/ + math/ + serialization/ + shared/ + asset/ + AssetRef/ + LocalID/ + ResourceType/ + CatalogFormat/ + PackageFormat/ + cooked/ + CompiledTexture/ + CompiledMaterial/ + CompiledShader/ + CompiledModel/ + CompiledUIDocument/ + CompiledVolume/ + CompiledGaussianSplat/ + runtime/ + platform/ + asset/ + AssetManager/ + ResourceStore/ + AsyncLoad/ + Catalog/ + Package/ + CookedReaders/ + resources/ + Texture/ + Mesh/ + Material/ + Shader/ + Model/ + UI/ + Volume/ + GaussianSplat/ + scene/ + rendering/ + core/ + passes/ + pipelines/ + frame_graph/ + gpu_cache/ + audio/ + physics/ + scripting/ + pipeline/ + core/ + SourceAssetDB/ + ArtifactDB/ + SourceIndex/ + DependencyGraph/ + BuildCache/ + BuildGraph/ + ImportService/ + importers/ + TextureImporter/ + MaterialImporter/ + ShaderImporter/ + ModelImporter/ + UIDocumentImporter/ + VolumeImporter/ + GaussianSplatImporter/ + compilers/ + ShaderCompiler/ + UIDocumentCompiler/ + writers/ + ArtifactWriter/ + CatalogWriter/ + PackageWriter/ + cook/ + tools/ + editor/ + programs/ + xcgame/ + xcassetbuild/ + xccook/ + xcshadercook/ + managed/ + runtime/ + editor/ + project/ + Assets/ + Settings/ + Library/ + SourceDB/ + ArtifactDB/ + Artifacts/ + ImportLogs/ + ScriptAssemblies/ + tests/ + foundation/ + shared/ + runtime/ + pipeline/ + editor/ + e2e/ + docs/ + tools/ + third_party/ +``` + +## 5. 各层职责 + +### 5.1 `engine/foundation` + +基础设施层承载全仓通用能力: + +- 基础类型 +- 日志与诊断 +- 字符串和容器 +- 数学库 +- 线程和同步 +- 内存管理 +- 序列化 +- 文件系统封装 + +### 5.2 `engine/shared` + +共享契约层承载跨阶段稳定数据结构: + +- 资源标识 +- 本地标识 +- 资源类型枚举 +- catalog / package 描述结构 +- 各类 cooked payload 结构 +- 版本常量 + +这一层只放数据契约,不放 manager、loader、cache、importer、orchestrator。 + +### 5.3 `engine/runtime` + +运行时层负责: + +- 资源加载与解析 +- 运行时对象缓存 +- 异步加载调度 +- scene 系统 +- rendering +- audio +- physics +- scripting + +runtime 侧收敛原则: + +- 面向运行时消费资源 +- 不承载源资源导入 +- 不承载构建缓存管理 +- 不直接持有 pipeline 服务实现 + +### 5.4 `pipeline` + +pipeline 层负责: + +- 源资源数据库 +- 中间产物数据库 +- 源资源索引 +- 依赖关系图 +- 构建缓存 +- 构建图 +- import service +- 各类 importer 和 compiler +- 中间产物写入和后续打包写入 + +`project/Library` 下的导入和构建状态由这一层统一负责。 + +### 5.5 `editor` + +当前阶段保留现有顶层 `editor` 应用树,职责包括: + +- workspace / project 相关交互 +- runtime 与 pipeline 的编排 +- editor 场景和工具工作流 + +editor 内部目录重组不属于第一阶段工作。 + +### 5.6 `programs` + +各类可执行入口统一放在这里,例如: + +- 游戏程序 +- 资源构建工具 +- 烘焙工具 +- shader 工具 + +## 6. 当前系统到目标结构的映射 + +当前主要系统映射如下: + +- `ResourceManager` -> `engine/runtime/asset/AssetManager` +- `ResourceCache` -> `engine/runtime/asset/ResourceStore` +- `AsyncLoader` -> `engine/runtime/asset/AsyncLoad` +- `ProjectAssetIndex` -> `pipeline/core/SourceIndex` +- `AssetImportService` -> `pipeline/core/ImportService` +- `AssetDatabase` -> `pipeline/core/SourceAssetDB` + `pipeline/core/ArtifactDB` +- `RenderResourceCache` -> `engine/runtime/rendering/gpu_cache` + +重构过程中,editor 迭代路径先保持在现有项目工作流上,再逐步完成职责分拆。 + +## 7. 构建目标规划 + +建议的目标族如下: ```text xc_foundation -xc_shared_content +xc_shared_asset xc_shared_cooked xc_runtime_platform -xc_runtime_content +xc_runtime_asset xc_runtime_resources xc_runtime_scene xc_runtime_rendering @@ -382,73 +324,90 @@ xccook xcshadercook ``` -Build constraints: +构建侧落地原则: -- shipping game links `xc_runtime` -- current editor targets remain unchanged in this phase -- asset tools link `xc_pipeline` +- 运行时可执行程序链接 `xc_runtime` +- 资源工具链接 `xc_pipeline` +- editor 入口结构在前几个阶段保持现状 -## 7. Current-to-Target Mapping +## 8. 分阶段执行计划 -### 7.1 Content System +### 阶段一:构建边界拆分 -Current: +目标: -- `ResourceManager` -- `ResourceCache` -- `AsyncLoader` -- `ProjectAssetIndex` -- `AssetImportService` -- `AssetDatabase` +- 先把现有代码拆成 foundation、shared、runtime、pipeline 四类目标 +- 这一阶段尽量少动物理文件位置 -Target: +工作项: -- `ResourceManager` -> `engine/runtime/content/AssetManager` -- `ResourceCache` -> `engine/runtime/content/ResourceStore` -- `AsyncLoader` -> `engine/runtime/content/AsyncLoad` -- `ProjectAssetIndex` -> `pipeline/core/SourceIndex` -- `AssetImportService` -> `pipeline/core/ImportService` -- `AssetDatabase` -> split into `pipeline/core/SourceAssetDB` and `pipeline/core/ArtifactDB` +- 拆分 CMake target +- 收敛公共头文件暴露面 +- 消除 target 级别的 runtime / pipeline 混杂依赖 -Critical rule: +阶段验收: -- runtime asset load path must stop rebuilding source artifacts +1. 编译 `XCEditor` +2. 执行 12 秒 editor 冒烟测试 -### 7.2 Rendering Cache +### 阶段二:API 边界拆分 -Current: +目标: -- `RenderResourceCache` +- 把 runtime 侧资源访问接口和 pipeline 侧导入/构建职责拆开 -Target: +工作项: -- `engine/runtime/rendering/gpu_cache` +- 将 import / build 逻辑收拢到 pipeline 服务 +- 将 runtime manager 收窄到 load / resolve / cache 范围 +- 保持 editor 当前迭代工作流稳定 -This remains runtime code, but is explicitly recognized as GPU-side cache rather than authoring cache. +阶段验收: -### 7.3 Editor +1. 编译 `XCEditor` +2. 执行 12 秒 editor 冒烟测试 -Editor restructuring is explicitly out of scope for this phase. +### 阶段三:物理目录迁移 -The current editor source tree and editor build targets remain unchanged. +目标: -## 8. Hard Architectural Rules +- 让磁盘目录结构和依赖边界对齐 -The following rules are non-negotiable: +工作项: -1. `AssetManager` must not expose reimport, clear-library, or build-cache APIs. -2. Runtime load paths must consume cooked outputs, not `Library` internal artifacts as a long-term contract. -3. `Library` is a pipeline-owned cache, not runtime-owned content. -4. Cross-stage shared code must contain stable data contracts only. -5. New runtime and pipeline features must enter through the correct layer instead of extending the nearest existing folder. +- 把文件迁移到 `engine/foundation`、`engine/shared`、`engine/runtime`、`pipeline` +- 更新 include、CMake 和测试 +- 除非前置拆分强依赖,否则不在这一阶段重组 editor 内部树 -## 9. Migration Direction +阶段验收: -Refactoring should proceed in this order: +1. 编译 `XCEditor` +2. 执行 12 秒 editor 冒烟测试 -1. Split build targets before moving many files. -2. Separate runtime content loading from pipeline import and rebuild logic. -3. Move project-generated outputs under `project/Library` and `project/Content`. -4. Align tests with new boundaries. +### 阶段四:交付与打包阶段 -This sequence matters. File moves without target and dependency separation will only recreate the current coupling under new folder names. +目标: + +- 在 runtime、pipeline、editor 职责已经清晰之后,再引入独立交付流程 + +工作项: + +- 增加专用 cook / build 可执行工具 +- 明确交付产物格式和输出路径 +- 将交付流程与 editor 日常迭代流程分离 + +阶段验收: + +1. 编译 `XCEditor` +2. 执行 12 秒 editor 冒烟测试 + +## 9. 硬性规则 + +整个重构过程都遵守以下规则: + +1. `Library` 是 pipeline 侧的缓存和中间状态。 +2. runtime API 只收敛到运行时资源访问职责。 +3. 大规模文件迁移必须排在 target 和 API 边界稳定之后。 +4. shared 层只保留稳定数据契约。 +5. phases 1 到 3 期间,editor 日常迭代路径保持连续可用。 +6. 每个阶段完成后都要先编译 `XCEditor`,再执行 12 秒冒烟测试。