455 lines
12 KiB
Markdown
455 lines
12 KiB
Markdown
# XCEngine Target Architecture Plan
|
|
|
|
## 1. Goal
|
|
|
|
This document defines the target repository architecture for XCEngine.
|
|
|
|
The primary goals are:
|
|
|
|
- Separate runtime and pipeline responsibilities at the source and build level.
|
|
- Stop pipeline-only code from leaking into runtime targets.
|
|
- Establish a stable shared contract layer for content identifiers and cooked payload formats.
|
|
- Separate project source assets, import cache, and final cooked output.
|
|
- Reshape the repository so future refactors are driven by dependency boundaries instead of directory names.
|
|
|
|
## 2. Core Design Rules
|
|
|
|
### 2.1 Stage Separation
|
|
|
|
The codebase is split by execution stage:
|
|
|
|
- `engine/runtime`: game runtime code
|
|
- `pipeline`: asset import, compilation, cook, package, catalog generation
|
|
- `editor`: current editor application tree, kept as-is in this phase
|
|
- `programs`: executable entry points
|
|
|
|
### 2.2 Dependency Separation
|
|
|
|
The codebase is split by dependency direction:
|
|
|
|
- `foundation` is the lowest layer
|
|
- `shared` contains stable cross-stage data contracts
|
|
- `runtime` depends on `foundation` and `shared`
|
|
- `pipeline` depends on `foundation` and `shared`, but must not depend on `runtime`
|
|
- current top-level `editor` remains a consumer of runtime and pipeline, but its internal restructuring is out of scope for this phase
|
|
|
|
### 2.3 Artifact Separation
|
|
|
|
Project data is split by lifecycle:
|
|
|
|
- `project/Assets`: source assets
|
|
- `project/Library`: import cache and intermediate artifacts
|
|
- `project/Content`: final cooked output
|
|
|
|
This separation is mandatory. Runtime code must not treat `Library` artifacts as its main content source.
|
|
|
|
## 3. Target Repository Layout
|
|
|
|
```text
|
|
XCEngine/
|
|
├─ CMakeLists.txt
|
|
├─ cmake/
|
|
│ ├─ presets/
|
|
│ ├─ toolchains/
|
|
│ └─ targets/
|
|
├─ engine/
|
|
│ ├─ foundation/
|
|
│ │ ├─ base/
|
|
│ │ ├─ containers/
|
|
│ │ ├─ memory/
|
|
│ │ ├─ threading/
|
|
│ │ ├─ diagnostics/
|
|
│ │ ├─ filesystem/
|
|
│ │ ├─ math/
|
|
│ │ └─ serialization/
|
|
│ ├─ shared/
|
|
│ │ ├─ content/
|
|
│ │ │ ├─ AssetRef/
|
|
│ │ │ ├─ LocalID/
|
|
│ │ │ ├─ ResourceType/
|
|
│ │ │ ├─ CatalogFormat/
|
|
│ │ │ └─ PackageFormat/
|
|
│ │ └─ cooked/
|
|
│ │ ├─ CompiledTexture/
|
|
│ │ ├─ CompiledMaterial/
|
|
│ │ ├─ CompiledShader/
|
|
│ │ ├─ CompiledModel/
|
|
│ │ ├─ CompiledUIDocument/
|
|
│ │ ├─ CompiledVolume/
|
|
│ │ └─ CompiledGaussianSplat/
|
|
│ ├─ runtime/
|
|
│ │ ├─ platform/
|
|
│ │ ├─ content/
|
|
│ │ │ ├─ AssetManager/
|
|
│ │ │ ├─ ResourceStore/
|
|
│ │ │ ├─ AsyncLoad/
|
|
│ │ │ ├─ Catalog/
|
|
│ │ │ ├─ Package/
|
|
│ │ │ └─ CookedReaders/
|
|
│ │ ├─ resources/
|
|
│ │ │ ├─ Texture/
|
|
│ │ │ ├─ Mesh/
|
|
│ │ │ ├─ Material/
|
|
│ │ │ ├─ Shader/
|
|
│ │ │ ├─ Model/
|
|
│ │ │ ├─ UI/
|
|
│ │ │ ├─ Volume/
|
|
│ │ │ └─ GaussianSplat/
|
|
│ │ ├─ scene/
|
|
│ │ ├─ rendering/
|
|
│ │ │ ├─ core/
|
|
│ │ │ ├─ passes/
|
|
│ │ │ ├─ pipelines/
|
|
│ │ │ ├─ frame_graph/
|
|
│ │ │ └─ gpu_cache/
|
|
│ │ ├─ audio/
|
|
│ │ ├─ physics/
|
|
│ │ └─ scripting/
|
|
│ └─ builtin_content/
|
|
│ ├─ source/
|
|
│ └─ cooked/
|
|
├─ pipeline/
|
|
│ ├─ core/
|
|
│ │ ├─ SourceAssetDB/
|
|
│ │ ├─ ArtifactDB/
|
|
│ │ ├─ SourceIndex/
|
|
│ │ ├─ DependencyGraph/
|
|
│ │ ├─ BuildCache/
|
|
│ │ ├─ BuildGraph/
|
|
│ │ └─ ImportService/
|
|
│ ├─ importers/
|
|
│ │ ├─ TextureImporter/
|
|
│ │ ├─ MaterialImporter/
|
|
│ │ ├─ ShaderImporter/
|
|
│ │ ├─ ModelImporter/
|
|
│ │ ├─ UIDocumentImporter/
|
|
│ │ ├─ VolumeImporter/
|
|
│ │ └─ GaussianSplatImporter/
|
|
│ ├─ compilers/
|
|
│ │ ├─ ShaderCompiler/
|
|
│ │ └─ UIDocumentCompiler/
|
|
│ ├─ writers/
|
|
│ │ ├─ ArtifactWriter/
|
|
│ │ ├─ CatalogWriter/
|
|
│ │ └─ PackageWriter/
|
|
│ ├─ cook/
|
|
│ └─ tools/
|
|
├─ editor/ # existing editor tree retained in current phase
|
|
├─ programs/
|
|
│ ├─ xcgame/
|
|
│ ├─ xcassetbuild/
|
|
│ ├─ xccook/
|
|
│ └─ xcshadercook/
|
|
├─ managed/
|
|
│ ├─ runtime/
|
|
│ └─ editor/
|
|
├─ project/
|
|
│ ├─ Assets/
|
|
│ ├─ Settings/
|
|
│ ├─ Library/
|
|
│ │ ├─ SourceDB/
|
|
│ │ ├─ ArtifactDB/
|
|
│ │ ├─ Artifacts/
|
|
│ │ ├─ ImportLogs/
|
|
│ │ └─ ScriptAssemblies/
|
|
│ └─ Content/
|
|
│ ├─ Catalogs/
|
|
│ └─ Packages/
|
|
├─ tests/
|
|
│ ├─ foundation/
|
|
│ ├─ shared/
|
|
│ ├─ runtime/
|
|
│ ├─ pipeline/
|
|
│ ├─ editor/
|
|
│ └─ e2e/
|
|
├─ docs/
|
|
├─ tools/
|
|
└─ third_party/
|
|
```
|
|
|
|
## 4. Directory Responsibilities
|
|
|
|
### 4.1 `engine/foundation`
|
|
|
|
Lowest-level common code used by the whole repository.
|
|
|
|
Typical contents:
|
|
|
|
- primitive types
|
|
- logging and diagnostics
|
|
- string and container utilities
|
|
- math
|
|
- threading
|
|
- memory allocation
|
|
- binary/text serialization helpers
|
|
- low-level filesystem wrappers
|
|
|
|
This layer must not know about assets, scene, rendering, editor, or pipeline stages.
|
|
|
|
### 4.2 `engine/shared`
|
|
|
|
Stable cross-stage contracts shared by runtime, pipeline, and editor.
|
|
|
|
`shared/content` contains content addressing and package-level protocols:
|
|
|
|
- `AssetRef`
|
|
- `LocalID`
|
|
- `ResourceType`
|
|
- catalog entry structures
|
|
- package entry structures
|
|
- versioning constants
|
|
|
|
`shared/cooked` contains cooked payload definitions for each resource type:
|
|
|
|
- compiled texture payload
|
|
- compiled shader payload
|
|
- compiled material payload
|
|
- compiled model payload
|
|
- compiled UI payload
|
|
- compiled volume payload
|
|
- compiled gaussian splat payload
|
|
|
|
This layer must contain data contracts only. It must not contain managers, importers, loaders, caches, or build orchestration code.
|
|
|
|
### 4.3 `engine/runtime`
|
|
|
|
Game runtime code only.
|
|
|
|
Key rules:
|
|
|
|
- runtime reads cooked content
|
|
- runtime must not own reimport or build-cache logic
|
|
- runtime must not call importers or artifact builders
|
|
- runtime must not depend on `pipeline`
|
|
|
|
Runtime content subsystem split:
|
|
|
|
- `AssetManager`: runtime-facing load, unload, resolve, and async load entry point
|
|
- `ResourceStore`: CPU-side runtime object cache
|
|
- `AsyncLoad`: background load scheduling
|
|
- `Catalog`: cooked catalog resolution
|
|
- `Package`: cooked package access
|
|
- `CookedReaders`: parse cooked payloads into runtime objects
|
|
|
|
Rendering split:
|
|
|
|
- `core`: renderer base types and context
|
|
- `passes`: render passes
|
|
- `pipelines`: scene/game/render pipelines
|
|
- `frame_graph`: frame execution graph and scheduling
|
|
- `gpu_cache`: GPU-side residency and upload cache
|
|
|
|
### 4.4 `pipeline`
|
|
|
|
Authoring pipeline and cook system.
|
|
|
|
`pipeline/core` responsibilities:
|
|
|
|
- source asset database
|
|
- artifact database
|
|
- path-to-asset index
|
|
- dependency graph
|
|
- build cache
|
|
- build graph
|
|
- high-level import service
|
|
|
|
`pipeline/importers` responsibilities:
|
|
|
|
- source file ingestion
|
|
- asset-specific import logic
|
|
- import settings interpretation
|
|
|
|
`pipeline/compilers` responsibilities:
|
|
|
|
- shader compilation
|
|
- UI document compilation
|
|
|
|
`pipeline/writers` responsibilities:
|
|
|
|
- write Library artifacts
|
|
- write cooked catalogs
|
|
- write cooked packages
|
|
|
|
This layer is the only place allowed to rebuild `Library` artifacts.
|
|
|
|
### 4.5 top-level `editor`
|
|
|
|
The editor codebase remains in its current structure for this phase.
|
|
|
|
This plan does not prescribe a refactor of:
|
|
|
|
- `editor/app`
|
|
- `editor/include/XCEditor`
|
|
- `editor/src/UI`
|
|
- `editor/src/Product`
|
|
- `editor/resources`
|
|
- editor-specific build target layout
|
|
|
|
The editor remains a consumer of runtime and pipeline outputs, but its internal restructuring is not
|
|
part of the current architecture plan.
|
|
|
|
### 4.6 `programs`
|
|
|
|
Executable entry points.
|
|
|
|
Examples:
|
|
|
|
- runtime game executable
|
|
- editor executable
|
|
- asset build tool
|
|
- cook tool
|
|
- shader cook tool
|
|
|
|
Tool `main()` functions belong here, not inside runtime or pipeline libraries.
|
|
|
|
### 4.7 `project`
|
|
|
|
Project-side content and generated outputs.
|
|
|
|
- `Assets`: source assets under authoring
|
|
- `Settings`: project configuration
|
|
- `Library`: imported cache and intermediate state
|
|
- `Content`: final cooked output
|
|
|
|
The repository root must not contain a global `generated/` directory for project outputs.
|
|
|
|
### 4.8 `tests`
|
|
|
|
Tests mirror architecture boundaries.
|
|
|
|
- `foundation`: low-level base library tests
|
|
- `shared`: payload and protocol tests
|
|
- `runtime`: runtime subsystem tests
|
|
- `pipeline`: importer/build/cook tests
|
|
- `editor`: editor application and editor-engine bridge tests
|
|
- `e2e`: full flow tests across project, pipeline, and runtime
|
|
|
|
## 5. Dependency Rules
|
|
|
|
The dependency graph must converge to:
|
|
|
|
```text
|
|
foundation
|
|
^
|
|
|
|
|
shared
|
|
^ ^
|
|
| |
|
|
runtime pipeline
|
|
^ ^
|
|
| |
|
|
+--------editor (existing structure, unchanged in this phase)
|
|
|
|
programs -> runtime / pipeline
|
|
```
|
|
|
|
Mandatory constraints:
|
|
|
|
- `pipeline` must not depend on `runtime`
|
|
- `runtime` must not depend on `pipeline`
|
|
- top-level `editor` can orchestrate both runtime and pipeline
|
|
- shared code must remain data-contract-oriented
|
|
|
|
## 6. Build Target Plan
|
|
|
|
Suggested target families:
|
|
|
|
```text
|
|
xc_foundation
|
|
xc_shared_content
|
|
xc_shared_cooked
|
|
|
|
xc_runtime_platform
|
|
xc_runtime_content
|
|
xc_runtime_resources
|
|
xc_runtime_scene
|
|
xc_runtime_rendering
|
|
xc_runtime_audio
|
|
xc_runtime_physics
|
|
xc_runtime_scripting
|
|
xc_runtime
|
|
|
|
xc_pipeline_core
|
|
xc_pipeline_importers
|
|
xc_pipeline_compilers
|
|
xc_pipeline_writers
|
|
xc_pipeline_cook
|
|
xc_pipeline
|
|
|
|
xcgame
|
|
xcassetbuild
|
|
xccook
|
|
xcshadercook
|
|
```
|
|
|
|
Build constraints:
|
|
|
|
- shipping game links `xc_runtime`
|
|
- current editor targets remain unchanged in this phase
|
|
- asset tools link `xc_pipeline`
|
|
|
|
## 7. Current-to-Target Mapping
|
|
|
|
### 7.1 Content System
|
|
|
|
Current:
|
|
|
|
- `ResourceManager`
|
|
- `ResourceCache`
|
|
- `AsyncLoader`
|
|
- `ProjectAssetIndex`
|
|
- `AssetImportService`
|
|
- `AssetDatabase`
|
|
|
|
Target:
|
|
|
|
- `ResourceManager` -> `engine/runtime/content/AssetManager`
|
|
- `ResourceCache` -> `engine/runtime/content/ResourceStore`
|
|
- `AsyncLoader` -> `engine/runtime/content/AsyncLoad`
|
|
- `ProjectAssetIndex` -> `pipeline/core/SourceIndex`
|
|
- `AssetImportService` -> `pipeline/core/ImportService`
|
|
- `AssetDatabase` -> split into `pipeline/core/SourceAssetDB` and `pipeline/core/ArtifactDB`
|
|
|
|
Critical rule:
|
|
|
|
- runtime asset load path must stop rebuilding source artifacts
|
|
|
|
### 7.2 Rendering Cache
|
|
|
|
Current:
|
|
|
|
- `RenderResourceCache`
|
|
|
|
Target:
|
|
|
|
- `engine/runtime/rendering/gpu_cache`
|
|
|
|
This remains runtime code, but is explicitly recognized as GPU-side cache rather than authoring cache.
|
|
|
|
### 7.3 Editor
|
|
|
|
Editor restructuring is explicitly out of scope for this phase.
|
|
|
|
The current editor source tree and editor build targets remain unchanged.
|
|
|
|
## 8. Hard Architectural Rules
|
|
|
|
The following rules are non-negotiable:
|
|
|
|
1. `AssetManager` must not expose reimport, clear-library, or build-cache APIs.
|
|
2. Runtime load paths must consume cooked outputs, not `Library` internal artifacts as a long-term contract.
|
|
3. `Library` is a pipeline-owned cache, not runtime-owned content.
|
|
4. Cross-stage shared code must contain stable data contracts only.
|
|
5. New runtime and pipeline features must enter through the correct layer instead of extending the nearest existing folder.
|
|
|
|
## 9. Migration Direction
|
|
|
|
Refactoring should proceed in this order:
|
|
|
|
1. Split build targets before moving many files.
|
|
2. Separate runtime content loading from pipeline import and rebuild logic.
|
|
3. Move project-generated outputs under `project/Library` and `project/Content`.
|
|
4. Align tests with new boundaries.
|
|
|
|
This sequence matters. File moves without target and dependency separation will only recreate the current coupling under new folder names.
|