XCEngine/docs/plan/target-architecture.md

XCEngine Target Architecture Plan

1. Goal

This document defines the target repository architecture for XCEngine.

The primary goals are:

• Separate runtime and pipeline responsibilities at the source and build level.
• Stop pipeline-only code from leaking into runtime targets.
• Establish a stable shared contract layer for content identifiers and cooked payload formats.
• Separate project source assets, import cache, and final cooked output.
• Reshape the repository so future refactors are driven by dependency boundaries instead of directory names.

2. Core Design Rules

2.1 Stage Separation

The codebase is split by execution stage:

• engine/runtime: game runtime code
• pipeline: asset import, compilation, cook, package, catalog generation
• editor: current editor application tree, kept as-is in this phase
• programs: executable entry points

2.2 Dependency Separation

The codebase is split by dependency direction:

• foundation is the lowest layer
• shared contains stable cross-stage data contracts
• runtime depends on foundation and shared
• pipeline depends on foundation and shared, but must not depend on runtime
• current top-level editor remains a consumer of runtime and pipeline, but its internal restructuring is out of scope for this phase

2.3 Artifact Separation

Project data is split by lifecycle:

• project/Assets: source assets
• project/Library: import cache and intermediate artifacts
• project/Content: final cooked output

This separation is mandatory. Runtime code must not treat Library artifacts as its main content source.

3. Target Repository Layout

XCEngine/
├─ CMakeLists.txt
├─ cmake/
│  ├─ presets/
│  ├─ toolchains/
│  └─ targets/
├─ engine/
│  ├─ foundation/
│  │  ├─ base/
│  │  ├─ containers/
│  │  ├─ memory/
│  │  ├─ threading/
│  │  ├─ diagnostics/
│  │  ├─ filesystem/
│  │  ├─ math/
│  │  └─ serialization/
│  ├─ shared/
│  │  ├─ content/
│  │  │  ├─ AssetRef/
│  │  │  ├─ LocalID/
│  │  │  ├─ ResourceType/
│  │  │  ├─ CatalogFormat/
│  │  │  └─ PackageFormat/
│  │  └─ cooked/
│  │     ├─ CompiledTexture/
│  │     ├─ CompiledMaterial/
│  │     ├─ CompiledShader/
│  │     ├─ CompiledModel/
│  │     ├─ CompiledUIDocument/
│  │     ├─ CompiledVolume/
│  │     └─ CompiledGaussianSplat/
│  ├─ runtime/
│  │  ├─ platform/
│  │  ├─ content/
│  │  │  ├─ AssetManager/
│  │  │  ├─ ResourceStore/
│  │  │  ├─ AsyncLoad/
│  │  │  ├─ Catalog/
│  │  │  ├─ Package/
│  │  │  └─ CookedReaders/
│  │  ├─ resources/
│  │  │  ├─ Texture/
│  │  │  ├─ Mesh/
│  │  │  ├─ Material/
│  │  │  ├─ Shader/
│  │  │  ├─ Model/
│  │  │  ├─ UI/
│  │  │  ├─ Volume/
│  │  │  └─ GaussianSplat/
│  │  ├─ scene/
│  │  ├─ rendering/
│  │  │  ├─ core/
│  │  │  ├─ passes/
│  │  │  ├─ pipelines/
│  │  │  ├─ frame_graph/
│  │  │  └─ gpu_cache/
│  │  ├─ audio/
│  │  ├─ physics/
│  │  └─ scripting/
│  └─ builtin_content/
│     ├─ source/
│     └─ cooked/
├─ pipeline/
│  ├─ core/
│  │  ├─ SourceAssetDB/
│  │  ├─ ArtifactDB/
│  │  ├─ SourceIndex/
│  │  ├─ DependencyGraph/
│  │  ├─ BuildCache/
│  │  ├─ BuildGraph/
│  │  └─ ImportService/
│  ├─ importers/
│  │  ├─ TextureImporter/
│  │  ├─ MaterialImporter/
│  │  ├─ ShaderImporter/
│  │  ├─ ModelImporter/
│  │  ├─ UIDocumentImporter/
│  │  ├─ VolumeImporter/
│  │  └─ GaussianSplatImporter/
│  ├─ compilers/
│  │  ├─ ShaderCompiler/
│  │  └─ UIDocumentCompiler/
│  ├─ writers/
│  │  ├─ ArtifactWriter/
│  │  ├─ CatalogWriter/
│  │  └─ PackageWriter/
│  ├─ cook/
│  └─ tools/
├─ editor/                           # existing editor tree retained in current phase
├─ programs/
│  ├─ xcgame/
│  ├─ xcassetbuild/
│  ├─ xccook/
│  └─ xcshadercook/
├─ managed/
│  ├─ runtime/
│  └─ editor/
├─ project/
│  ├─ Assets/
│  ├─ Settings/
│  ├─ Library/
│  │  ├─ SourceDB/
│  │  ├─ ArtifactDB/
│  │  ├─ Artifacts/
│  │  ├─ ImportLogs/
│  │  └─ ScriptAssemblies/
│  └─ Content/
│     ├─ Catalogs/
│     └─ Packages/
├─ tests/
│  ├─ foundation/
│  ├─ shared/
│  ├─ runtime/
│  ├─ pipeline/
│  ├─ editor/
│  └─ e2e/
├─ docs/
├─ tools/
└─ third_party/

4. Directory Responsibilities

4.1 engine/foundation

Lowest-level common code used by the whole repository.

Typical contents:

• primitive types
• logging and diagnostics
• string and container utilities
• math
• threading
• memory allocation
• binary/text serialization helpers
• low-level filesystem wrappers

This layer must not know about assets, scene, rendering, editor, or pipeline stages.

4.2 engine/shared

Stable cross-stage contracts shared by runtime, pipeline, and editor.

shared/content contains content addressing and package-level protocols:

• AssetRef
• LocalID
• ResourceType
• catalog entry structures
• package entry structures
• versioning constants

shared/cooked contains cooked payload definitions for each resource type:

• compiled texture payload
• compiled shader payload
• compiled material payload
• compiled model payload
• compiled UI payload
• compiled volume payload
• compiled gaussian splat payload

This layer must contain data contracts only. It must not contain managers, importers, loaders, caches, or build orchestration code.

4.3 engine/runtime

Game runtime code only.

Key rules:

• runtime reads cooked content
• runtime must not own reimport or build-cache logic
• runtime must not call importers or artifact builders
• runtime must not depend on pipeline

Runtime content subsystem split:

• AssetManager: runtime-facing load, unload, resolve, and async load entry point
• ResourceStore: CPU-side runtime object cache
• AsyncLoad: background load scheduling
• Catalog: cooked catalog resolution
• Package: cooked package access
• CookedReaders: parse cooked payloads into runtime objects

Rendering split:

• core: renderer base types and context
• passes: render passes
• pipelines: scene/game/render pipelines
• frame_graph: frame execution graph and scheduling
• gpu_cache: GPU-side residency and upload cache

4.4 pipeline

Authoring pipeline and cook system.

pipeline/core responsibilities:

• source asset database
• artifact database
• path-to-asset index
• dependency graph
• build cache
• build graph
• high-level import service

pipeline/importers responsibilities:

• source file ingestion
• asset-specific import logic
• import settings interpretation

pipeline/compilers responsibilities:

• shader compilation
• UI document compilation

pipeline/writers responsibilities:

• write Library artifacts
• write cooked catalogs
• write cooked packages

This layer is the only place allowed to rebuild Library artifacts.

4.5 top-level editor

The editor codebase remains in its current structure for this phase.

This plan does not prescribe a refactor of:

• editor/app
• editor/include/XCEditor
• editor/src/UI
• editor/src/Product
• editor/resources
• editor-specific build target layout

The editor remains a consumer of runtime and pipeline outputs, but its internal restructuring is not part of the current architecture plan.

4.6 programs

Executable entry points.

Examples:

• runtime game executable
• editor executable
• asset build tool
• cook tool
• shader cook tool

Tool main() functions belong here, not inside runtime or pipeline libraries.

4.7 project

Project-side content and generated outputs.

• Assets: source assets under authoring
• Settings: project configuration
• Library: imported cache and intermediate state
• Content: final cooked output

The repository root must not contain a global generated/ directory for project outputs.

4.8 tests

Tests mirror architecture boundaries.

• foundation: low-level base library tests
• shared: payload and protocol tests
• runtime: runtime subsystem tests
• pipeline: importer/build/cook tests
• editor: editor application and editor-engine bridge tests
• e2e: full flow tests across project, pipeline, and runtime

5. Dependency Rules

The dependency graph must converge to:

foundation
  ^
  |
shared
  ^        ^
  |        |
runtime   pipeline
  ^        ^
  |        |
  +--------editor (existing structure, unchanged in this phase)

programs -> runtime / pipeline

Mandatory constraints:

• pipeline must not depend on runtime
• runtime must not depend on pipeline
• top-level editor can orchestrate both runtime and pipeline
• shared code must remain data-contract-oriented

6. Build Target Plan

Suggested target families:

xc_foundation
xc_shared_content
xc_shared_cooked

xc_runtime_platform
xc_runtime_content
xc_runtime_resources
xc_runtime_scene
xc_runtime_rendering
xc_runtime_audio
xc_runtime_physics
xc_runtime_scripting
xc_runtime

xc_pipeline_core
xc_pipeline_importers
xc_pipeline_compilers
xc_pipeline_writers
xc_pipeline_cook
xc_pipeline

xcgame
xcassetbuild
xccook
xcshadercook

Build constraints:

• shipping game links xc_runtime
• current editor targets remain unchanged in this phase
• asset tools link xc_pipeline

7. Current-to-Target Mapping

7.1 Content System

Current:

• ResourceManager
• ResourceCache
• AsyncLoader
• ProjectAssetIndex
• AssetImportService
• AssetDatabase

Target:

• ResourceManager -> engine/runtime/content/AssetManager
• ResourceCache -> engine/runtime/content/ResourceStore
• AsyncLoader -> engine/runtime/content/AsyncLoad
• ProjectAssetIndex -> pipeline/core/SourceIndex
• AssetImportService -> pipeline/core/ImportService
• AssetDatabase -> split into pipeline/core/SourceAssetDB and pipeline/core/ArtifactDB

Critical rule:

• runtime asset load path must stop rebuilding source artifacts

7.2 Rendering Cache

Current:

• RenderResourceCache

Target:

• engine/runtime/rendering/gpu_cache

This remains runtime code, but is explicitly recognized as GPU-side cache rather than authoring cache.

7.3 Editor

Editor restructuring is explicitly out of scope for this phase.

The current editor source tree and editor build targets remain unchanged.

8. Hard Architectural Rules

The following rules are non-negotiable:

1. AssetManager must not expose reimport, clear-library, or build-cache APIs.
2. Runtime load paths must consume cooked outputs, not Library internal artifacts as a long-term contract.
3. Library is a pipeline-owned cache, not runtime-owned content.
4. Cross-stage shared code must contain stable data contracts only.
5. New runtime and pipeline features must enter through the correct layer instead of extending the nearest existing folder.

9. Migration Direction

Refactoring should proceed in this order:

1. Split build targets before moving many files.
2. Separate runtime content loading from pipeline import and rebuild logic.
3. Move project-generated outputs under project/Library and project/Content.
4. Align tests with new boundaries.

This sequence matters. File moves without target and dependency separation will only recreate the current coupling under new folder names.