Add shader artifact import pipeline

This commit is contained in:
2026-04-03 14:56:51 +08:00
parent 0f51f553c8
commit d4afa022c1
8 changed files with 1095 additions and 4 deletions

View File

@@ -0,0 +1,383 @@
# Shader 与 Material 系统下一阶段计划
日期:`2026-04-03`
## 1. 阶段结论
当前 Renderer 这一轮主线已经完成收口,但完成的是“基础运行时闭环”,不是“最终的 Unity 风格 shader/material 体系”。
已经完成并应视为当前基线的内容:
- `Shader` 运行时契约已具备 `properties / passes / resources / backend variants`
- builtin shader 资源已经外置,运行时不再依赖 C++ 内嵌 fallback
- `Material` 对 builtin forward 的基础属性解析已经优先走 shader semantic
- `BuiltinForwardPipeline` 已按 shader pass `resources` 合约生成 pipeline layout 与 descriptor binding
- `Shader` 已接入 `AssetDatabase / Library` 流程shader import 会生成 `main.xcshader`
- `ShaderLoader` 已支持加载 `.xcshader` artifact
- shader manifest 依赖与 `Material -> Shader` 依赖已进入资产追踪链
- `rendering_unit_tests`
- `rendering_integration_textured_quad_scene`
- `rendering_integration_backpack_lit_scene`
- `shader_tests`
- `material_tests`
三后端验证当前是稳定的:
- D3D12通过
- OpenGL通过
- Vulkan通过
但这仍然只是“Shader / Material Runtime 的第一层骨架”。
当前真正还没有完成的是:
- Unity 风格的 shader authoring 入口
- 正式的 material schema / instance contract
- 从 shader property layout 到 GPU material layout 的通用映射
- 从 builtin forward 扩展到更通用 pass 的正式执行模型
## 2. 旧计划归档说明
以下两份计划文档已经完成历史使命,应归入 `docs/used/`
- `Renderer模块设计与实现.md`
- `Renderer下一阶段_ShaderMaterial与Pass体系设计.md`
原因不是它们“写错了”,而是:
- 第一份解决的是 Renderer 模块从无到有的问题,当前骨架已经落地
- 第二份解决的是“为什么下一阶段先做 shader/material/pass contract而不是 render graph”的阶段判断这份判断已经部分兑现整体已过期
本文件接手它们之后的主线,只保留对当前 checkout 仍然有效的目标。
## 3. 当前真实问题
### 3.1 Shader 运行时有了,但 authoring 还没有正式化
现在的运行时已经能消费:
- shader property
- pass tag
- pass resource binding
- backend variant
但作者侧仍然不是最终形态。
当前还缺:
- 面向用户的 Unity 风格 `.shader` 语法入口
- import 阶段把 authoring 语法转换为 runtime contract 的正式流程
- shader 资产与 Library artifact 的稳定产物边界
### 3.2 Material 仍偏“资源容器”,还不是正式材质实例系统
当前 `Material` 已有:
- shader 引用
- render state
- property / texture 覆盖
- tag / queue
但它还缺少真正用于 Renderer 执行的正式约束:
- 基于 shader property schema 的类型验证
- property 默认值与 override 的统一解析
- per-pass material constant layout
- texture / sampler / buffer 到 pass resource 的正式映射
- renderer 侧可缓存、可失效、可复用的 material binding plan
### 3.3 现有 pass contract 仍偏 builtin-forward 视角
当前 forward 主链已经能跑,但完整的 pass contract 还没有正式化为可扩展系统。
后续至少要能稳定承接:
- `ForwardLit`
- `Unlit`
- `DepthOnly`
- `ShadowCaster`
- `ObjectId`
否则后面一旦开始做阴影、深度预通道、更多 editor/runtime helper pass就会重新退化回 pipeline 内部的条件分支拼装。
### 3.4 三后端问题的本质不是“语法不同”,而是“资产如何统一”
当前真正要解决的,不是简单回答“到底用 GLSL 还是 HLSL”而是明确三层边界
1. 对外 authoring 语法是什么
2. import 后的内部运行时资产是什么
3. 每个 backend 最终执行的 variant 是什么
这三层不分开,后面一定会把 authoring、runtime、backend 编译链搅在一起。
## 4. 下一阶段的目标
下一阶段只做一件事:把 `Shader``Material` 从“能支撑当前 builtin forward 的运行时拼装”升级为“能长期承接 Unity 风格渲染架构的正式系统”。
### 4.1 对外 authoring 语法目标:严格向 Unity 对齐
最终对外公开的 shader 语法目标,必须与 Unity 的使用方式保持一致。
目标形态应当是:
- 单个 `.shader` 文件作为逻辑 shader 入口
- `Shader / Properties / SubShader / Pass` 的层级结构
- pass 内通过 `HLSLPROGRAM ... ENDHLSL` 或等价块组织代码
- 通过 `#pragma vertex` / `#pragma fragment` 指定 stage 入口
也就是说:
- 对外 authoring 视角应当是“Unity 风格的一体化 shader 文件”
- 不是要求作者直接去维护一堆 runtime JSON manifest
- 也不是让上层逻辑直接感知 D3D12/OpenGL/Vulkan 各自的底层差异
### 4.2 对内 runtime 资产目标:继续保留 contract 模型
虽然 authoring 目标要严格向 Unity 靠拢,但 runtime 不应直接拿 authoring AST 当执行数据。
运行时仍应落到清晰的 contract
```text
Shader Asset
-> Properties
-> Passes
-> Tags
-> Resource Bindings
-> Backend Variants
```
原因很直接:
- Renderer 需要稳定、扁平、可缓存的数据结构
- 三后端最终执行的仍然是 backend variant
- material schema 与 pass binding 都需要基于 import 结果,而不是原始文本
结论是:
- 外部写法要像 Unity
- 内部执行模型继续使用现在这套 runtime contract并进一步完善
### 4.3 Material 目标:从资源对象升级为正式材质实例
Material 下一阶段的核心不是“多支持几个 SetFloat”而是建立正式实例语义。
Material 至少要明确:
- 引用哪个 shader
- 使用哪个 pass 或 pass 策略
- 每个 property 的默认值、覆盖值、类型与序列化规则
- 每个 pass 对应的 material constant block 如何布局
- 每个 texture / sampler / buffer 如何映射到 pass resource
Renderer 侧则必须能把这些内容稳定编译成:
- material binding key
- material constant payload
- descriptor update plan
- pipeline compatibility key
## 5. 推荐架构
### 5.1 分成三层,不混写
推荐明确拆成三层:
```text
Unity-like Shader Authoring (.shader)
-> Shader Importer
-> Runtime Shader Contract
-> Backend Variants / Material Binding Plan
```
三层职责分别是:
- authoring 层:给人写、给 editor 看
- importer 层:把 authoring 语法转成稳定运行时资产
- runtime 层:给 renderer 执行、缓存、绑定、测试
### 5.2 Shader 的 public contract 与 backend contract 分离
下一阶段不建议把“Unity 风格语法”直接等同于“单源码自动跨平台编译已完全成熟”。
更务实的路线是:
- public contract 先统一成 Unity 风格 `.shader`
- importer 先产出统一 runtime contract
- backend variant 暂时仍允许按 backend 持有各自的编译输入或中间产物
这意味着:
- 作者看到的是统一 shader
- Renderer 消费的是统一 runtime contract
- backend 最终执行的仍然可以是不同 variant
这个分层既不违背 Unity 风格目标,也不会过早把工程拖进复杂的全平台 shader 编译链泥潭。
### 5.3 Vertex / Fragment 的外部写法按 Unity 组织,内部可拆分
对外语义上vertex / fragment 应当属于同一个 pass。
也就是说public authoring 角度要符合 Unity
- 一个 shader
- 一个或多个 subshader
- 一个或多个 pass
- 每个 pass 里通过 pragma 指定 vertex / fragment
但内部 import/runtime 完全可以把它们拆成:
- pass descriptor
- vertex stage variant
- fragment stage variant
外部合一,内部拆开,这是最稳妥的做法。
## 6. 下一阶段实施顺序
### 阶段 D0先打通 Shader Import / Artifact 基础设施(已完成)
这是当前 checkout 在 `2026-04-03` 已经完成的新增里程碑。
完成内容:
- `ShaderImporter` 已接管 `.shader / .hlsl / .glsl / .vert / .frag / .geom / .comp`
- shader import 结果会写入 `Library/Artifacts/.../main.xcshader`
- `ShaderLoader` 已支持直接读取 `.xcshader`
- shader manifest 中声明的 stage 源文件会进入依赖追踪
- `Material` 对引用 shader 的直接依赖也已进入依赖追踪
- `shader_tests``material_tests` 已覆盖 shader artifact 生成、加载与重导场景
这一步的意义不是“最终方案已完成”,而是先把 shader 纳入和 texture/material/mesh 一致的资产闭环。
没有这一步,后续不管做 Unity 风格 frontend还是做 material schema都会一直建立在“运行时临时解析源码”的不稳定基础上。
### 阶段 0当前基线确认
这部分已经完成,不再作为待办:
- runtime shader contract 已建立
- builtin forward 已按 pass resources 驱动
- 三后端渲染回归通过
### 阶段 A建立 Unity 风格 Shader Authoring Frontend
目标:
- 新增 Unity 风格 `.shader` authoring 入口
- importer 能解析最小闭环子集
第一批建议支持的子集:
- `Shader`
- `Properties`
- `SubShader`
- `Tags`
- `Pass`
- `HLSLPROGRAM / ENDHLSL`
- `#pragma vertex`
- `#pragma fragment`
交付标准:
- builtin `forward-lit`
- builtin `unlit`
- builtin `object-id`
- builtin `infinite-grid`
至少迁移其中一条主线并成功跑通三后端测试。
### 阶段 B建立正式的 Material Schema 与 Instance Contract
目标:
- shader importer 输出可供 material 消费的 property schema
- material 对 property override 做类型校验与默认值回退
- 为 pass 生成 material constant layout 与 resource mapping
交付标准:
- material 不再只靠 builtin alias 名字兜底
- shader property semantic 变成正式主路径,而不是兼容性补丁
- renderer 能从 shader schema 生成 material binding payload
当前建议把这一阶段作为下一步主线。
原因:
- shader artifact 与依赖追踪已经到位shader 现在可以作为稳定 schema 来源
- material 仍然缺少基于 shader property 的正式类型校验、默认值回退和资源映射
- renderer 目前虽然能消费 pass resources但 material binding 仍偏 builtin-forward 特判
### 阶段 C把 Pass Binding 扩展为正式材质执行链路
目标:
- 不只是 builtin forward 能吃 pass resources
- `Unlit``ObjectId` 也逐步切到同一套 shader/material contract
- pipeline state key 与 descriptor binding plan 从“按功能写逻辑”升级到“按 shader pass contract 解析”
交付标准:
- 至少 `ForwardLit + Unlit + ObjectId` 共用同一套 shader/material 执行边界
- 新增 pass 不再默认要求先写一套新的硬编码 binding 路径
### 阶段 D扩展 AssetDatabase / Library Artifact 能力
目标:
- 在已完成的 shader artifact 基础上,继续扩展 import 产物边界
- backend variant 的编译输入、中间产物或缓存策略进入 `Library/Artifacts`
- 为后续 Unity 风格 `.shader` frontend 预留稳定 importer 输出层
交付标准:
- 资源改动能够稳定触发 shader/material 重新导入
- editor/runtime 读取的是 import 后资产,不是源码临时解析
- shader importer 不再只服务当前 JSON manifest 兼容路径,也能承接下一步 authoring frontend
### 阶段 E测试与回归收口
目标:
- 给 shader importer、material schema、binding plan 增加 unit tests
- 对 forward/unlit/object-id 增加 integration coverage
- 保持 D3D12 / OpenGL / Vulkan 一致回归
最低验证集:
- `shader_tests`
- `material_tests`
- `rendering_unit_tests`
- `rendering_integration_textured_quad_scene`
- `rendering_integration_backpack_lit_scene`
必要时新增:
- `rendering_integration_unlit_scene`
- `rendering_integration_object_id_scene`
## 7. 当前阶段明确不做
下一阶段不应把范围扩散到下面这些方向:
- render graph
- shader graph
- 全平台单源码自动转译一次性做完
- 完整 SRP scripting API
- 大规模后处理框架
这些都依赖 shader/material 正式体系先稳定下来。
## 8. 成功标准
这个阶段完成时,应该满足下面几个判断:
- 作者侧已经可以写 Unity 风格的 `.shader`
- runtime 已不再依赖手写 JSON manifest 才能描述 pass contract
- material 能基于 shader schema 做正式绑定,而不是 builtin 特判兜底
- 至少 `ForwardLit / Unlit / ObjectId` 三类 pass 共用统一 shader/material 执行边界
- 三后端回归测试仍稳定通过
## 9. 一句话总结
下一阶段不是继续给 builtin forward 打补丁,而是把 `Shader``Material` 正式提升为 Unity 风格渲染架构中的稳定中层资产与执行契约。

View File

@@ -12,6 +12,7 @@ namespace Resources {
constexpr Core::uint32 kTextureArtifactSchemaVersion = 1; constexpr Core::uint32 kTextureArtifactSchemaVersion = 1;
constexpr Core::uint32 kMaterialArtifactSchemaVersion = 1; constexpr Core::uint32 kMaterialArtifactSchemaVersion = 1;
constexpr Core::uint32 kMeshArtifactSchemaVersion = 2; constexpr Core::uint32 kMeshArtifactSchemaVersion = 2;
constexpr Core::uint32 kShaderArtifactSchemaVersion = 1;
struct TextureArtifactHeader { struct TextureArtifactHeader {
char magic[8] = { 'X', 'C', 'T', 'E', 'X', '0', '1', '\0' }; char magic[8] = { 'X', 'C', 'T', 'E', 'X', '0', '1', '\0' };
@@ -60,5 +61,38 @@ struct MaterialPropertyArtifact {
MaterialProperty::Value value = {}; MaterialProperty::Value value = {};
}; };
struct ShaderArtifactFileHeader {
char magic[8] = { 'X', 'C', 'S', 'H', 'D', '0', '1', '\0' };
Core::uint32 schemaVersion = kShaderArtifactSchemaVersion;
};
struct ShaderArtifactHeader {
Core::uint32 propertyCount = 0;
Core::uint32 passCount = 0;
};
struct ShaderPassArtifactHeader {
Core::uint32 tagCount = 0;
Core::uint32 resourceCount = 0;
Core::uint32 variantCount = 0;
};
struct ShaderPropertyArtifact {
Core::uint32 propertyType = 0;
};
struct ShaderResourceArtifact {
Core::uint32 resourceType = 0;
Core::uint32 set = 0;
Core::uint32 binding = 0;
};
struct ShaderVariantArtifactHeader {
Core::uint32 stage = 0;
Core::uint32 language = 0;
Core::uint32 backend = 0;
Core::uint64 compiledBinarySize = 0;
};
} // namespace Resources } // namespace Resources
} // namespace XCEngine } // namespace XCEngine

View File

@@ -86,7 +86,7 @@ public:
const Containers::String& GetLibraryRoot() const { return m_libraryRoot; } const Containers::String& GetLibraryRoot() const { return m_libraryRoot; }
private: private:
static constexpr Core::uint32 kCurrentImporterVersion = 3; static constexpr Core::uint32 kCurrentImporterVersion = 4;
void EnsureProjectLayout(); void EnsureProjectLayout();
void LoadSourceAssetDB(); void LoadSourceAssetDB();
@@ -123,6 +123,8 @@ private:
ArtifactRecord& outRecord); ArtifactRecord& outRecord);
bool ImportModelAsset(const SourceAssetRecord& sourceRecord, bool ImportModelAsset(const SourceAssetRecord& sourceRecord,
ArtifactRecord& outRecord); ArtifactRecord& outRecord);
bool ImportShaderAsset(const SourceAssetRecord& sourceRecord,
ArtifactRecord& outRecord);
Containers::String BuildArtifactKey( Containers::String BuildArtifactKey(
const SourceAssetRecord& sourceRecord, const SourceAssetRecord& sourceRecord,
@@ -142,6 +144,8 @@ private:
std::vector<ArtifactDependencyRecord>& outDependencies) const; std::vector<ArtifactDependencyRecord>& outDependencies) const;
bool CollectMaterialDependencies(const Material& material, bool CollectMaterialDependencies(const Material& material,
std::vector<ArtifactDependencyRecord>& outDependencies) const; std::vector<ArtifactDependencyRecord>& outDependencies) const;
bool CollectShaderDependencies(const SourceAssetRecord& sourceRecord,
std::vector<ArtifactDependencyRecord>& outDependencies) const;
Containers::String m_projectRoot; Containers::String m_projectRoot;
Containers::String m_assetsRoot; Containers::String m_assetsRoot;

View File

@@ -16,6 +16,8 @@ public:
bool CanLoad(const Containers::String& path) const override; bool CanLoad(const Containers::String& path) const override;
LoadResult Load(const Containers::String& path, const ImportSettings* settings = nullptr) override; LoadResult Load(const Containers::String& path, const ImportSettings* settings = nullptr) override;
ImportSettings* GetDefaultSettings() const override; ImportSettings* GetDefaultSettings() const override;
bool CollectSourceDependencies(const Containers::String& path,
Containers::Array<Containers::String>& outDependencies) const;
private: private:
ShaderType DetectShaderType(const Containers::String& path, const Containers::String& source); ShaderType DetectShaderType(const Containers::String& path, const Containers::String& source);

View File

@@ -4,7 +4,7 @@
#include <XCEngine/Debug/Logger.h> #include <XCEngine/Debug/Logger.h>
#include <XCEngine/Resources/Material/MaterialLoader.h> #include <XCEngine/Resources/Material/MaterialLoader.h>
#include <XCEngine/Resources/Mesh/MeshLoader.h> #include <XCEngine/Resources/Mesh/MeshLoader.h>
#include <XCEngine/Resources/Shader/Shader.h> #include <XCEngine/Resources/Shader/ShaderLoader.h>
#include <XCEngine/Resources/Texture/TextureLoader.h> #include <XCEngine/Resources/Texture/TextureLoader.h>
#include <algorithm> #include <algorithm>
@@ -406,6 +406,81 @@ bool WriteMaterialArtifactFile(
return static_cast<bool>(output); return static_cast<bool>(output);
} }
bool WriteShaderArtifactFile(const fs::path& artifactPath, const Shader& shader) {
std::ofstream output(artifactPath, std::ios::binary | std::ios::trunc);
if (!output.is_open()) {
return false;
}
ShaderArtifactFileHeader fileHeader;
output.write(reinterpret_cast<const char*>(&fileHeader), sizeof(fileHeader));
WriteString(output, shader.GetName());
WriteString(output, NormalizeArtifactPathString(shader.GetPath()));
ShaderArtifactHeader header;
header.propertyCount = static_cast<Core::uint32>(shader.GetProperties().Size());
header.passCount = shader.GetPassCount();
output.write(reinterpret_cast<const char*>(&header), sizeof(header));
for (const ShaderPropertyDesc& property : shader.GetProperties()) {
WriteString(output, property.name);
WriteString(output, property.displayName);
WriteString(output, property.defaultValue);
WriteString(output, property.semantic);
ShaderPropertyArtifact propertyArtifact;
propertyArtifact.propertyType = static_cast<Core::uint32>(property.type);
output.write(reinterpret_cast<const char*>(&propertyArtifact), sizeof(propertyArtifact));
}
for (const ShaderPass& pass : shader.GetPasses()) {
WriteString(output, pass.name);
ShaderPassArtifactHeader passHeader;
passHeader.tagCount = static_cast<Core::uint32>(pass.tags.Size());
passHeader.resourceCount = static_cast<Core::uint32>(pass.resources.Size());
passHeader.variantCount = static_cast<Core::uint32>(pass.variants.Size());
output.write(reinterpret_cast<const char*>(&passHeader), sizeof(passHeader));
for (const ShaderPassTagEntry& tag : pass.tags) {
WriteString(output, tag.name);
WriteString(output, tag.value);
}
for (const ShaderResourceBindingDesc& resource : pass.resources) {
WriteString(output, resource.name);
WriteString(output, resource.semantic);
ShaderResourceArtifact resourceArtifact;
resourceArtifact.resourceType = static_cast<Core::uint32>(resource.type);
resourceArtifact.set = resource.set;
resourceArtifact.binding = resource.binding;
output.write(reinterpret_cast<const char*>(&resourceArtifact), sizeof(resourceArtifact));
}
for (const ShaderStageVariant& variant : pass.variants) {
ShaderVariantArtifactHeader variantHeader;
variantHeader.stage = static_cast<Core::uint32>(variant.stage);
variantHeader.language = static_cast<Core::uint32>(variant.language);
variantHeader.backend = static_cast<Core::uint32>(variant.backend);
variantHeader.compiledBinarySize = static_cast<Core::uint64>(variant.compiledBinary.Size());
output.write(reinterpret_cast<const char*>(&variantHeader), sizeof(variantHeader));
WriteString(output, variant.entryPoint);
WriteString(output, variant.profile);
WriteString(output, variant.sourceCode);
if (!variant.compiledBinary.Empty()) {
output.write(
reinterpret_cast<const char*>(variant.compiledBinary.Data()),
static_cast<std::streamsize>(variant.compiledBinary.Size()));
}
}
}
return static_cast<bool>(output);
}
bool WriteMeshArtifactFile(const fs::path& artifactPath, bool WriteMeshArtifactFile(const fs::path& artifactPath,
const Mesh& mesh, const Mesh& mesh,
const std::vector<Containers::String>& materialArtifactPaths) { const std::vector<Containers::String>& materialArtifactPaths) {
@@ -489,6 +564,7 @@ void AssetDatabase::Initialize(const Containers::String& projectRoot) {
LoadSourceAssetDB(); LoadSourceAssetDB();
LoadArtifactDB(); LoadArtifactDB();
ScanAssets(); ScanAssets();
SaveArtifactDB();
} }
void AssetDatabase::Shutdown() { void AssetDatabase::Shutdown() {
@@ -996,6 +1072,10 @@ Containers::String AssetDatabase::GetImporterNameForPath(const Containers::Strin
if (ext == ".obj" || ext == ".fbx" || ext == ".gltf" || ext == ".glb" || ext == ".dae" || ext == ".stl") { if (ext == ".obj" || ext == ".fbx" || ext == ".gltf" || ext == ".glb" || ext == ".dae" || ext == ".stl") {
return Containers::String("ModelImporter"); return Containers::String("ModelImporter");
} }
if (ext == ".shader" || ext == ".hlsl" || ext == ".glsl" || ext == ".vert" || ext == ".frag" ||
ext == ".geom" || ext == ".comp") {
return Containers::String("ShaderImporter");
}
if (ext == ".mat" || ext == ".material" || ext == ".json") { if (ext == ".mat" || ext == ".material" || ext == ".json") {
return Containers::String("MaterialImporter"); return Containers::String("MaterialImporter");
} }
@@ -1012,6 +1092,9 @@ ResourceType AssetDatabase::GetPrimaryResourceTypeForImporter(const Containers::
if (importerName == "MaterialImporter") { if (importerName == "MaterialImporter") {
return ResourceType::Material; return ResourceType::Material;
} }
if (importerName == "ShaderImporter") {
return ResourceType::Shader;
}
return ResourceType::Unknown; return ResourceType::Unknown;
} }
@@ -1049,6 +1132,8 @@ bool AssetDatabase::ImportAsset(const SourceAssetRecord& sourceRecord,
return ImportMaterialAsset(sourceRecord, outRecord); return ImportMaterialAsset(sourceRecord, outRecord);
case ResourceType::Mesh: case ResourceType::Mesh:
return ImportModelAsset(sourceRecord, outRecord); return ImportModelAsset(sourceRecord, outRecord);
case ResourceType::Shader:
return ImportShaderAsset(sourceRecord, outRecord);
default: default:
return false; return false;
} }
@@ -1362,6 +1447,59 @@ bool AssetDatabase::ImportModelAsset(const SourceAssetRecord& sourceRecord,
return true; return true;
} }
bool AssetDatabase::ImportShaderAsset(const SourceAssetRecord& sourceRecord,
ArtifactRecord& outRecord) {
ShaderLoader loader;
const Containers::String absolutePath =
NormalizePathString(fs::path(m_projectRoot.CStr()) / sourceRecord.relativePath.CStr());
LoadResult result = loader.Load(absolutePath);
if (!result || result.resource == nullptr) {
return false;
}
Shader* shader = static_cast<Shader*>(result.resource);
std::vector<ArtifactDependencyRecord> dependencies;
if (!CollectShaderDependencies(sourceRecord, dependencies)) {
delete shader;
return false;
}
const Containers::String artifactKey = BuildArtifactKey(sourceRecord, dependencies);
const Containers::String artifactDir = BuildArtifactDirectory(artifactKey);
const Containers::String mainArtifactPath =
NormalizePathString(fs::path(artifactDir.CStr()) / "main.xcshader");
std::error_code ec;
fs::remove_all(fs::path(m_projectRoot.CStr()) / artifactDir.CStr(), ec);
ec.clear();
fs::create_directories(fs::path(m_projectRoot.CStr()) / artifactDir.CStr(), ec);
if (ec) {
delete shader;
return false;
}
const bool writeOk =
WriteShaderArtifactFile(fs::path(m_projectRoot.CStr()) / mainArtifactPath.CStr(), *shader);
delete shader;
if (!writeOk) {
return false;
}
outRecord.artifactKey = artifactKey;
outRecord.assetGuid = sourceRecord.guid;
outRecord.importerName = sourceRecord.importerName;
outRecord.importerVersion = sourceRecord.importerVersion;
outRecord.resourceType = ResourceType::Shader;
outRecord.artifactDirectory = artifactDir;
outRecord.mainArtifactPath = mainArtifactPath;
outRecord.sourceHash = sourceRecord.sourceHash;
outRecord.metaHash = sourceRecord.metaHash;
outRecord.sourceFileSize = sourceRecord.sourceFileSize;
outRecord.sourceWriteTime = sourceRecord.sourceWriteTime;
outRecord.mainLocalID = kMainAssetLocalID;
outRecord.dependencies = std::move(dependencies);
return true;
}
Containers::String AssetDatabase::BuildArtifactKey( Containers::String AssetDatabase::BuildArtifactKey(
const AssetDatabase::SourceAssetRecord& sourceRecord, const AssetDatabase::SourceAssetRecord& sourceRecord,
const std::vector<AssetDatabase::ArtifactDependencyRecord>& dependencies) const { const std::vector<AssetDatabase::ArtifactDependencyRecord>& dependencies) const {
@@ -1528,6 +1666,19 @@ bool AssetDatabase::CollectMaterialDependencies(
outDependencies.clear(); outDependencies.clear();
std::unordered_set<std::string> seenDependencyPaths; std::unordered_set<std::string> seenDependencyPaths;
if (material.GetShader() != nullptr) {
const Containers::String shaderPath = material.GetShader()->GetPath();
if (!shaderPath.Empty() && !HasVirtualPathScheme(shaderPath)) {
ArtifactDependencyRecord dependency;
if (CaptureDependencyRecord(ResolveDependencyPath(shaderPath), dependency)) {
const std::string dependencyKey = ToStdString(dependency.path);
if (seenDependencyPaths.insert(dependencyKey).second) {
outDependencies.push_back(dependency);
}
}
}
}
for (Core::uint32 bindingIndex = 0; bindingIndex < material.GetTextureBindingCount(); ++bindingIndex) { for (Core::uint32 bindingIndex = 0; bindingIndex < material.GetTextureBindingCount(); ++bindingIndex) {
const Containers::String texturePath = material.GetTextureBindingPath(bindingIndex); const Containers::String texturePath = material.GetTextureBindingPath(bindingIndex);
if (texturePath.Empty()) { if (texturePath.Empty()) {
@@ -1548,6 +1699,39 @@ bool AssetDatabase::CollectMaterialDependencies(
return true; return true;
} }
bool AssetDatabase::CollectShaderDependencies(
const SourceAssetRecord& sourceRecord,
std::vector<AssetDatabase::ArtifactDependencyRecord>& outDependencies) const {
outDependencies.clear();
ShaderLoader loader;
const Containers::String absolutePath =
NormalizePathString(fs::path(m_projectRoot.CStr()) / sourceRecord.relativePath.CStr());
Containers::Array<Containers::String> dependencyPaths;
if (!loader.CollectSourceDependencies(absolutePath, dependencyPaths)) {
return false;
}
std::unordered_set<std::string> seenDependencyPaths;
for (const Containers::String& dependencyPath : dependencyPaths) {
if (dependencyPath.Empty() || HasVirtualPathScheme(dependencyPath)) {
continue;
}
ArtifactDependencyRecord dependency;
if (!CaptureDependencyRecord(ResolveDependencyPath(dependencyPath), dependency)) {
continue;
}
const std::string dependencyKey = ToStdString(dependency.path);
if (seenDependencyPaths.insert(dependencyKey).second) {
outDependencies.push_back(dependency);
}
}
return true;
}
Containers::String AssetDatabase::BuildArtifactDirectory(const Containers::String& artifactKey) const { Containers::String AssetDatabase::BuildArtifactDirectory(const Containers::String& artifactKey) const {
if (artifactKey.Length() < 2) { if (artifactKey.Length() < 2) {
return Containers::String("Library/Artifacts/00/invalid"); return Containers::String("Library/Artifacts/00/invalid");

View File

@@ -1,15 +1,18 @@
#include <XCEngine/Resources/Shader/ShaderLoader.h> #include <XCEngine/Resources/Shader/ShaderLoader.h>
#include <XCEngine/Core/Asset/ArtifactFormats.h>
#include <XCEngine/Core/Asset/ResourceManager.h> #include <XCEngine/Core/Asset/ResourceManager.h>
#include <XCEngine/Core/Asset/ResourceTypes.h> #include <XCEngine/Core/Asset/ResourceTypes.h>
#include <XCEngine/Resources/BuiltinResources.h> #include <XCEngine/Resources/BuiltinResources.h>
#include <cctype> #include <cctype>
#include <cstring>
#include <filesystem> #include <filesystem>
#include <fstream> #include <fstream>
#include <functional> #include <functional>
#include <memory> #include <memory>
#include <string> #include <string>
#include <unordered_set>
#include <vector> #include <vector>
namespace XCEngine { namespace XCEngine {
@@ -21,6 +24,10 @@ std::string ToStdString(const Containers::Array<Core::uint8>& data) {
return std::string(reinterpret_cast<const char*>(data.Data()), data.Size()); return std::string(reinterpret_cast<const char*>(data.Data()), data.Size());
} }
std::string ToStdString(const Containers::String& value) {
return std::string(value.CStr());
}
Containers::Array<Core::uint8> TryReadFileData( Containers::Array<Core::uint8> TryReadFileData(
const std::filesystem::path& filePath, const std::filesystem::path& filePath,
bool& opened) { bool& opened) {
@@ -576,6 +583,40 @@ bool ReadTextFile(const Containers::String& path, Containers::String& outText) {
return true; return true;
} }
template<typename T>
bool ReadShaderArtifactValue(const Containers::Array<Core::uint8>& data, size_t& offset, T& outValue) {
if (offset + sizeof(T) > data.Size()) {
return false;
}
std::memcpy(&outValue, data.Data() + offset, sizeof(T));
offset += sizeof(T);
return true;
}
bool ReadShaderArtifactString(const Containers::Array<Core::uint8>& data,
size_t& offset,
Containers::String& outValue) {
Core::uint32 length = 0;
if (!ReadShaderArtifactValue(data, offset, length)) {
return false;
}
if (length == 0) {
outValue.Clear();
return true;
}
if (offset + length > data.Size()) {
return false;
}
outValue = Containers::String(
std::string(reinterpret_cast<const char*>(data.Data() + offset), length).c_str());
offset += length;
return true;
}
bool TryParseUnsignedValue(const std::string& json, const char* key, Core::uint32& outValue) { bool TryParseUnsignedValue(const std::string& json, const char* key, Core::uint32& outValue) {
size_t valuePos = 0; size_t valuePos = 0;
if (!FindValueStart(json, key, valuePos)) { if (!FindValueStart(json, key, valuePos)) {
@@ -645,6 +686,51 @@ bool LooksLikeShaderManifest(const std::string& sourceText) {
sourceText.find("\"passes\"") != std::string::npos; sourceText.find("\"passes\"") != std::string::npos;
} }
bool CollectShaderManifestDependencyPaths(const Containers::String& path,
const std::string& jsonText,
Containers::Array<Containers::String>& outDependencies) {
outDependencies.Clear();
std::string passesArray;
if (!TryExtractArray(jsonText, "passes", passesArray)) {
return false;
}
std::vector<std::string> passObjects;
if (!SplitTopLevelArrayElements(passesArray, passObjects)) {
return false;
}
std::unordered_set<std::string> seenPaths;
for (const std::string& passObject : passObjects) {
std::string variantsArray;
if (!TryExtractArray(passObject, "variants", variantsArray)) {
return false;
}
std::vector<std::string> variantObjects;
if (!SplitTopLevelArrayElements(variantsArray, variantObjects)) {
return false;
}
for (const std::string& variantObject : variantObjects) {
Containers::String sourcePath;
if (!TryParseStringValue(variantObject, "source", sourcePath) &&
!TryParseStringValue(variantObject, "sourcePath", sourcePath)) {
continue;
}
const Containers::String resolvedPath = ResolveShaderDependencyPath(sourcePath, path);
const std::string key = ToStdString(resolvedPath);
if (!key.empty() && seenPaths.insert(key).second) {
outDependencies.PushBack(resolvedPath);
}
}
}
return true;
}
LoadResult LoadShaderManifest(const Containers::String& path, const std::string& jsonText) { LoadResult LoadShaderManifest(const Containers::String& path, const std::string& jsonText) {
std::string passesArray; std::string passesArray;
if (!TryExtractArray(jsonText, "passes", passesArray)) { if (!TryExtractArray(jsonText, "passes", passesArray)) {
@@ -816,6 +902,143 @@ LoadResult LoadShaderManifest(const Containers::String& path, const std::string&
return LoadResult(shader.release()); return LoadResult(shader.release());
} }
LoadResult LoadShaderArtifact(const Containers::String& path) {
const Containers::Array<Core::uint8> data = ReadShaderFileData(path);
if (data.Empty()) {
return LoadResult("Failed to read shader artifact: " + path);
}
size_t offset = 0;
ShaderArtifactFileHeader fileHeader;
if (!ReadShaderArtifactValue(data, offset, fileHeader)) {
return LoadResult("Failed to parse shader artifact header: " + path);
}
const std::string magic(fileHeader.magic, fileHeader.magic + 7);
if (magic != "XCSHD01" || fileHeader.schemaVersion != kShaderArtifactSchemaVersion) {
return LoadResult("Invalid shader artifact header: " + path);
}
auto shader = std::make_unique<Shader>();
Containers::String shaderName;
Containers::String shaderSourcePath;
if (!ReadShaderArtifactString(data, offset, shaderName) ||
!ReadShaderArtifactString(data, offset, shaderSourcePath)) {
return LoadResult("Failed to parse shader artifact strings: " + path);
}
shader->m_name = shaderName.Empty() ? path : shaderName;
shader->m_path = shaderSourcePath.Empty() ? path : shaderSourcePath;
shader->m_guid = ResourceGUID::Generate(shader->m_path);
ShaderArtifactHeader shaderHeader;
if (!ReadShaderArtifactValue(data, offset, shaderHeader)) {
return LoadResult("Failed to parse shader artifact body: " + path);
}
for (Core::uint32 propertyIndex = 0; propertyIndex < shaderHeader.propertyCount; ++propertyIndex) {
ShaderPropertyDesc property = {};
ShaderPropertyArtifact propertyArtifact;
if (!ReadShaderArtifactString(data, offset, property.name) ||
!ReadShaderArtifactString(data, offset, property.displayName) ||
!ReadShaderArtifactString(data, offset, property.defaultValue) ||
!ReadShaderArtifactString(data, offset, property.semantic) ||
!ReadShaderArtifactValue(data, offset, propertyArtifact)) {
return LoadResult("Failed to read shader artifact properties: " + path);
}
property.type = static_cast<ShaderPropertyType>(propertyArtifact.propertyType);
shader->AddProperty(property);
}
for (Core::uint32 passIndex = 0; passIndex < shaderHeader.passCount; ++passIndex) {
Containers::String passName;
ShaderPassArtifactHeader passHeader;
if (!ReadShaderArtifactString(data, offset, passName) ||
!ReadShaderArtifactValue(data, offset, passHeader)) {
return LoadResult("Failed to read shader artifact passes: " + path);
}
ShaderPass pass = {};
pass.name = passName;
shader->AddPass(pass);
for (Core::uint32 tagIndex = 0; tagIndex < passHeader.tagCount; ++tagIndex) {
Containers::String tagName;
Containers::String tagValue;
if (!ReadShaderArtifactString(data, offset, tagName) ||
!ReadShaderArtifactString(data, offset, tagValue)) {
return LoadResult("Failed to read shader artifact pass tags: " + path);
}
shader->SetPassTag(passName, tagName, tagValue);
}
for (Core::uint32 resourceIndex = 0; resourceIndex < passHeader.resourceCount; ++resourceIndex) {
ShaderResourceBindingDesc binding = {};
ShaderResourceArtifact resourceArtifact;
if (!ReadShaderArtifactString(data, offset, binding.name) ||
!ReadShaderArtifactString(data, offset, binding.semantic) ||
!ReadShaderArtifactValue(data, offset, resourceArtifact)) {
return LoadResult("Failed to read shader artifact pass resources: " + path);
}
binding.type = static_cast<ShaderResourceType>(resourceArtifact.resourceType);
binding.set = resourceArtifact.set;
binding.binding = resourceArtifact.binding;
shader->AddPassResourceBinding(passName, binding);
}
for (Core::uint32 variantIndex = 0; variantIndex < passHeader.variantCount; ++variantIndex) {
ShaderStageVariant variant = {};
ShaderVariantArtifactHeader variantHeader;
if (!ReadShaderArtifactValue(data, offset, variantHeader) ||
!ReadShaderArtifactString(data, offset, variant.entryPoint) ||
!ReadShaderArtifactString(data, offset, variant.profile) ||
!ReadShaderArtifactString(data, offset, variant.sourceCode)) {
return LoadResult("Failed to read shader artifact variants: " + path);
}
variant.stage = static_cast<ShaderType>(variantHeader.stage);
variant.language = static_cast<ShaderLanguage>(variantHeader.language);
variant.backend = static_cast<ShaderBackend>(variantHeader.backend);
if (variantHeader.compiledBinarySize > 0) {
if (offset + variantHeader.compiledBinarySize > data.Size()) {
return LoadResult("Shader artifact variant binary payload is truncated: " + path);
}
variant.compiledBinary.Resize(static_cast<size_t>(variantHeader.compiledBinarySize));
std::memcpy(
variant.compiledBinary.Data(),
data.Data() + offset,
static_cast<size_t>(variantHeader.compiledBinarySize));
offset += static_cast<size_t>(variantHeader.compiledBinarySize);
}
shader->AddPassVariant(passName, variant);
}
}
if (shader->GetPassCount() == 1) {
const ShaderPass* defaultPass = shader->FindPass("Default");
if (defaultPass != nullptr && defaultPass->variants.Size() == 1u) {
const ShaderStageVariant& variant = defaultPass->variants[0];
if (variant.backend == ShaderBackend::Generic) {
shader->SetShaderType(variant.stage);
shader->SetShaderLanguage(variant.language);
shader->SetSourceCode(variant.sourceCode);
shader->SetCompiledBinary(variant.compiledBinary);
}
}
}
shader->m_isValid = true;
shader->m_memorySize = CalculateShaderMemorySize(*shader);
return LoadResult(shader.release());
}
LoadResult LoadLegacySingleStageShader(const Containers::String& path, const std::string& sourceText) { LoadResult LoadLegacySingleStageShader(const Containers::String& path, const std::string& sourceText) {
auto shader = std::make_unique<Shader>(); auto shader = std::make_unique<Shader>();
shader->m_path = path; shader->m_path = path;
@@ -856,6 +1079,7 @@ Containers::Array<Containers::String> ShaderLoader::GetSupportedExtensions() con
extensions.PushBack("glsl"); extensions.PushBack("glsl");
extensions.PushBack("hlsl"); extensions.PushBack("hlsl");
extensions.PushBack("shader"); extensions.PushBack("shader");
extensions.PushBack("xcshader");
return extensions; return extensions;
} }
@@ -866,7 +1090,8 @@ bool ShaderLoader::CanLoad(const Containers::String& path) const {
const Containers::String ext = GetExtension(path).ToLower(); const Containers::String ext = GetExtension(path).ToLower();
return ext == "vert" || ext == "frag" || ext == "geom" || return ext == "vert" || ext == "frag" || ext == "geom" ||
ext == "comp" || ext == "glsl" || ext == "hlsl" || ext == "shader"; ext == "comp" || ext == "glsl" || ext == "hlsl" ||
ext == "shader" || ext == "xcshader";
} }
LoadResult ShaderLoader::Load(const Containers::String& path, const ImportSettings* settings) { LoadResult ShaderLoader::Load(const Containers::String& path, const ImportSettings* settings) {
@@ -876,13 +1101,17 @@ LoadResult ShaderLoader::Load(const Containers::String& path, const ImportSettin
return CreateBuiltinShaderResource(path); return CreateBuiltinShaderResource(path);
} }
const Containers::String ext = GetPathExtension(path).ToLower();
if (ext == "xcshader") {
return LoadShaderArtifact(path);
}
const Containers::Array<Core::uint8> data = ReadShaderFileData(path); const Containers::Array<Core::uint8> data = ReadShaderFileData(path);
if (data.Empty()) { if (data.Empty()) {
return LoadResult("Failed to read shader file: " + path); return LoadResult("Failed to read shader file: " + path);
} }
const std::string sourceText = ToStdString(data); const std::string sourceText = ToStdString(data);
const Containers::String ext = GetPathExtension(path).ToLower();
if (ext == "shader" && LooksLikeShaderManifest(sourceText)) { if (ext == "shader" && LooksLikeShaderManifest(sourceText)) {
return LoadShaderManifest(path, sourceText); return LoadShaderManifest(path, sourceText);
} }
@@ -894,6 +1123,32 @@ ImportSettings* ShaderLoader::GetDefaultSettings() const {
return nullptr; return nullptr;
} }
bool ShaderLoader::CollectSourceDependencies(const Containers::String& path,
Containers::Array<Containers::String>& outDependencies) const {
outDependencies.Clear();
if (IsBuiltinShaderPath(path)) {
return true;
}
const Containers::String ext = GetPathExtension(path).ToLower();
if (ext != "shader") {
return true;
}
const Containers::Array<Core::uint8> data = ReadShaderFileData(path);
if (data.Empty()) {
return false;
}
const std::string sourceText = ToStdString(data);
if (!LooksLikeShaderManifest(sourceText)) {
return true;
}
return CollectShaderManifestDependencyPaths(path, sourceText, outDependencies);
}
ShaderType ShaderLoader::DetectShaderType(const Containers::String& path, const Containers::String& source) { ShaderType ShaderLoader::DetectShaderType(const Containers::String& path, const Containers::String& source) {
(void)source; (void)source;
return DetectShaderTypeFromPath(path); return DetectShaderTypeFromPath(path);

View File

@@ -59,6 +59,13 @@ void FlipLastByte(const std::filesystem::path& path) {
ASSERT_TRUE(static_cast<bool>(output)); ASSERT_TRUE(static_cast<bool>(output));
} }
void WriteTextFile(const std::filesystem::path& path, const std::string& contents) {
std::ofstream output(path, std::ios::binary | std::ios::trunc);
ASSERT_TRUE(output.is_open());
output << contents;
ASSERT_TRUE(static_cast<bool>(output));
}
TEST(MaterialLoader, GetResourceType) { TEST(MaterialLoader, GetResourceType) {
MaterialLoader loader; MaterialLoader loader;
EXPECT_EQ(loader.GetResourceType(), ResourceType::Material); EXPECT_EQ(loader.GetResourceType(), ResourceType::Material);
@@ -438,6 +445,84 @@ TEST(MaterialLoader, AssetDatabaseReimportsMaterialWhenTextureDependencyChanges)
fs::remove_all(projectRoot); fs::remove_all(projectRoot);
} }
TEST(MaterialLoader, AssetDatabaseReimportsMaterialWhenShaderDependencyChanges) {
namespace fs = std::filesystem;
using namespace std::chrono_literals;
ResourceManager& manager = ResourceManager::Get();
manager.Initialize();
const fs::path projectRoot = fs::temp_directory_path() / "xc_material_shader_dependency_reimport_test";
const fs::path assetsDir = projectRoot / "Assets";
const fs::path shaderDir = assetsDir / "Shaders";
const fs::path shaderManifestPath = shaderDir / "lit.shader";
const fs::path materialPath = assetsDir / "textured.material";
fs::remove_all(projectRoot);
fs::create_directories(shaderDir);
WriteTextFile(shaderDir / "lit.vert.glsl", "#version 430\nvoid main() {}\n");
WriteTextFile(shaderDir / "lit.frag.glsl", "#version 430\nvoid main() {}\n");
{
std::ofstream manifest(shaderManifestPath);
ASSERT_TRUE(manifest.is_open());
manifest << "{\n";
manifest << " \"name\": \"MaterialDependencyShader\",\n";
manifest << " \"passes\": [\n";
manifest << " {\n";
manifest << " \"name\": \"ForwardLit\",\n";
manifest << " \"variants\": [\n";
manifest << " { \"stage\": \"Vertex\", \"backend\": \"OpenGL\", \"language\": \"GLSL\", \"source\": \"lit.vert.glsl\" },\n";
manifest << " { \"stage\": \"Fragment\", \"backend\": \"OpenGL\", \"language\": \"GLSL\", \"source\": \"lit.frag.glsl\" }\n";
manifest << " ]\n";
manifest << " }\n";
manifest << " ]\n";
manifest << "}\n";
}
{
std::ofstream materialFile(materialPath);
ASSERT_TRUE(materialFile.is_open());
materialFile << "{\n";
materialFile << " \"shader\": \"Assets/Shaders/lit.shader\",\n";
materialFile << " \"shaderPass\": \"ForwardLit\",\n";
materialFile << " \"renderQueue\": \"geometry\"\n";
materialFile << "}\n";
}
manager.SetResourceRoot(projectRoot.string().c_str());
AssetDatabase database;
database.Initialize(projectRoot.string().c_str());
AssetDatabase::ResolvedAsset firstResolve;
ASSERT_TRUE(database.EnsureArtifact("Assets/textured.material", ResourceType::Material, firstResolve));
ASSERT_TRUE(firstResolve.artifactReady);
const String firstArtifactPath = firstResolve.artifactMainPath;
database.Shutdown();
std::this_thread::sleep_for(50ms);
{
std::ofstream manifest(shaderManifestPath, std::ios::app);
ASSERT_TRUE(manifest.is_open());
manifest << "\n";
ASSERT_TRUE(static_cast<bool>(manifest));
}
database.Initialize(projectRoot.string().c_str());
AssetDatabase::ResolvedAsset secondResolve;
ASSERT_TRUE(database.EnsureArtifact("Assets/textured.material", ResourceType::Material, secondResolve));
ASSERT_TRUE(secondResolve.artifactReady);
EXPECT_NE(firstArtifactPath, secondResolve.artifactMainPath);
EXPECT_TRUE(fs::exists(secondResolve.artifactMainPath.CStr()));
database.Shutdown();
manager.SetResourceRoot("");
manager.Shutdown();
fs::remove_all(projectRoot);
}
TEST(MaterialLoader, LoadMaterialArtifactDefersTexturePayloadUntilRequested) { TEST(MaterialLoader, LoadMaterialArtifactDefersTexturePayloadUntilRequested) {
namespace fs = std::filesystem; namespace fs = std::filesystem;

View File

@@ -1,13 +1,16 @@
#include <gtest/gtest.h> #include <gtest/gtest.h>
#include <XCEngine/Core/Asset/AssetDatabase.h>
#include <XCEngine/Resources/BuiltinResources.h> #include <XCEngine/Resources/BuiltinResources.h>
#include <XCEngine/Core/Asset/ResourceManager.h> #include <XCEngine/Core/Asset/ResourceManager.h>
#include <XCEngine/Resources/Shader/ShaderLoader.h> #include <XCEngine/Resources/Shader/ShaderLoader.h>
#include <XCEngine/Core/Asset/ResourceTypes.h> #include <XCEngine/Core/Asset/ResourceTypes.h>
#include <XCEngine/Core/Containers/Array.h> #include <XCEngine/Core/Containers/Array.h>
#include <chrono>
#include <cstdio> #include <cstdio>
#include <filesystem> #include <filesystem>
#include <fstream> #include <fstream>
#include <thread>
using namespace XCEngine::Resources; using namespace XCEngine::Resources;
using namespace XCEngine::Containers; using namespace XCEngine::Containers;
@@ -38,6 +41,7 @@ TEST(ShaderLoader, CanLoad) {
EXPECT_TRUE(loader.CanLoad("test.frag")); EXPECT_TRUE(loader.CanLoad("test.frag"));
EXPECT_TRUE(loader.CanLoad("test.glsl")); EXPECT_TRUE(loader.CanLoad("test.glsl"));
EXPECT_TRUE(loader.CanLoad("test.hlsl")); EXPECT_TRUE(loader.CanLoad("test.hlsl"));
EXPECT_TRUE(loader.CanLoad("test.xcshader"));
EXPECT_TRUE(loader.CanLoad(GetBuiltinForwardLitShaderPath())); EXPECT_TRUE(loader.CanLoad(GetBuiltinForwardLitShaderPath()));
EXPECT_TRUE(loader.CanLoad(GetBuiltinObjectIdShaderPath())); EXPECT_TRUE(loader.CanLoad(GetBuiltinObjectIdShaderPath()));
EXPECT_TRUE(loader.CanLoad(GetBuiltinObjectIdOutlineShaderPath())); EXPECT_TRUE(loader.CanLoad(GetBuiltinObjectIdOutlineShaderPath()));
@@ -285,6 +289,146 @@ TEST(ShaderLoader, ResourceManagerLoadsShaderManifestRelativeToResourceRoot) {
fs::remove_all(projectRoot); fs::remove_all(projectRoot);
} }
TEST(ShaderLoader, AssetDatabaseCreatesShaderArtifactAndLoaderReadsItBack) {
namespace fs = std::filesystem;
const fs::path projectRoot = fs::temp_directory_path() / "xc_shader_artifact_test";
const fs::path shaderDir = projectRoot / "Assets" / "Shaders";
const fs::path manifestPath = shaderDir / "lit.shader";
fs::remove_all(projectRoot);
fs::create_directories(shaderDir);
WriteTextFile(shaderDir / "lit.vert.glsl", "#version 430\n// ARTIFACT_GL_VS\nvoid main() {}\n");
WriteTextFile(shaderDir / "lit.frag.glsl", "#version 430\n// ARTIFACT_GL_PS\nvoid main() {}\n");
{
std::ofstream manifest(manifestPath);
ASSERT_TRUE(manifest.is_open());
manifest << "{\n";
manifest << " \"name\": \"ArtifactShader\",\n";
manifest << " \"properties\": [\n";
manifest << " {\n";
manifest << " \"name\": \"_MainTex\",\n";
manifest << " \"displayName\": \"Main Tex\",\n";
manifest << " \"type\": \"2D\",\n";
manifest << " \"defaultValue\": \"white\",\n";
manifest << " \"semantic\": \"BaseColorTexture\"\n";
manifest << " }\n";
manifest << " ],\n";
manifest << " \"passes\": [\n";
manifest << " {\n";
manifest << " \"name\": \"ForwardLit\",\n";
manifest << " \"tags\": { \"LightMode\": \"ForwardBase\" },\n";
manifest << " \"resources\": [\n";
manifest << " { \"name\": \"BaseColorTexture\", \"type\": \"Texture2D\", \"set\": 3, \"binding\": 0, \"semantic\": \"BaseColorTexture\" }\n";
manifest << " ],\n";
manifest << " \"variants\": [\n";
manifest << " { \"stage\": \"Vertex\", \"backend\": \"OpenGL\", \"language\": \"GLSL\", \"source\": \"lit.vert.glsl\" },\n";
manifest << " { \"stage\": \"Fragment\", \"backend\": \"OpenGL\", \"language\": \"GLSL\", \"source\": \"lit.frag.glsl\" }\n";
manifest << " ]\n";
manifest << " }\n";
manifest << " ]\n";
manifest << "}\n";
}
AssetDatabase database;
database.Initialize(projectRoot.string().c_str());
AssetDatabase::ResolvedAsset resolvedAsset;
ASSERT_TRUE(database.EnsureArtifact("Assets/Shaders/lit.shader", ResourceType::Shader, resolvedAsset));
ASSERT_TRUE(resolvedAsset.artifactReady);
EXPECT_EQ(fs::path(resolvedAsset.artifactMainPath.CStr()).extension().string(), ".xcshader");
EXPECT_TRUE(fs::exists(resolvedAsset.artifactMainPath.CStr()));
ShaderLoader loader;
LoadResult result = loader.Load(resolvedAsset.artifactMainPath.CStr());
ASSERT_TRUE(result);
ASSERT_NE(result.resource, nullptr);
auto* shader = static_cast<Shader*>(result.resource);
ASSERT_NE(shader, nullptr);
EXPECT_TRUE(shader->IsValid());
EXPECT_EQ(shader->GetName(), "ArtifactShader");
EXPECT_EQ(
fs::path(shader->GetPath().CStr()).lexically_normal().generic_string(),
manifestPath.lexically_normal().generic_string());
const ShaderPass* pass = shader->FindPass("ForwardLit");
ASSERT_NE(pass, nullptr);
ASSERT_EQ(pass->variants.Size(), 2u);
ASSERT_EQ(pass->resources.Size(), 1u);
const ShaderStageVariant* fragmentVariant =
shader->FindVariant("ForwardLit", ShaderType::Fragment, ShaderBackend::OpenGL);
ASSERT_NE(fragmentVariant, nullptr);
EXPECT_NE(std::string(fragmentVariant->sourceCode.CStr()).find("ARTIFACT_GL_PS"), std::string::npos);
delete shader;
database.Shutdown();
fs::remove_all(projectRoot);
}
TEST(ShaderLoader, AssetDatabaseReimportsShaderWhenStageDependencyChanges) {
namespace fs = std::filesystem;
using namespace std::chrono_literals;
const fs::path projectRoot = fs::temp_directory_path() / "xc_shader_dependency_reimport_test";
const fs::path shaderDir = projectRoot / "Assets" / "Shaders";
const fs::path manifestPath = shaderDir / "lit.shader";
const fs::path fragmentPath = shaderDir / "lit.frag.glsl";
fs::remove_all(projectRoot);
fs::create_directories(shaderDir);
WriteTextFile(shaderDir / "lit.vert.glsl", "#version 430\nvoid main() {}\n");
WriteTextFile(fragmentPath, "#version 430\nvoid main() {}\n");
{
std::ofstream manifest(manifestPath);
ASSERT_TRUE(manifest.is_open());
manifest << "{\n";
manifest << " \"name\": \"DependencyShader\",\n";
manifest << " \"passes\": [\n";
manifest << " {\n";
manifest << " \"name\": \"ForwardLit\",\n";
manifest << " \"variants\": [\n";
manifest << " { \"stage\": \"Vertex\", \"backend\": \"OpenGL\", \"language\": \"GLSL\", \"source\": \"lit.vert.glsl\" },\n";
manifest << " { \"stage\": \"Fragment\", \"backend\": \"OpenGL\", \"language\": \"GLSL\", \"source\": \"lit.frag.glsl\" }\n";
manifest << " ]\n";
manifest << " }\n";
manifest << " ]\n";
manifest << "}\n";
}
AssetDatabase database;
database.Initialize(projectRoot.string().c_str());
AssetDatabase::ResolvedAsset firstResolve;
ASSERT_TRUE(database.EnsureArtifact("Assets/Shaders/lit.shader", ResourceType::Shader, firstResolve));
ASSERT_TRUE(firstResolve.artifactReady);
const String firstArtifactPath = firstResolve.artifactMainPath;
database.Shutdown();
std::this_thread::sleep_for(50ms);
{
std::ofstream fragmentFile(fragmentPath, std::ios::app);
ASSERT_TRUE(fragmentFile.is_open());
fragmentFile << "\n// force dependency reimport\n";
ASSERT_TRUE(static_cast<bool>(fragmentFile));
}
database.Initialize(projectRoot.string().c_str());
AssetDatabase::ResolvedAsset secondResolve;
ASSERT_TRUE(database.EnsureArtifact("Assets/Shaders/lit.shader", ResourceType::Shader, secondResolve));
ASSERT_TRUE(secondResolve.artifactReady);
EXPECT_NE(firstArtifactPath, secondResolve.artifactMainPath);
EXPECT_TRUE(fs::exists(secondResolve.artifactMainPath.CStr()));
database.Shutdown();
fs::remove_all(projectRoot);
}
TEST(ShaderLoader, LoadBuiltinForwardLitShaderBuildsBackendVariants) { TEST(ShaderLoader, LoadBuiltinForwardLitShaderBuildsBackendVariants) {
ShaderLoader loader; ShaderLoader loader;
LoadResult result = loader.Load(GetBuiltinForwardLitShaderPath()); LoadResult result = loader.Load(GetBuiltinForwardLitShaderPath());