feat(stage-godot): rim light (#2032)

## Description

Adds the Godot Stage rim light pipeline and development observation
tooling.

- Replaces the old avatar glow-only compositor path with a staged
post-process compositor.
- Adds avatar-scoped edge light, glow, material overlay, and final
colour mapping stages.
- Implements a dev observation adapter for exporting
viewport/render-stage artefacts.
- Adds visual verification tooling and baseline docs for Godot Stage
rendering work.
- Documents the technical art workflow and rendering effect behaviour.
This commit is contained in:
Lilia_Chen
2026-07-07 05:21:15 +08:00
committed by GitHub
parent 55e850d71a
commit 7c45aa86bc
25 changed files with 4901 additions and 1565 deletions
@@ -252,6 +252,15 @@ function resolveGodotStageDebugLaunchOptions() {
}
}
function resolveGodotStageProcessEnv(): NodeJS.ProcessEnv {
return {
...process.env,
AIRI_GODOT_STAGE_DEV_MODE: app.isPackaged
? process.env.AIRI_GODOT_STAGE_DEV_MODE ?? '0'
: process.env.AIRI_GODOT_STAGE_DEV_MODE ?? '1',
}
}
interface GodotBinaryResolution {
executable: string
mode: 'engine' | 'exported'
@@ -758,6 +767,7 @@ export function createGodotStageManager(): GodotStageManager {
spawnArgs,
{
cwd: spawnCwd,
env: resolveGodotStageProcessEnv(),
stdio: ['ignore', 'pipe', 'pipe'],
windowsHide: false,
},
@@ -1,6 +1,7 @@
# Godot 4+ specific ignores
.godot/
/android/
/artifacts/
# .NET build artifacts
bin/
+39 -13
View File
@@ -165,23 +165,26 @@ Godot Environment Glow, and leaves Godot tonemap/adjustment neutral. The current
custom color mapping is applied by the stage compositor instead of Godot's
Environment adjustment controls.
## Avatar Glow And Color Mapping
## Avatar Mask, Glow, And Color Mapping
The current avatar presentation path uses a camera-local compositor:
The current avatar presentation path uses stage-owned overlay and compositor
owners:
1. `StageAvatarGlowRuntime` assigns a `Compositor` to the active `Camera3D`.
2. The runtime marks loaded avatar `GeometryInstance3D` nodes with a
depth-tested `MaterialOverlay` that writes stencil reference `1`.
3. `StageAvatarGlowCompositorEffect` runs at `PostTransparent`, extracts
stencil-marked avatar pixels from the resolved scene color, builds the bloom
pyramid, composites avatar glare, and applies the current NAES/toon color
mapping before Godot's neutral output path.
1. `StageRenderEffectsRuntime` wires the active camera and loaded avatar into the
stage render-effect owners.
2. `StageMaterialOverlayOwner` records render-effect source claims and assigns
the shared avatar mask overlay material to loaded avatar `GeometryInstance3D`
nodes.
3. `StageCompositorOwner` installs the stage post-process compositor effect on
the active `Camera3D`.
4. `StagePostProcessCompositorEffect` runs at `PostTransparent` and applies the
ordered scene copy, avatar glow, and final color mapping stages.
This is not Godot Environment Glow and does not use material emission as the
avatar-style glow source. The color mapping currently lives in the same
compositor effect as avatar glow; before adding more post effects such as rim
light or screen-space outlines, pass orchestration and shared transient render
textures should move behind a shared post-process owner.
avatar-style glow source. The color mapping is an independently enabled final
stage inside the stage post-process compositor. Future avatar-only edge light
work should establish same-scene visual comparison artifacts before changing the
pipeline.
Design notes live in [`docs/rendering-effects.md`](docs/rendering-effects.md).
@@ -203,6 +206,29 @@ outline passes, and mesh shadow casters. The current A/B fixtures do not contain
unlit materials, so this check reports `unlit = 0` and does not treat that as a
failure.
## Visual Observation
Renderer-facing changes should produce comparable render artifacts before they
are accepted:
```powershell
pnpm -F @proj-airi/stage-tamagotchi-godot dump:render-stages
```
For rim or edge-light shape checks, prefer the upper-body camera preset:
```powershell
pnpm -F @proj-airi/stage-tamagotchi-godot dump:render-stages:upper-body
```
The command launches the real `StageRoot` scene through the local WebSocket
protocol, loads `AvatarSample_A.vrm`, captures the visible Godot window client
area as the final rendered output, and writes diagnostic render-stage artifacts
under `artifacts/`.
The Technical Art workflow and acceptance rules live in
[`docs/technical-art-workflow.md`](docs/technical-art-workflow.md).
## Live Debugging From The Godot Editor
Use this path when Electron is the real host and the model is selected from the
@@ -23,29 +23,38 @@ CompositorEffect
For avatar glow, this means:
- `StageAvatarGlowRuntime` marks avatar meshes with an overlay material that
depth-tests against the scene and writes stencil reference `1` without writing
scene depth.
- `StageAvatarGlowCompositorEffect` reads the stencil-marked scene color,
extracts the glow source, diffuses it, composites it, and applies the current
toon color mapping.
- `StageRenderEffectsRuntime` wires the stage-level overlay and compositor
owners for the active camera and loaded avatar.
- `StageMaterialOverlayOwner` records render-effect source claims and assigns
the shared avatar mask overlay material. The overlay depth-tests against the
scene and writes stencil reference `1` without writing scene depth.
- `StagePostProcessCompositorEffect` always owns the final NAES/toon color
mapping stage. Its internal pipeline currently runs scene copy, avatar glow,
and final color mapping in that order.
- When avatar glow is active, it extracts bright source from the stencil-marked
scene color, diffuses it, and composites avatar glare into the HDR input
consumed by final color mapping.
- The vendored V-Sekai MToon shader remains responsible for the avatar's base
material look.
Current implementation details:
- `StageAvatarGlowRuntime` currently assigns a new `Compositor` directly to the
active `Camera3D`. This is acceptable while avatar glow is the only stage
compositor, but multiple camera effects need a shared compositor owner.
- `StageAvatarGlowCompositorEffect` currently owns both avatar glow and the
stage toon color mapping. That is an implementation coupling, not the desired
long-term feature boundary.
- The effect owns its transient bloom textures. It uses Godot's
- `StageCompositorOwner` owns the active `Camera3D` compositor slot and registers
the stage post-process compositor effect without feature runtimes replacing
the camera compositor independently.
- `StageMaterialOverlayOwner` owns `GeometryInstance3D.MaterialOverlay` writes
for stage render-effect source passes.
- The compositor effect owns its transient post-process textures. It uses Godot's
`FramebufferCacheRD` and `UniformSetCacheRD` for derived framebuffer and
uniform-set RIDs, while texture ownership stays local to the effect until a
shared post-process resource context exists.
uniform-set RIDs, while texture ownership stays local to the stage
post-process pass graph.
- The final NAES/toon color mapping is a separate compositor stage and no
longer depends on avatar glow being enabled. Later source effects can feed the
same final color stage before display output after same-scene visual
comparison.
- Godot Environment Glow is disabled in `StageVisualPreset`; avatar-style glow
source selection comes from the stencil overlay, not from material emission.
source selection comes from the shared stencil overlay, not from material
emission.
## Why Overlay First
@@ -57,7 +66,7 @@ Use overlay first when an effect can be represented as an extra whole-geometry
pass:
- stencil or object-mask tagging;
- avatar-only glow source selection;
- avatar mask selection for glow and future avatar-only effects;
- silhouette, outline, selection, or interaction masks;
- depth-tested x-ray or rim/shell source passes;
- temporary verification passes that should not mutate imported materials.
@@ -86,11 +95,11 @@ authoring data that the overlay pass can read.
## Known Constraints
- `MaterialOverlay` is a single slot on each `GeometryInstance3D`. Multiple
effects cannot assign it independently without an owner that arbitrates the
slot.
- `Camera3D.Compositor` is also a single owner slot in the current stage code.
Additional post effects should be registered through a shared compositor owner
instead of independently replacing the camera compositor.
effects must go through `StageMaterialOverlayOwner` instead of assigning it
independently.
- `Camera3D.Compositor` is also a single owner slot. Additional post effects
must be registered through the stage compositor owner instead of independently
replacing the camera compositor.
- The overlay material applies to the whole geometry and all surfaces unless
the mesh is split or the overlay shader has reliable mask data.
- Overlay adds draw work for every marked geometry. This is acceptable for the
@@ -102,9 +111,8 @@ authoring data that the overlay pass can read.
- Screen-space diffusion, blur pyramids, and color mapping still belong in a
same-frame compositor. Overlay should produce source information, not replace
the compositor pass graph.
- The current color mapping is tied to `StageAvatarGlowCompositorEffect`.
Splitting color mapping into a stage-level post-process pass should happen
together with the next multi-effect refactor.
- The current final color mapping still lives inside the stage post-process
compositor, but it is an independently enabled stage, not a glow sub-pass.
## Decision Rule
@@ -0,0 +1,144 @@
# Technical Art Visual Workflow
This workflow exists for renderer and Technical Art work where visual judgment
is part of correctness. It is specifically written for the Godot stage runtime.
## Required Evidence
Every renderer-facing change needs comparable evidence before it can be
accepted:
1. Reference evidence.
Use screenshots, Blender captures, shader references, or shipped-game
examples. Record the camera, model, active feature state, and what visual
property the reference is meant to prove.
2. Before/after AIRI evidence.
Capture the current AIRI output before and after changing the renderer when
the question depends on a visual difference.
3. Pipeline evidence.
Capture the relevant intermediate render stages when debugging compositor or
material pipeline behavior. Do not rely on memory or a manually inspected
live viewport alone.
The current render-stage dump command is:
```powershell
pnpm -F @proj-airi/stage-tamagotchi-godot dump:render-stages
```
For rim or edge-light shape checks, prefer the upper-body camera preset:
```powershell
pnpm -F @proj-airi/stage-tamagotchi-godot dump:render-stages:upper-body
```
The dump is end-to-end: it launches the real main stage, loads the tracked
avatar through the WebSocket host path, and captures the visible Godot window
client area. Internal viewport texture readback is not acceptable as final
renderer evidence because it can bypass the final display conversion and
produce darker, higher-contrast PNGs than the actual window.
The command writes:
```text
engines/stage-tamagotchi-godot/artifacts/render-stages/
```
The upper-body variant writes:
```text
engines/stage-tamagotchi-godot/artifacts/render-stages-upper-body/
```
Default outputs are:
- `scene-copy.png`
- `avatar-mask.png`
- `avatar-edge-mask.png`
- `after-avatar-edge-light.png`
- `after-avatar-glow.png`
- `final.png`
- `final-edge-off.png`
These are diagnostic outputs, not accepted baselines. Use them to locate the
first divergent stage before changing the next stage.
This workflow intentionally does not define a project-wide visual baseline yet.
If visual baselines are added later, they need a separate design for scope,
fixture ownership, feature-specific acceptance, and update policy.
## Work Order
1. Define the visual question.
Write the narrow question first: color mapping parity, avatar-only glow,
rim-light shape, outline width, alpha sorting, or another specific property.
Do not start from an implementation idea.
2. Choose the reference and fixture.
Use a tracked fixture when the result should stay reproducible from the
repository. Use a private fixture only for temporary research, and record the
path and limitation in the working notes.
3. Capture the current output.
Dump the relevant render stages and keep the artifact path in the working
notes or PR context. For refactors, this is the behavior that must remain
visually stable.
4. Make one renderer change.
Keep the change aligned to one pipeline stage or one material/source owner
decision. A refactor and a new visual effect should not be accepted in the
same comparison unless there is an explicit reason.
5. Compare before judging.
Dump the same scene after the change and compare the relevant artifacts
before changing code again.
6. Classify the difference.
Put the difference into one bucket before fixing:
- expected visual change
- expected drift caused by the intended change
- regression in an unrelated visual property
- capture instability
- fixture or camera mismatch
7. Record the acceptance.
If the image changes intentionally, record why and keep the artifact path. If
the code change is supposed to preserve behavior, compare the before/after
artifacts and document the result.
## Debug Rules
- Start from the rendered artifact, not from a shader guess.
- Use final window output for visual acceptance. `GetViewport().GetTexture()`
readback is only an intermediate diagnostic and must not become the accepted
final evidence for color, contrast, glow, rim light, or tone-mapping
judgments.
- Compare the same model, camera, viewport size, and frame timing.
- For pipeline work, identify the stage that first diverges before modifying the
next stage.
- For reference-driven compositor work, compare the reference stage output and
Godot stage output side by side. Do not accept the final image until the
intermediate mask and after-stage image explain the final difference.
- For material work, inspect imported material parameters before changing the
post-process shader.
- For color issues, separate scene color, source mask, glow composite, and final
color mapping. Do not merge these explanations into one cause.
- If three local fixes do not explain the visual difference, stop and re-check
the pipeline ownership model before adding another patch.
## Gate For Edge Light Work
Edge light work can start only after these are true:
1. Current render-stage artifacts exist and have been manually inspected.
2. Refactor-only changes have matching before/after output for the relevant
views.
3. The reference case has a documented camera and enabled/disabled comparison.
4. The planned edge-light implementation is behind a toggle or stage boundary so
feature-off output can still be captured and compared.
5. Xiaoer reference stages and Godot render stages have been dumped for the same
visual question: edge mask, after-edge image, and final enabled/disabled
comparison.
This avoids repeating the previous failure mode: changing rendering architecture
and a new visual effect at the same time without proving same-scene parity.
@@ -30,12 +30,13 @@ A visual renderer script should:
3. Load the target avatar or fixture through the same runtime path being tested.
4. Create one active `Camera3D` and use fixed poses.
5. Install the renderer feature runtime being tested. For avatar glow, create
`StageAvatarGlowRuntime(camera)` and call `UseAvatar(avatar)` after loading
the avatar; `StageVisualPreset.Apply(this)` alone does not install the glow
compositor.
`StageRenderEffectsRuntime(camera)` and call `UseAvatar(avatar)` after
loading the avatar; `StageVisualPreset.Apply(this)` alone does not install
the glow compositor.
6. Wait several frames before each capture so imports, materials, and compositor
resources settle.
7. Save PNG captures to `Path.Combine(Path.GetTempPath(), "<check-name>")`.
7. Save PNG captures to `Path.Combine(Path.GetTempPath(), "<check-name>")` only
for intermediate pass diagnostics.
8. Print compact diagnostics that can be compared across runs.
9. Quit with `GetTree().Quit(0)` after the final capture.
@@ -79,9 +80,10 @@ public override void _Process(double delta)
}
```
The capture helper should create the output directory, call
`GetViewport().GetTexture().GetImage()`, save the image, and throw if
`SavePng` fails.
The in-scene capture helper may call `GetViewport().GetTexture().GetImage()` for
intermediate data, but this is not final display output. Do not use viewport
readback PNGs as acceptance evidence for color mapping, contrast, glow, rim
light, or other final-frame judgments.
## Run Command
@@ -93,15 +95,30 @@ C:\Godot_v4.6.2-stable_mono_win64\Godot_v4.6.2-stable_mono_win64.exe `
--scene "res://tests/<check-name>/<checkName>.tscn"
```
For shipped stage visuals, follow this with the main stage verification flow:
For shipped stage visuals, export render-stage artifacts instead of relying on a
temporary manual screenshot:
```powershell
pnpm -F @proj-airi/stage-tamagotchi-godot dump:render-stages
```
For rim or edge-light shape checks, prefer the upper-body camera preset:
```powershell
pnpm -F @proj-airi/stage-tamagotchi-godot dump:render-stages:upper-body
```
The dump command runs this flow:
1. Start a local WebSocket host.
2. Launch the main Godot stage with `-- --airi-ws-url=ws://127.0.0.1:<port>/`.
2. Launch the main Godot stage with a fixed `--resolution`.
3. Wait for `stage.ready`.
4. Send `host.scene.apply` with the target VRM.
4. Send `host.scene.apply` with the tracked AvatarSample A VRM.
5. Wait for `scene.applied`.
6. Bring the Godot window to the front and capture it.
7. Send `host.shutdown`.
6. Capture the visible Godot window client area for each requested render-stage
view.
7. Write diagnostic PNGs under `artifacts/`.
8. Send `host.shutdown`.
## Cleanup
+3 -1
View File
@@ -17,6 +17,8 @@
},
"scripts": {
"build": "dotnet build ./stage-tamagotchi-godot.csproj",
"typecheck": "dotnet build ./stage-tamagotchi-godot.csproj"
"typecheck": "dotnet build ./stage-tamagotchi-godot.csproj",
"dump:render-stages": "node ./tools/dumpRenderStages.mjs --dump-render-stages ./artifacts/render-stages",
"dump:render-stages:upper-body": "node ./tools/dumpRenderStages.mjs --dump-render-stages ./artifacts/render-stages-upper-body --view-preset upper-body"
}
}
@@ -1,4 +1,6 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Text.Json;
using System.Text.Json.Serialization;
using Godot;
@@ -21,6 +23,7 @@ public partial class StageRoot : Node3D
private const string CameraNodeName = "Camera3D";
private const string EditorPreviewRootNodeName = "EditorPreviewRoot";
private const string WebSocketUrlArgumentPrefix = "--airi-ws-url=";
private const int DevRenderExportSettleFrames = 2;
private readonly JsonSerializerOptions _jsonOptions = new()
{
@@ -33,9 +36,14 @@ public partial class StageRoot : Node3D
private StageSceneController _sceneController = null!;
private StageViewController _viewController = null!;
private StageCameraInputController _cameraInputController = null!;
private StageAvatarGlowRuntime _avatarGlowRuntime = null!;
private StageRenderEffectsRuntime _renderEffectsRuntime = null!;
private StageViewRuntime _viewRuntime = null!;
private StageDevObservationAdapter _devObservationAdapter;
private string _activeSceneModelId;
private StageSceneApplyPayload _activeScenePayload;
private StageViewCapturePngRequestPayload _pendingViewCapture;
private int _pendingViewCaptureFrames;
private DevRenderExportRun _pendingDevRenderExport;
private bool _shutdownRequested;
/// <inheritdoc/>
@@ -48,7 +56,7 @@ public partial class StageRoot : Node3D
var camera = ResolveCamera();
_sceneController = new StageSceneController(avatarRoot, new VrmAvatarLoader());
InitializeViewRuntime(avatarRoot, camera);
_avatarGlowRuntime = new StageAvatarGlowRuntime(camera);
_renderEffectsRuntime = new StageRenderEffectsRuntime(camera);
var webSocketUrl = ResolveWebSocketUrl();
if (string.IsNullOrWhiteSpace(webSocketUrl))
@@ -69,6 +77,8 @@ public partial class StageRoot : Node3D
GetTree().Quit();
return;
}
StartDevObservationAdapterIfEnabled();
}
/// <inheritdoc/>
@@ -80,14 +90,18 @@ public partial class StageRoot : Node3D
}
_bridge.Poll();
_devObservationAdapter?.Poll();
_viewRuntime?.Process(delta);
_cameraInputController?.Process(delta);
ProcessPendingViewCapture();
ProcessPendingDevRenderExport();
}
/// <inheritdoc/>
public override void _ExitTree()
{
_avatarGlowRuntime?.Dispose();
_devObservationAdapter?.Dispose();
_renderEffectsRuntime?.Dispose();
}
/// <inheritdoc/>
@@ -98,11 +112,21 @@ public partial class StageRoot : Node3D
private void HandleBridgeOpened()
{
if (_devObservationAdapter != null)
{
_devObservationAdapter.AiriBridgeConnected = true;
}
_bridge.SendEnvelope("stage.ready");
}
private void HandleBridgeClosed(string message)
{
if (_devObservationAdapter != null)
{
_devObservationAdapter.AiriBridgeConnected = false;
}
if (_shutdownRequested)
{
GetTree().Quit();
@@ -179,6 +203,15 @@ public partial class StageRoot : Node3D
case "host.view.request_snapshot":
RequestViewSnapshot(envelope.Payload);
break;
case "host.view.capture_png":
QueueViewPngCapture(envelope.Payload);
break;
case "host.render.set_debug_view":
SetRenderDebugView(envelope.Payload);
break;
case "host.render.set_avatar_edge_light":
SetAvatarEdgeLight(envelope.Payload);
break;
case "host.shutdown":
_shutdownRequested = true;
GetTree().Quit();
@@ -221,11 +254,13 @@ public partial class StageRoot : Node3D
// avatar already loaded.
var avatar = _sceneController.Apply(payload);
_viewController?.UseAvatar(avatar);
_avatarGlowRuntime?.UseAvatar(avatar);
_renderEffectsRuntime?.UseAvatar(avatar);
_viewRuntime?.BootstrapForAvatar();
_activeSceneModelId = payload.ModelId;
}
_activeScenePayload = payload;
_bridge.SendEnvelope("scene.applied", new
{
modelId = payload.ModelId,
@@ -278,6 +313,129 @@ public partial class StageRoot : Node3D
}
}
private void QueueViewPngCapture(JsonElement? payloadElement)
{
if (payloadElement == null)
{
SendViewCaptureError("View PNG capture request payload was empty.");
return;
}
var requestId = StageViewJson.TryReadRequestId(payloadElement.Value);
try
{
if (_pendingViewCapture != null)
{
throw new InvalidOperationException("A viewport PNG capture is already pending.");
}
var payload = StageViewJson.ParseCapturePngRequest(payloadElement.Value);
_pendingViewCapture = payload;
_pendingViewCaptureFrames = payload.SettleFrames;
}
catch (Exception error)
{
SendViewCaptureError(error.Message, requestId);
}
}
private void SetRenderDebugView(JsonElement? payloadElement)
{
if (payloadElement == null)
{
SendRenderDebugViewError("Render debug view request payload was empty.");
return;
}
var requestId = StageViewJson.TryReadRequestId(payloadElement.Value);
try
{
var payload = StageViewJson.ParseRenderDebugViewRequest(payloadElement.Value);
var appliedView = _renderEffectsRuntime.SetDebugView(payload.View);
_bridge.SendEnvelope("stage.render.debug_view", new StageRenderDebugViewPayload(
payload.RequestId,
appliedView
));
}
catch (Exception error)
{
SendRenderDebugViewError(error.Message, requestId);
}
}
private void SetAvatarEdgeLight(JsonElement? payloadElement)
{
if (payloadElement == null)
{
SendAvatarEdgeLightError("Avatar edge-light request payload was empty.");
return;
}
var requestId = StageViewJson.TryReadRequestId(payloadElement.Value);
try
{
var payload = StageViewJson.ParseRenderAvatarEdgeLightRequest(payloadElement.Value);
var enabled = _renderEffectsRuntime.SetAvatarEdgeLightEnabled(payload.Enabled);
_bridge.SendEnvelope("stage.render.avatar_edge_light", new StageRenderAvatarEdgeLightPayload(
payload.RequestId,
enabled
));
}
catch (Exception error)
{
SendAvatarEdgeLightError(error.Message, requestId);
}
}
private void ProcessPendingViewCapture()
{
if (_pendingViewCapture == null)
{
return;
}
if (_pendingViewCaptureFrames > 0)
{
_pendingViewCaptureFrames--;
return;
}
var payload = _pendingViewCapture;
_pendingViewCapture = null;
_pendingViewCaptureFrames = 0;
CaptureViewPng(payload);
}
private void CaptureViewPng(StageViewCapturePngRequestPayload payload)
{
try
{
var image = GetViewport().GetTexture().GetImage();
var directory = Path.GetDirectoryName(payload.Path);
if (!string.IsNullOrWhiteSpace(directory))
{
Directory.CreateDirectory(directory);
}
var saveError = image.SavePng(payload.Path);
if (saveError != Error.Ok)
{
throw new InvalidOperationException($"Godot failed to save viewport PNG: {saveError}.");
}
_bridge.SendEnvelope("stage.view.capture_png", new StageViewCapturePngPayload(
payload.RequestId,
payload.Path,
image.GetWidth(),
image.GetHeight()
));
}
catch (Exception error)
{
SendViewCaptureError(error.Message, payload.RequestId);
}
}
private static string ResolveWebSocketUrl()
{
return ResolveArgumentValue(WebSocketUrlArgumentPrefix);
@@ -313,6 +471,195 @@ public partial class StageRoot : Node3D
_cameraInputController = new StageCameraInputController(_viewRuntime, cameraController);
}
private void StartDevObservationAdapterIfEnabled()
{
if (!StageDevObservationAdapter.IsEnabled())
{
return;
}
_devObservationAdapter = new StageDevObservationAdapter(
_jsonOptions,
ProjectSettings.GlobalizePath("res://"),
new[]
{
StageRenderDebugViewNames.Final,
StageRenderDebugViewNames.SceneCopy,
StageRenderDebugViewNames.AvatarMask,
StageRenderDebugViewNames.AvatarEdgeMask,
StageRenderDebugViewNames.AfterAvatarEdgeLight,
StageRenderDebugViewNames.AfterAvatarGlow,
}
);
_devObservationAdapter.RenderExportRequested += QueueDevRenderExport;
try
{
_devObservationAdapter.Start();
}
catch (Exception error)
{
GD.PushWarning(error.Message);
_devObservationAdapter.Dispose();
_devObservationAdapter = null;
}
}
private void QueueDevRenderExport(StageDevObservationRenderExportRequest request)
{
if (_pendingDevRenderExport != null)
{
_devObservationAdapter?.SendRenderExportError(
request.RequestId,
"export_busy",
"A render export is already pending."
);
return;
}
if (_renderEffectsRuntime == null)
{
_devObservationAdapter?.SendRenderExportError(
request.RequestId,
"not_ready",
"Render effects runtime is not ready."
);
return;
}
foreach (var stage in request.Stages)
{
if (!StageRenderDebugViewNames.TryParse(stage, out _))
{
_devObservationAdapter?.SendRenderExportError(
request.RequestId,
"invalid_payload",
$"Unknown render stage: {stage}."
);
return;
}
}
_pendingDevRenderExport = new DevRenderExportRun(
request,
_renderEffectsRuntime.CurrentDebugView
);
}
private void ProcessPendingDevRenderExport()
{
if (_pendingDevRenderExport == null)
{
return;
}
try
{
if (_pendingDevRenderExport.ActiveStage == null)
{
BeginNextDevRenderExportStage();
return;
}
if (_pendingDevRenderExport.SettleFramesRemaining > 0)
{
_pendingDevRenderExport.SettleFramesRemaining--;
return;
}
CaptureActiveDevRenderExportStage();
if (_pendingDevRenderExport.NextStageIndex >= _pendingDevRenderExport.Request.Stages.Length)
{
CompleteDevRenderExport();
return;
}
BeginNextDevRenderExportStage();
}
catch (Exception error)
{
FailDevRenderExport("export_failed", error.Message);
}
}
private void BeginNextDevRenderExportStage()
{
var stage = _pendingDevRenderExport.Request.Stages[_pendingDevRenderExport.NextStageIndex];
_pendingDevRenderExport.NextStageIndex++;
_renderEffectsRuntime.SetDebugView(stage);
_pendingDevRenderExport.ActiveStage = stage;
_pendingDevRenderExport.SettleFramesRemaining = DevRenderExportSettleFrames;
}
private void CaptureActiveDevRenderExportStage()
{
var stage = _pendingDevRenderExport.ActiveStage;
var outputDir = _pendingDevRenderExport.Request.OutputDir;
Directory.CreateDirectory(outputDir);
var image = GetViewport().GetTexture().GetImage();
var path = Path.Combine(outputDir, $"{stage}.png");
var saveError = image.SavePng(path);
if (saveError != Error.Ok)
{
throw new InvalidOperationException($"Godot failed to save render stage {stage}: {saveError}.");
}
_pendingDevRenderExport.ExportedFiles.Add(new StageDevObservationExportedFile(
stage,
path,
image.GetWidth(),
image.GetHeight()
));
_pendingDevRenderExport.ActiveStage = null;
}
private void CompleteDevRenderExport()
{
var export = _pendingDevRenderExport;
_pendingDevRenderExport = null;
RestoreRenderDebugView(export.RestoreDebugView);
_devObservationAdapter?.SendRenderExportResponse(new StageDevObservationRenderExportResult(
export.Request.RequestId,
export.Request.OutputDir,
export.ExportedFiles.ToArray(),
BuildDevObservationContext()
));
}
private StageDevObservationContext BuildDevObservationContext()
{
return new StageDevObservationContext(
_viewRuntime?.CreateSnapshot("dev-render-observation"),
_activeScenePayload,
_renderEffectsRuntime?.CurrentDebugView,
DateTimeOffset.UtcNow.ToString("O")
);
}
private void FailDevRenderExport(string code, string message)
{
var export = _pendingDevRenderExport;
_pendingDevRenderExport = null;
if (export != null)
{
RestoreRenderDebugView(export.RestoreDebugView);
_devObservationAdapter?.SendRenderExportError(export.Request.RequestId, code, message);
}
}
private void RestoreRenderDebugView(string debugView)
{
try
{
_renderEffectsRuntime?.SetDebugView(debugView);
}
catch (Exception error)
{
GD.PushWarning($"Failed to restore render debug view {debugView}: {error.Message}");
}
}
private void SendSceneError(string message)
{
_bridge.SendEnvelope("scene.error", new
@@ -320,4 +667,62 @@ public partial class StageRoot : Node3D
message,
});
}
private void SendViewCaptureError(string message, string requestId = null)
{
_bridge.SendEnvelope("stage.view.capture_error", new StageViewErrorPayload(
"view-capture-failed",
message,
requestId
));
}
private void SendRenderDebugViewError(string message, string requestId = null)
{
_bridge.SendEnvelope("stage.render.debug_view_error", new StageViewErrorPayload(
"render-debug-view-failed",
message,
requestId
));
}
private void SendAvatarEdgeLightError(string message, string requestId = null)
{
_bridge.SendEnvelope("stage.render.avatar_edge_light_error", new StageViewErrorPayload(
"avatar-edge-light-failed",
message,
requestId
));
}
private sealed class DevRenderExportRun
{
public DevRenderExportRun(StageDevObservationRenderExportRequest request, string restoreDebugView)
{
Request = request;
RestoreDebugView = restoreDebugView;
}
public StageDevObservationRenderExportRequest Request
{
get;
}
public string RestoreDebugView
{
get;
}
public int NextStageIndex
{
get; set;
}
public string ActiveStage
{
get; set;
}
public int SettleFramesRemaining
{
get; set;
}
public List<StageDevObservationExportedFile> ExportedFiles { get; } = new();
}
}
@@ -0,0 +1,463 @@
using System;
using System.Diagnostics;
using System.IO;
using System.Text.Json;
using Godot;
/// <summary>
/// Dev-only localhost adapter for AUV observation tools.
/// </summary>
public sealed class StageDevObservationAdapter : IDisposable
{
private const string EnableEnvironmentVariable = "AIRI_GODOT_STAGE_DEV_MODE";
private const string Transport = "websocket-json";
private const string CapabilityQueryMessageType = "capability.query";
private const string RenderExportStagesMessageType = "render.export_stages";
private const int FallbackPortStart = 43170;
private const int FallbackPortEnd = 43199;
private readonly JsonSerializerOptions _jsonOptions;
private readonly string[] _renderStages;
private readonly int _processId;
private readonly string _projectPath;
private readonly string _token;
private TcpServer _server;
private WebSocketPeer _peer;
private string _instancePath;
private bool _disposed;
public bool AiriBridgeConnected
{
get;
set;
}
public event Action<StageDevObservationRenderExportRequest> RenderExportRequested;
public StageDevObservationAdapter(
JsonSerializerOptions jsonOptions,
string projectPath,
string[] renderStages
)
{
_jsonOptions = jsonOptions;
_projectPath = projectPath;
_renderStages = renderStages;
_processId = Process.GetCurrentProcess().Id;
_token = Guid.NewGuid().ToString("N");
}
public static bool IsEnabled()
{
var rawValue = System.Environment.GetEnvironmentVariable(EnableEnvironmentVariable);
if (string.IsNullOrWhiteSpace(rawValue))
{
return false;
}
var normalizedValue = rawValue.Trim().ToLowerInvariant();
return normalizedValue == "1"
|| normalizedValue == "true"
|| normalizedValue == "yes"
|| normalizedValue == "on";
}
public void Start()
{
if (_disposed || _server != null)
{
return;
}
_server = new TcpServer();
var listenError = _server.Listen(0, "127.0.0.1");
if (listenError != Error.Ok || _server.GetLocalPort() <= 0)
{
_server.Dispose();
_server = new TcpServer();
listenError = ListenOnFallbackPort();
}
if (listenError != Error.Ok)
{
throw new InvalidOperationException($"Failed to start Godot dev observation adapter: {listenError}.");
}
WriteDiscoveryRecord(_server.GetLocalPort());
GD.Print($"AIRI Godot dev observation adapter listening on 127.0.0.1:{_server.GetLocalPort()}.");
}
public void Poll()
{
if (_disposed || _server == null)
{
return;
}
AcceptPendingConnection();
PollPeer();
}
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
CleanupDiscoveryRecord();
if (_peer != null)
{
_peer.Close();
_peer.Dispose();
_peer = null;
}
if (_server != null)
{
_server.Dispose();
_server = null;
}
}
private Error ListenOnFallbackPort()
{
for (var port = FallbackPortStart; port <= FallbackPortEnd; port++)
{
var listenError = _server.Listen((ushort)port, "127.0.0.1");
if (listenError == Error.Ok)
{
return Error.Ok;
}
}
return Error.CantCreate;
}
private void AcceptPendingConnection()
{
if (!_server.IsConnectionAvailable())
{
return;
}
var stream = _server.TakeConnection();
var nextPeer = new WebSocketPeer();
var acceptError = nextPeer.AcceptStream(stream);
if (acceptError != Error.Ok)
{
GD.PushWarning($"AIRI Godot dev observation adapter rejected connection: {acceptError}.");
nextPeer.Dispose();
stream.Dispose();
return;
}
if (_peer != null)
{
_peer.Close();
_peer.Dispose();
}
_peer = nextPeer;
}
private void PollPeer()
{
if (_peer == null)
{
return;
}
_peer.Poll();
switch (_peer.GetReadyState())
{
case WebSocketPeer.State.Open:
DrainPeerMessages();
break;
case WebSocketPeer.State.Closed:
_peer.Dispose();
_peer = null;
break;
}
}
private void DrainPeerMessages()
{
while (_peer.GetAvailablePacketCount() > 0)
{
var rawMessage = _peer.GetPacket().GetStringFromUtf8();
HandleMessage(rawMessage);
}
}
private void HandleMessage(string rawMessage)
{
StageDevObservationRequest request = null;
try
{
request = JsonSerializer.Deserialize<StageDevObservationRequest>(rawMessage, _jsonOptions);
if (request == null || string.IsNullOrWhiteSpace(request.Type))
{
SendError(null, "invalid_request", "Request must include a type.");
return;
}
if (!string.Equals(request.Token, _token, StringComparison.Ordinal))
{
SendError(request.RequestId, "unauthorized", "Request token does not match this Godot dev instance.");
return;
}
switch (request.Type)
{
case CapabilityQueryMessageType:
SendCapabilityResponse(request.RequestId);
break;
case RenderExportStagesMessageType:
QueueRenderExport(request);
break;
default:
SendError(request.RequestId, "unknown_message_type", $"Unsupported message type: {request.Type}.");
break;
}
}
catch (Exception error)
{
SendError(request?.RequestId, "invalid_json", error.Message);
}
}
private void SendCapabilityResponse(string requestId)
{
SendMessage(new
{
type = "capability.query.response",
requestId,
status = "ok",
result = new
{
transport = Transport,
features = new[]
{
CapabilityQueryMessageType,
RenderExportStagesMessageType,
},
renderStages = _renderStages,
cameraPresets = Array.Empty<string>(),
process = new
{
pid = _processId,
projectPath = _projectPath,
airiBridgeConnected = AiriBridgeConnected,
},
},
});
}
private void QueueRenderExport(StageDevObservationRequest request)
{
if (RenderExportRequested == null)
{
SendError(request.RequestId, "not_ready", "No render export handler is available.");
return;
}
StageDevObservationRenderExportPayload payload;
try
{
payload = request.Payload.Deserialize<StageDevObservationRenderExportPayload>(_jsonOptions);
}
catch (Exception error)
{
SendError(request.RequestId, "invalid_payload", error.Message);
return;
}
if (payload == null || string.IsNullOrWhiteSpace(payload.OutputDir))
{
SendError(request.RequestId, "invalid_payload", "Render export payload must include outputDir.");
return;
}
if (payload.Stages == null || payload.Stages.Length == 0)
{
SendError(request.RequestId, "invalid_payload", "Render export payload must include at least one stage.");
return;
}
RenderExportRequested.Invoke(new StageDevObservationRenderExportRequest(
request.RequestId,
payload.OutputDir,
payload.Stages
));
}
public void SendRenderExportResponse(StageDevObservationRenderExportResult result)
{
SendMessage(new
{
type = "render.export_stages.response",
requestId = result.RequestId,
status = "ok",
result = new
{
outputDir = result.OutputDir,
exportedFiles = result.ExportedFiles,
context = result.Context,
},
});
}
public void SendRenderExportError(string requestId, string code, string message)
{
SendMessage(new
{
type = "render.export_stages.response",
requestId,
status = "error",
error = new
{
code,
message,
},
});
}
private void SendError(string requestId, string code, string message)
{
SendMessage(new
{
type = "error",
requestId,
status = "error",
error = new
{
code,
message,
},
});
}
private void SendMessage(object message)
{
if (_peer?.GetReadyState() != WebSocketPeer.State.Open)
{
return;
}
_peer.SendText(JsonSerializer.Serialize(message, _jsonOptions));
}
private void WriteDiscoveryRecord(int port)
{
var discoveryRoot = ResolveDiscoveryRoot();
var instancesDirectory = Path.Combine(discoveryRoot, "instances");
Directory.CreateDirectory(instancesDirectory);
_instancePath = Path.Combine(instancesDirectory, $"{_processId}.json");
var startedAt = DateTimeOffset.UtcNow.ToString("O");
var instanceRecord = new
{
schemaVersion = 1,
kind = "airi-godot-stage-dev-observation-instance",
pid = _processId,
projectPath = _projectPath,
transport = Transport,
endpoint = $"127.0.0.1:{port}",
token = _token,
startedAt,
};
File.WriteAllText(_instancePath, JsonSerializer.Serialize(instanceRecord, _jsonOptions));
var currentRecord = new
{
schemaVersion = 1,
kind = "airi-godot-stage-dev-observation-current",
pid = _processId,
projectPath = _projectPath,
instancePath = _instancePath,
updatedAt = startedAt,
};
File.WriteAllText(
Path.Combine(discoveryRoot, "current.json"),
JsonSerializer.Serialize(currentRecord, _jsonOptions)
);
}
private void CleanupDiscoveryRecord()
{
if (string.IsNullOrWhiteSpace(_instancePath))
{
return;
}
try
{
if (File.Exists(_instancePath))
{
File.Delete(_instancePath);
}
var currentPath = Path.Combine(ResolveDiscoveryRoot(), "current.json");
if (IsCurrentDiscoveryRecord(currentPath))
{
File.Delete(currentPath);
}
}
catch (Exception error)
{
GD.PushWarning($"Failed to clean AIRI Godot dev observation discovery record: {error.Message}");
}
}
private bool IsCurrentDiscoveryRecord(string currentPath)
{
if (!File.Exists(currentPath))
{
return false;
}
var currentRecord = JsonSerializer.Deserialize<StageDevObservationCurrentRecord>(
File.ReadAllText(currentPath),
_jsonOptions
);
return string.Equals(currentRecord?.InstancePath, _instancePath, StringComparison.Ordinal);
}
private static string ResolveDiscoveryRoot()
{
var userProfile = System.Environment.GetFolderPath(System.Environment.SpecialFolder.UserProfile);
if (string.IsNullOrWhiteSpace(userProfile))
{
userProfile = OS.GetUserDataDir();
}
return Path.Combine(userProfile, ".airi", "godot-stage", "dev");
}
private sealed record StageDevObservationRequest(string Type, string RequestId, string Token, JsonElement Payload);
private sealed record StageDevObservationCurrentRecord(string InstancePath);
private sealed record StageDevObservationRenderExportPayload(string OutputDir, string[] Stages);
}
public sealed record StageDevObservationRenderExportRequest(string RequestId, string OutputDir, string[] Stages);
public sealed record StageDevObservationRenderExportResult(
string RequestId,
string OutputDir,
StageDevObservationExportedFile[] ExportedFiles,
StageDevObservationContext Context
);
public sealed record StageDevObservationExportedFile(string Stage, string Path, int Width, int Height);
public sealed record StageDevObservationContext(
StageViewSnapshotPayload ViewSnapshot,
StageSceneApplyPayload Scene,
string RenderDebugView,
string CapturedAt
);
@@ -83,6 +83,125 @@ public static class StageViewJson
return new StageViewSnapshotRequestPayload(requestId);
}
public static StageViewCapturePngRequestPayload ParseCapturePngRequest(JsonElement payload)
{
ExpectObject(payload, "view PNG capture request");
string requestId = null;
string path = null;
var settleFrames = 1;
foreach (var property in payload.EnumerateObject())
{
switch (property.Name)
{
case "requestId":
requestId = ReadRequiredRequestId(property.Value, "View PNG capture requestId");
break;
case "path":
path = ReadRequiredString(property.Value, "View PNG capture path");
break;
case "settleFrames":
settleFrames = ReadNonNegativeInt32(
property.Value,
"View PNG capture settleFrames"
);
break;
default:
throw Invalid($"Unknown view PNG capture request field: {property.Name}.");
}
}
if (requestId == null)
{
throw Invalid("View PNG capture requestId is required.");
}
if (path == null)
{
throw Invalid("View PNG capture path is required.");
}
return new StageViewCapturePngRequestPayload(requestId, path, settleFrames);
}
public static StageRenderDebugViewRequestPayload ParseRenderDebugViewRequest(
JsonElement payload
)
{
ExpectObject(payload, "render debug view request");
string requestId = null;
string view = null;
foreach (var property in payload.EnumerateObject())
{
switch (property.Name)
{
case "requestId":
requestId = ReadRequiredRequestId(
property.Value,
"Render debug view requestId"
);
break;
case "view":
view = ReadRequiredString(property.Value, "Render debug view");
break;
default:
throw Invalid($"Unknown render debug view request field: {property.Name}.");
}
}
if (requestId == null)
{
throw Invalid("Render debug view requestId is required.");
}
if (view == null)
{
throw Invalid("Render debug view is required.");
}
return new StageRenderDebugViewRequestPayload(requestId, view);
}
public static StageRenderAvatarEdgeLightRequestPayload ParseRenderAvatarEdgeLightRequest(
JsonElement payload
)
{
ExpectObject(payload, "render avatar edge-light request");
string requestId = null;
bool? enabled = null;
foreach (var property in payload.EnumerateObject())
{
switch (property.Name)
{
case "requestId":
requestId = ReadRequiredRequestId(
property.Value,
"Render avatar edge-light requestId"
);
break;
case "enabled":
enabled = ReadBoolean(property.Value, "Render avatar edge-light enabled");
break;
default:
throw Invalid($"Unknown render avatar edge-light request field: {property.Name}.");
}
}
if (requestId == null)
{
throw Invalid("Render avatar edge-light requestId is required.");
}
if (enabled == null)
{
throw Invalid("Render avatar edge-light enabled is required.");
}
return new StageRenderAvatarEdgeLightRequestPayload(requestId, enabled.Value);
}
public static StageViewState ParseState(JsonElement payload)
{
ExpectObject(payload, "view state");
@@ -282,6 +401,12 @@ public static class StageViewJson
}
private static string ReadRequiredRequestId(JsonElement payload, string field)
{
var value = ReadRequiredString(payload, field);
return value;
}
private static string ReadRequiredString(JsonElement payload, string field)
{
var value = ReadString(payload, field);
if (string.IsNullOrWhiteSpace(value))
@@ -312,6 +437,27 @@ public static class StageViewJson
return value;
}
private static int ReadNonNegativeInt32(JsonElement payload, string field)
{
var value = ReadInt32(payload, field);
if (value < 0)
{
throw Invalid($"Expected {field} to be non-negative.");
}
return value;
}
private static bool ReadBoolean(JsonElement payload, string field)
{
if (payload.ValueKind != JsonValueKind.True && payload.ValueKind != JsonValueKind.False)
{
throw Invalid($"Expected {field} to be a boolean.");
}
return payload.GetBoolean();
}
private static long ReadInt64(JsonElement payload, string field)
{
if (payload.ValueKind != JsonValueKind.Number || !payload.TryGetInt64(out var value))
@@ -71,6 +71,25 @@ public sealed record StageViewPatchRequestPayload(string RequestId, StageViewPat
/// </summary>
public sealed record StageViewSnapshotRequestPayload(string RequestId);
/// <summary>
/// Host-origin diagnostic request to save the current stage viewport as a PNG.
/// </summary>
public sealed record StageViewCapturePngRequestPayload(
string RequestId,
string Path,
int SettleFrames
);
/// <summary>
/// Host-origin request to display a render pipeline diagnostic stage in the final window.
/// </summary>
public sealed record StageRenderDebugViewRequestPayload(string RequestId, string View);
/// <summary>
/// Host-origin diagnostic request to enable or bypass the avatar edge-light stage.
/// </summary>
public sealed record StageRenderAvatarEdgeLightRequestPayload(string RequestId, bool Enabled);
/// <summary>
/// Snapshot emitted by Godot after load, mutation, local input, or request.
/// </summary>
@@ -89,3 +108,23 @@ public sealed record StageViewErrorPayload(
string Message,
string RequestId = null
);
/// <summary>
/// Diagnostic PNG capture emitted after Godot writes the current viewport image.
/// </summary>
public sealed record StageViewCapturePngPayload(
string RequestId,
string Path,
int Width,
int Height
);
/// <summary>
/// Acknowledgement emitted after Godot applies a render pipeline diagnostic stage.
/// </summary>
public sealed record StageRenderDebugViewPayload(string RequestId, string View);
/// <summary>
/// Acknowledgement emitted after Godot applies the avatar edge-light diagnostic toggle.
/// </summary>
public sealed record StageRenderAvatarEdgeLightPayload(string RequestId, bool Enabled);
@@ -98,6 +98,21 @@ public sealed class StageViewRuntime
EmitSnapshot("loaded");
}
public StageViewSnapshotPayload CreateSnapshot(string reason, string requestId = null)
{
if (!_hasViewState)
{
return null;
}
return new StageViewSnapshotPayload(
_state,
reason,
requestId,
_controller.ResolveAvatarBounds()
);
}
public void EmitInvalidPayload(string message, string requestId = null)
{
EmitError("invalid-payload", message, requestId);
@@ -155,12 +170,11 @@ public sealed class StageViewRuntime
private void EmitSnapshot(string reason, string requestId = null)
{
SnapshotReady?.Invoke(new StageViewSnapshotPayload(
_state,
reason,
requestId,
_controller.ResolveAvatarBounds()
));
var snapshot = CreateSnapshot(reason, requestId);
if (snapshot != null)
{
SnapshotReady?.Invoke(snapshot);
}
}
private void EmitError(string code, string message, string requestId = null)
@@ -1,119 +0,0 @@
using System;
using System.Collections.Generic;
using Godot;
/// <summary>
/// Owns avatar-only stencil masking and the camera-local glow compositor.
/// </summary>
public sealed class StageAvatarGlowRuntime : IDisposable
{
private const int AvatarStencilReference = 1;
private readonly Camera3D _camera;
private readonly StageAvatarGlowCompositorEffect _avatarGlowEffect;
private readonly StandardMaterial3D _avatarGlowMaskMaterial;
private readonly Dictionary<GeometryInstance3D, Material> _previousOverlays = new();
private bool _disposed;
public StageAvatarGlowRuntime(Camera3D camera)
{
_camera = camera ?? throw new ArgumentNullException(nameof(camera));
_avatarGlowEffect = new StageAvatarGlowCompositorEffect(AvatarStencilReference);
_avatarGlowMaskMaterial = CreateAvatarGlowMaskMaterial();
// NOTICE:
// Camera3D owns compositor effects. Avatar glow is the only stage compositor today;
// replace this direct assignment with a shared owner before adding more camera passes.
var compositor = new Compositor();
compositor.CompositorEffects = new Godot.Collections.Array<CompositorEffect>
{
_avatarGlowEffect,
};
_camera.Compositor = compositor;
}
public void UseAvatar(Node avatar)
{
if (_disposed)
{
return;
}
ClearAvatarMask();
if (avatar == null)
{
_avatarGlowEffect.Enabled = false;
return;
}
MarkAvatarMask(avatar);
_avatarGlowEffect.Enabled = _previousOverlays.Count > 0;
}
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
_avatarGlowEffect.Enabled = false;
ClearAvatarMask();
if (_camera.Compositor?.CompositorEffects.Contains(_avatarGlowEffect) == true)
{
_camera.Compositor = null;
}
_avatarGlowEffect.ReleaseRenderingResources();
}
private StandardMaterial3D CreateAvatarGlowMaskMaterial() => new()
{
AlbedoColor = new Color(0.0f, 0.0f, 0.0f, 0.0f),
CullMode = BaseMaterial3D.CullModeEnum.Disabled,
// NOTICE:
// Keep the mask depth-tested but non-writing: occluded avatar fragments do not mark
// stencil, and the mask pass does not mutate scene depth.
DepthDrawMode = BaseMaterial3D.DepthDrawModeEnum.Disabled,
DisableFog = true,
NoDepthTest = false,
RenderPriority = (int)Material.RenderPriorityMax,
ShadingMode = BaseMaterial3D.ShadingModeEnum.Unshaded,
StencilCompare = BaseMaterial3D.StencilCompareEnum.Always,
StencilFlags = (int)BaseMaterial3D.StencilFlagsEnum.Write,
StencilMode = BaseMaterial3D.StencilModeEnum.Custom,
StencilReference = AvatarStencilReference,
Transparency = BaseMaterial3D.TransparencyEnum.Alpha,
};
private void MarkAvatarMask(Node node)
{
if (node is GeometryInstance3D geometry)
{
// NOTICE:
// MaterialOverlay is a single instance-level extra pass. Save and restore it while
// this runtime uses the slot as a temporary avatar mask producer.
_previousOverlays[geometry] = geometry.MaterialOverlay;
geometry.MaterialOverlay = _avatarGlowMaskMaterial;
}
foreach (Node child in node.GetChildren())
{
MarkAvatarMask(child);
}
}
private void ClearAvatarMask()
{
foreach (var (geometry, previousOverlay) in _previousOverlays)
{
if (GodotObject.IsInstanceValid(geometry))
{
geometry.MaterialOverlay = previousOverlay;
}
}
_previousOverlays.Clear();
}
}
@@ -0,0 +1,65 @@
using System;
using Godot;
/// <summary>
/// Owns the camera-local compositor slot for stage post-process effects.
/// </summary>
public sealed class StageCompositorOwner : IDisposable
{
private readonly Camera3D _camera;
private readonly Compositor _ownedCompositor;
private readonly Compositor _previousCompositor;
private bool _disposed;
public StageCompositorOwner(Camera3D camera, CompositorEffect stageEffect)
{
_camera = camera ?? throw new ArgumentNullException(nameof(camera));
_previousCompositor = camera.Compositor;
_ownedCompositor = new Compositor
{
CompositorEffects = CreateCompositorEffects(_previousCompositor, stageEffect),
};
_camera.Compositor = _ownedCompositor;
}
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
if (GodotObject.IsInstanceValid(_camera) && _camera.Compositor == _ownedCompositor)
{
_camera.Compositor = _previousCompositor;
}
}
private static Godot.Collections.Array<CompositorEffect> CreateCompositorEffects(
Compositor previousCompositor,
CompositorEffect stageEffect
)
{
if (stageEffect == null)
{
throw new ArgumentNullException(nameof(stageEffect));
}
var effects = new Godot.Collections.Array<CompositorEffect>();
if (previousCompositor?.CompositorEffects != null)
{
foreach (CompositorEffect effect in previousCompositor.CompositorEffects)
{
if (effect != null && effect != stageEffect)
{
effects.Add(effect);
}
}
}
effects.Add(stageEffect);
return effects;
}
}
@@ -0,0 +1,158 @@
using System;
using System.Collections.Generic;
using Godot;
/// <summary>
/// Identifies stage overlay source producers that compete for the single MaterialOverlay slot.
/// </summary>
public enum StageMaterialOverlaySourceKind
{
AvatarMask,
}
/// <summary>
/// Owns stage material overlay assignments used by render-effect source passes.
/// </summary>
public sealed class StageMaterialOverlayOwner : IDisposable
{
private readonly Material _avatarSourceOverlayMaterial;
private readonly Dictionary<StageMaterialOverlaySourceKind, StageOverlaySourceClaim> _claims = new();
private readonly Dictionary<GeometryInstance3D, Material> _previousOverlays = new();
private bool _disposed;
public StageMaterialOverlayOwner(int avatarStencilReference)
{
_avatarSourceOverlayMaterial = CreateAvatarSourceOverlayMaterial(avatarStencilReference);
}
public bool HasSource(StageMaterialOverlaySourceKind sourceKind) => _claims.ContainsKey(sourceKind);
public void UseAvatarMask(Node avatar)
{
if (_disposed)
{
return;
}
if (avatar == null)
{
ClearSource(StageMaterialOverlaySourceKind.AvatarMask);
return;
}
_claims[StageMaterialOverlaySourceKind.AvatarMask] = new StageOverlaySourceClaim(
avatar,
_avatarSourceOverlayMaterial
);
RefreshOverlayAssignments();
}
public void ClearSource(StageMaterialOverlaySourceKind sourceKind)
{
if (!_claims.Remove(sourceKind))
{
return;
}
RefreshOverlayAssignments();
}
public void ClearAllSources()
{
_claims.Clear();
ClearOverlayAssignments();
}
private void ClearOverlayAssignments()
{
foreach (var (geometry, previousOverlay) in _previousOverlays)
{
if (GodotObject.IsInstanceValid(geometry))
{
geometry.MaterialOverlay = previousOverlay;
}
}
_previousOverlays.Clear();
}
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
ClearAllSources();
}
private static StandardMaterial3D CreateAvatarSourceOverlayMaterial(int avatarStencilReference) =>
new()
{
AlbedoColor = new Color(0.0f, 0.0f, 0.0f, 0.0f),
CullMode = BaseMaterial3D.CullModeEnum.Disabled,
// NOTICE:
// Keep the mask depth-tested but non-writing: occluded avatar fragments do not mark
// stencil, and the mask pass does not mutate scene depth.
DepthDrawMode = BaseMaterial3D.DepthDrawModeEnum.Disabled,
DisableFog = true,
NoDepthTest = false,
RenderPriority = (int)Material.RenderPriorityMax,
ShadingMode = BaseMaterial3D.ShadingModeEnum.Unshaded,
StencilCompare = BaseMaterial3D.StencilCompareEnum.Always,
StencilFlags = (int)BaseMaterial3D.StencilFlagsEnum.Write,
StencilMode = BaseMaterial3D.StencilModeEnum.Custom,
StencilReference = avatarStencilReference,
Transparency = BaseMaterial3D.TransparencyEnum.Alpha,
};
private void RefreshOverlayAssignments()
{
ClearOverlayAssignments();
if (_claims.Count == 0)
{
return;
}
// NOTICE:
// Godot exposes only one MaterialOverlay slot per GeometryInstance3D. Until the
// overlay material encodes multiple source channels, the selected claim owns the
// physical overlay assignment for this refresh.
var claim = SelectOverlayClaim();
MarkSource(claim.Root, claim.OverlayMaterial);
}
private StageOverlaySourceClaim SelectOverlayClaim()
{
if (_claims.TryGetValue(StageMaterialOverlaySourceKind.AvatarMask, out var avatarMaskClaim))
{
return avatarMaskClaim;
}
throw new InvalidOperationException("Stage overlay owner had no selectable source claim.");
}
private void MarkSource(Node node, Material overlayMaterial)
{
if (node is GeometryInstance3D geometry
&& !_previousOverlays.ContainsKey(geometry))
{
// NOTICE:
// MaterialOverlay is a single instance-level extra pass. This owner is the only
// stage runtime allowed to occupy the slot for render-effect source passes.
_previousOverlays[geometry] = geometry.MaterialOverlay;
geometry.MaterialOverlay = overlayMaterial;
}
foreach (Node child in node.GetChildren())
{
MarkSource(child, overlayMaterial);
}
}
private readonly record struct StageOverlaySourceClaim(
Node Root,
Material OverlayMaterial
);
}
@@ -0,0 +1,283 @@
using System;
using Godot;
public partial class StagePostProcessCompositorEffect
{
private void RunPipeline()
{
foreach (var stage in PipelineStages)
{
DrawStage(stage);
}
DrawDebugOutputStage();
}
private void DrawStage(StageRenderPipelineStage stage)
{
switch (stage)
{
case StageRenderPipelineStage.SceneCopy:
DrawSceneCopyStage();
break;
case StageRenderPipelineStage.AvatarMask:
DrawAvatarMaskStage();
break;
case StageRenderPipelineStage.AvatarEdgeLight:
DrawAvatarEdgeLightStage();
break;
case StageRenderPipelineStage.AvatarGlow:
DrawAvatarGlowStage();
break;
case StageRenderPipelineStage.FinalColorMapping:
DrawFinalColorMappingStage();
break;
default:
throw new ArgumentOutOfRangeException(nameof(stage), stage, null);
}
}
private void DrawSceneCopyStage()
{
DrawPass(
_resources.SceneSourceFramebuffer,
_resources.SceneCopyPipeline,
_resources.SceneSourceUniformSet,
_resources.SceneCopyPushConstants,
clearColor: false,
preserveDepthStencil: false
);
}
private void DrawAvatarMaskStage()
{
if (!_resources.IncludesAvatarMask)
{
return;
}
DrawPass(
_resources.AvatarMaskFramebuffer,
_resources.AvatarMaskPipeline,
_resources.AvatarMaskUniformSet,
_resources.AvatarMaskPushConstants,
clearColor: true,
preserveDepthStencil: true
);
if (_resources.AvatarMaskResolveRequired)
{
var resolveError = _renderingDevice.TextureResolveMultisample(
_resources.AvatarMaskRenderTexture,
_resources.AvatarMaskTexture
);
if (resolveError != Error.Ok)
{
WarnOnce(
ref _avatarMaskResolveWarningPrinted,
$"Avatar mask MSAA resolve failed: {resolveError}."
);
}
}
}
private void DrawAvatarEdgeLightStage()
{
if (!_resources.IncludesAvatarEdgeLight)
{
return;
}
DrawPass(
_resources.AvatarEdgeLightFramebuffer,
_resources.AvatarEdgeLightPipeline,
_resources.AvatarEdgeLightUniformSet,
_resources.AvatarEdgeLightPushConstants,
clearColor: true,
preserveDepthStencil: false
);
}
private void DrawAvatarGlowStage()
{
if (!_resources.IncludesAvatarMask)
{
return;
}
DrawPass(
_resources.SourceFramebuffer,
_resources.ExtractPipeline,
_resources.SourceUniformSet,
_resources.ExtractPushConstants,
clearColor: true,
preserveDepthStencil: true
);
if (_resources.SourceResolveRequired)
{
var resolveError = _renderingDevice.TextureResolveMultisample(
_resources.SourceRenderTexture,
_resources.SourceTexture
);
if (resolveError != Error.Ok)
{
WarnOnce(
ref _sourceResolveWarningPrinted,
$"Avatar glow source MSAA resolve failed: {resolveError}."
);
return;
}
}
for (int level = 0; level < _resources.BloomLevels; level++)
{
DrawPass(
_resources.DownsampleFramebuffers[level],
_resources.DownsamplePipeline,
_resources.DownsampleUniformSets[level],
_resources.DownsamplePushConstants[level],
clearColor: true,
preserveDepthStencil: false
);
}
for (int level = _resources.BloomLevels - 2; level >= 0; level--)
{
DrawPass(
_resources.UpsampleFramebuffers[level],
_resources.UpsamplePipeline,
_resources.UpsampleUniformSets[level],
_resources.UpsamplePushConstants[level],
clearColor: true,
preserveDepthStencil: false
);
}
DrawPass(
_resources.GlowCompositeFramebuffer,
_resources.GlowCompositePipeline,
_resources.GlowCompositeUniformSet,
_resources.GlowCompositePushConstants,
clearColor: true,
preserveDepthStencil: false
);
}
private void DrawFinalColorMappingStage()
{
DrawPass(
_resources.FinalColorFramebuffer,
_resources.FinalColorPipeline,
_resources.FinalColorUniformSet,
_resources.FinalColorPushConstants,
clearColor: false,
preserveDepthStencil: false
);
}
private void DrawDebugOutputStage()
{
if (_debugView == StageRenderDebugView.Final)
{
return;
}
var uniformSet = ResolveDebugOutputUniformSet();
if (!uniformSet.IsValid)
{
return;
}
DrawPass(
_resources.FinalColorFramebuffer,
_resources.DebugOutputPipeline,
uniformSet,
Array.Empty<byte>(),
clearColor: false,
preserveDepthStencil: false
);
}
private Rid ResolveDebugOutputUniformSet()
{
return _debugView switch
{
StageRenderDebugView.SceneCopy => _resources.DebugSceneCopyUniformSet,
StageRenderDebugView.AvatarMask => _resources.DebugAvatarMaskUniformSet,
StageRenderDebugView.AvatarEdgeMask => _resources.DebugAvatarEdgeLightUniformSet,
StageRenderDebugView.AfterAvatarEdgeLight => _resources.DebugAvatarEdgeLightUniformSet,
StageRenderDebugView.AfterAvatarGlow => _resources.DebugGlowCompositeUniformSet,
_ => new Rid(),
};
}
}
internal enum StageRenderPipelineStage
{
SceneCopy,
AvatarMask,
AvatarEdgeLight,
AvatarGlow,
FinalColorMapping,
}
internal enum StageRenderDebugView
{
Final,
SceneCopy,
AvatarMask,
AvatarEdgeMask,
AfterAvatarEdgeLight,
AfterAvatarGlow,
}
internal static class StageRenderDebugViewNames
{
public const string Final = "final";
public const string SceneCopy = "scene-copy";
public const string AvatarMask = "avatar-mask";
public const string AvatarEdgeMask = "avatar-edge-mask";
public const string AfterAvatarEdgeLight = "after-avatar-edge-light";
public const string AfterAvatarGlow = "after-avatar-glow";
public static bool TryParse(string value, out StageRenderDebugView debugView)
{
switch (value)
{
case Final:
debugView = StageRenderDebugView.Final;
return true;
case SceneCopy:
debugView = StageRenderDebugView.SceneCopy;
return true;
case AvatarMask:
debugView = StageRenderDebugView.AvatarMask;
return true;
case AvatarEdgeMask:
debugView = StageRenderDebugView.AvatarEdgeMask;
return true;
case AfterAvatarEdgeLight:
debugView = StageRenderDebugView.AfterAvatarEdgeLight;
return true;
case AfterAvatarGlow:
debugView = StageRenderDebugView.AfterAvatarGlow;
return true;
default:
debugView = StageRenderDebugView.Final;
return false;
}
}
public static string ToTransportValue(StageRenderDebugView debugView)
{
return debugView switch
{
StageRenderDebugView.Final => Final,
StageRenderDebugView.SceneCopy => SceneCopy,
StageRenderDebugView.AvatarMask => AvatarMask,
StageRenderDebugView.AvatarEdgeMask => AvatarEdgeMask,
StageRenderDebugView.AfterAvatarEdgeLight => AfterAvatarEdgeLight,
StageRenderDebugView.AfterAvatarGlow => AfterAvatarGlow,
_ => Final,
};
}
}
@@ -0,0 +1,496 @@
public partial class StagePostProcessCompositorEffect
{
private const string FullscreenVertexShaderCode = """
#version 450
layout(location = 0) in vec2 position;
layout(location = 0) out vec2 uv;
void main()
{
uv = position * 0.5 + vec2(0.5);
gl_Position = vec4(position, 0.0, 1.0);
}
""";
private const string CopySceneFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D scene_texture;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
void main()
{
out_color = texture(scene_texture, uv);
}
""";
private const string AvatarMaskFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D scene_texture;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
void main()
{
out_color = vec4(1.0);
}
""";
private const string EdgeLightFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D scene_texture;
layout(set = 0, binding = 1) uniform sampler2D depth_texture;
layout(set = 0, binding = 2) uniform sampler2D normal_roughness_texture;
layout(set = 0, binding = 3) uniform sampler2D avatar_mask_texture;
layout(push_constant, std430) uniform Params
{
vec4 edge0;
vec4 edge1;
vec4 edge2;
} params;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
vec4 normal_roughness_compatibility(vec4 normal_roughness)
{
float roughness = normal_roughness.w;
if (roughness > 0.5)
{
roughness = 1.0 - roughness;
}
roughness /= 127.0 / 255.0;
return vec4(normalize(normal_roughness.xyz * 2.0 - 1.0) * 0.5 + 0.5, roughness);
}
float sample_mask(vec2 sample_uv)
{
return clamp(texture(avatar_mask_texture, sample_uv).r, 0.0, 1.0);
}
float linearize_reverse_z_depth(float depth, float near_plane, float far_plane)
{
return (near_plane * far_plane) / max(near_plane + depth * (far_plane - near_plane), 0.00001);
}
float sample_linear_depth(vec2 sample_uv, float near_plane, float far_plane)
{
float depth = texture(depth_texture, clamp(sample_uv, vec2(0.0), vec2(1.0))).r;
return linearize_reverse_z_depth(depth, near_plane, far_plane);
}
float sample_filtered_displaced_linear_depth(
vec2 sample_uv,
vec2 offset_uv,
vec2 pixel_size,
float near_plane,
float far_plane
)
{
vec2 major_axis = length(offset_uv) > 0.000001
? normalize(offset_uv)
: vec2(1.0, 0.0);
vec2 minor_axis = vec2(-major_axis.y, major_axis.x);
vec2 filter_major = major_axis * pixel_size;
vec2 filter_minor = minor_axis * pixel_size;
float filtered_depth = sample_linear_depth(sample_uv, near_plane, far_plane) * 0.5;
filtered_depth += sample_linear_depth(sample_uv + filter_major, near_plane, far_plane) * 0.125;
filtered_depth += sample_linear_depth(sample_uv - filter_major, near_plane, far_plane) * 0.125;
filtered_depth += sample_linear_depth(sample_uv + filter_minor, near_plane, far_plane) * 0.125;
filtered_depth += sample_linear_depth(sample_uv - filter_minor, near_plane, far_plane) * 0.125;
return filtered_depth;
}
void main()
{
vec4 scene = texture(scene_texture, uv);
float avatar_mask = sample_mask(uv);
float debug_edge_mask = params.edge2.x;
if (avatar_mask <= 0.001)
{
if (debug_edge_mask > 0.5)
{
out_color = vec4(0.0, 0.0, 0.0, 1.0);
return;
}
out_color = scene;
return;
}
float width_pixels = params.edge0.z;
float vertical_scale = params.edge0.w;
float threshold_start = params.edge1.x;
float threshold_end = params.edge1.y;
float strength = params.edge1.z;
float value_boost = params.edge1.w;
float near_plane = max(params.edge2.y, 0.00001);
float far_plane = max(params.edge2.z, near_plane + 0.00001);
float width_reference_depth = max(params.edge2.w, near_plane);
float current_depth = texture(depth_texture, uv).r;
float current_linear_depth = linearize_reverse_z_depth(current_depth, near_plane, far_plane);
float depth_width_scale = width_reference_depth / max(current_linear_depth, near_plane);
float effective_width_pixels = clamp(width_pixels * depth_width_scale, width_pixels * 0.35, width_pixels * 1.5);
vec2 pixel_size = params.edge0.xy;
vec4 normal_roughness = normal_roughness_compatibility(
texture(normal_roughness_texture, uv)
);
vec3 view_normal = normalize(normal_roughness.xyz * 2.0 - 1.0);
vec2 offset_pixels = view_normal.xy * vec2(-effective_width_pixels, -effective_width_pixels * vertical_scale);
vec2 offset_uv = offset_pixels * pixel_size;
vec2 shifted_uv = clamp(uv + offset_uv, vec2(0.0), vec2(1.0));
float shifted_linear_depth = sample_filtered_displaced_linear_depth(
shifted_uv,
offset_uv,
pixel_size,
near_plane,
far_plane
);
float linear_depth_delta = max(shifted_linear_depth - current_linear_depth, 0.0);
float depth_edge = smoothstep(
threshold_start,
threshold_end,
linear_depth_delta
);
float edge_mask = clamp(depth_edge * avatar_mask * strength, 0.0, 1.0);
if (debug_edge_mask > 0.5)
{
out_color = vec4(vec3(edge_mask), 1.0);
return;
}
vec3 brightened = scene.rgb * value_boost;
out_color = vec4(mix(scene.rgb, brightened, edge_mask), scene.a);
}
""";
private const string ExtractHighlightsFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D input_texture;
layout(push_constant, std430) uniform Params
{
vec4 values;
} params;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
float max_channel_of(vec3 color)
{
return max(max(color.r, color.g), color.b);
}
float smooth_min(float a, float b, float smoothness)
{
if (smoothness == 0.0)
{
return min(a, b);
}
float h = max(smoothness - abs(a - b), 0.0) / smoothness;
return min(a, b) - h * h * smoothness * 0.25;
}
float smooth_max(float a, float b, float smoothness)
{
return -smooth_min(-a, -b, smoothness);
}
float smooth_clamp(
float value,
float min_value,
float max_value,
float min_smoothness,
float max_smoothness
)
{
return smooth_min(
max_value,
smooth_max(min_value, value, min_smoothness),
max_smoothness
);
}
float adaptive_smooth_clamp(
float value,
float min_value,
float max_value,
float smoothness
)
{
float range_distance = abs(max_value - min_value);
float min_smoothness = min(smoothness, min(min_value, range_distance));
float max_smoothness = min(smoothness, min(max_value, range_distance));
return smooth_clamp(value, min_value, max_value, min_smoothness, max_smoothness);
}
void main()
{
vec3 color = texture(input_texture, uv).rgb;
float threshold = params.values.x;
float smoothness = params.values.y;
float max_brightness = params.values.z;
float value = max_channel_of(color);
float clamped_value = adaptive_smooth_clamp(
value,
threshold,
threshold + max_brightness,
smoothness
);
float extracted_value = max(clamped_value - threshold, 0.0);
float source = extracted_value / max(value, 0.001);
out_color = vec4(color * source, 1.0);
}
""";
private const string DownsampleFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D input_texture;
layout(push_constant, std430) uniform Params
{
vec4 values;
} params;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
float reduce_max(vec4 color)
{
return max(max(max(color.r, color.g), color.b), color.a);
}
vec4 weighted_sum(vec4 a, vec4 b, vec4 c, vec4 d, vec4 weights)
{
float total_weight = weights.x + weights.y + weights.z + weights.w;
return (a * weights.x + b * weights.y + c * weights.z + d * weights.w) /
max(total_weight, 0.0001);
}
vec4 karis_brightness_weighted_sum(vec4 a, vec4 b, vec4 c, vec4 d)
{
vec4 brightness = vec4(reduce_max(a), reduce_max(b), reduce_max(c), reduce_max(d));
vec4 weights = vec4(1.0) / (brightness + vec4(1.0));
return weighted_sum(a, b, c, d, weights);
}
void main()
{
vec2 pixel_size = params.values.xy;
float use_karis_average = params.values.z;
vec4 center = texture(input_texture, uv);
vec4 upper_left_near = texture(input_texture, uv + pixel_size * vec2(-1.0, 1.0));
vec4 upper_right_near = texture(input_texture, uv + pixel_size * vec2(1.0, 1.0));
vec4 lower_left_near = texture(input_texture, uv + pixel_size * vec2(-1.0, -1.0));
vec4 lower_right_near = texture(input_texture, uv + pixel_size * vec2(1.0, -1.0));
vec4 left_far = texture(input_texture, uv + pixel_size * vec2(-2.0, 0.0));
vec4 right_far = texture(input_texture, uv + pixel_size * vec2(2.0, 0.0));
vec4 upper_far = texture(input_texture, uv + pixel_size * vec2(0.0, 2.0));
vec4 lower_far = texture(input_texture, uv + pixel_size * vec2(0.0, -2.0));
vec4 upper_left_far = texture(input_texture, uv + pixel_size * vec2(-2.0, 2.0));
vec4 upper_right_far = texture(input_texture, uv + pixel_size * vec2(2.0, 2.0));
vec4 lower_left_far = texture(input_texture, uv + pixel_size * vec2(-2.0, -2.0));
vec4 lower_right_far = texture(input_texture, uv + pixel_size * vec2(2.0, -2.0));
vec4 result;
if (use_karis_average > 0.5)
{
vec4 center_weighted_sum = karis_brightness_weighted_sum(
upper_left_near,
upper_right_near,
lower_right_near,
lower_left_near
);
vec4 upper_left_weighted_sum = karis_brightness_weighted_sum(
upper_left_far,
upper_far,
center,
left_far
);
vec4 upper_right_weighted_sum = karis_brightness_weighted_sum(
upper_far,
upper_right_far,
right_far,
center
);
vec4 lower_right_weighted_sum = karis_brightness_weighted_sum(
center,
right_far,
lower_right_far,
lower_far
);
vec4 lower_left_weighted_sum = karis_brightness_weighted_sum(
left_far,
center,
lower_far,
lower_left_far
);
result = center_weighted_sum * (4.0 / 8.0) +
(
upper_left_weighted_sum +
upper_right_weighted_sum +
lower_left_weighted_sum +
lower_right_weighted_sum
) * (1.0 / 8.0);
}
else
{
result = center * (4.0 / 32.0) +
(
upper_left_near +
upper_right_near +
lower_left_near +
lower_right_near
) * (4.0 / 32.0) +
(left_far + right_far + upper_far + lower_far) * (2.0 / 32.0) +
(
upper_left_far +
upper_right_far +
lower_left_far +
lower_right_far
) * (1.0 / 32.0);
}
out_color = vec4(result.rgb, 1.0);
}
""";
private const string UpsampleFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D base_texture;
layout(set = 0, binding = 1) uniform sampler2D input_texture;
layout(push_constant, std430) uniform Params
{
vec4 values;
} params;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
void main()
{
vec2 pixel_size = params.values.xy;
vec4 upsampled = vec4(0.0);
upsampled += texture(input_texture, uv) * (4.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(-1.0, 0.0)) * (2.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(0.0, 1.0)) * (2.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(1.0, 0.0)) * (2.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(0.0, -1.0)) * (2.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(-1.0, -1.0)) * (1.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(-1.0, 1.0)) * (1.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(1.0, -1.0)) * (1.0 / 16.0);
upsampled += texture(input_texture, uv + pixel_size * vec2(1.0, 1.0)) * (1.0 / 16.0);
vec3 base = texture(base_texture, uv).rgb;
out_color = vec4(base + upsampled.rgb, 1.0);
}
""";
private const string GlowCompositeFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D scene_texture;
layout(set = 0, binding = 1) uniform sampler2D bloom_texture;
layout(push_constant, std430) uniform Params
{
vec4 bloom_tint_strength;
} params;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
void main()
{
vec4 scene = texture(scene_texture, uv);
vec3 bloom = texture(bloom_texture, uv).rgb;
vec3 hdr = max(
scene.rgb + bloom * params.bloom_tint_strength.rgb * params.bloom_tint_strength.a,
vec3(0.0)
);
out_color = vec4(hdr, scene.a);
}
""";
private const string FinalColorFragmentShaderCode = """
#version 450
layout(set = 0, binding = 0) uniform sampler2D hdr_texture;
layout(push_constant, std430) uniform Params
{
vec4 naes_curve0;
vec4 naes_curve1;
vec4 luma_curve;
vec4 color_grade0;
vec4 color_grade1;
} params;
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 out_color;
vec3 clamp_tonemap_input(vec3 color)
{
return clamp(max(color, vec3(0.0)), 0.0, params.naes_curve1.y);
}
vec3 naes_tonemap(vec3 color)
{
vec3 x = clamp_tonemap_input(color);
return (x * (params.naes_curve0.x * x + vec3(params.naes_curve0.y))) /
(
x * (params.naes_curve0.z * x + vec3(params.naes_curve0.w)) +
vec3(params.naes_curve1.x)
);
}
float apply_luma_curve(float luma)
{
float mid_gate =
smoothstep(params.luma_curve.x, params.luma_curve.y, luma) *
(1.0 - smoothstep(params.luma_curve.z, params.luma_curve.w, luma));
return max(luma * (1.0 - params.color_grade0.x * mid_gate), 0.0);
}
vec3 apply_toon_color_grade(vec3 color)
{
float luma = dot(color, vec3(0.2126, 0.7152, 0.0722));
float max_channel = max(max(color.r, color.g), color.b);
float min_channel = min(min(color.r, color.g), color.b);
float saturation = max_channel <= 0.001
? 0.0
: (max_channel - min_channel) / max_channel;
float luma_gate = smoothstep(params.color_grade0.y, params.color_grade0.z, luma);
float saturation_gate =
1.0 - smoothstep(params.color_grade0.w, params.color_grade1.x, saturation);
float chroma_scale =
params.color_grade1.y + params.color_grade1.z * luma_gate * saturation_gate;
float luma2 = apply_luma_curve(luma);
vec3 gray = vec3(luma);
return max(vec3(luma2) + (color - gray) * chroma_scale, vec3(0.0));
}
void main()
{
vec4 hdr = texture(hdr_texture, uv);
vec3 mapped = apply_toon_color_grade(naes_tonemap(hdr.rgb));
out_color = vec4(mapped, hdr.a);
}
""";
}
@@ -0,0 +1,289 @@
using System;
using Godot;
/// <summary>
/// Runs same-frame stage post-process work for avatar source effects and final color mapping.
/// </summary>
public partial class StagePostProcessCompositorEffect : CompositorEffect
{
private const int MaxBloomLevels = 9;
private const int BloomQualityFactor = 2;
private static readonly GlowSettings Glow = new(
BloomTint: new Color(1.0f, 0.6938719749450684f, 0.6795425415039062f),
BloomStrength: 0.09f,
BloomSize: 0.26f,
HighlightThreshold: 0.56f,
HighlightSmoothness: 0.5f,
MaxHighlightBrightness: 1.0e20f
);
private static readonly EdgeLightSettings EdgeLight = new(
WidthPixels: 8.5f,
VerticalScale: 0.6f,
DepthThresholdStart: 0.030f,
DepthThresholdEnd: 0.135f,
Strength: 1.0f,
ValueBoost: 2.1f,
WidthReferenceDepth: 0.65f
);
private static readonly NaesTonemapSettings NaesTonemap = new(
A: 1.36f,
B: 0.047f,
C: 0.93f,
D: 0.56f,
E: 0.14f,
InputMax: 10.0f
);
private static readonly ToonColorGradeSettings ToonColorGrade = new(
LumaRiseStart: 0.42f,
LumaRiseEnd: 0.58f,
LumaFallStart: 0.66f,
LumaFallEnd: 0.78f,
LumaMidDip: 0.050f,
VibranceLumaStart: 0.24f,
VibranceLumaEnd: 0.50f,
VibranceSaturationStart: 0.10f,
VibranceSaturationEnd: 0.70f,
ChromaBase: 1.05f,
ChromaBoost: 0.47f
);
private static readonly Color[] ClearColor = { new(0.0f, 0.0f, 0.0f, 0.0f) };
private readonly uint _stencilReference;
private readonly Camera3D _camera;
private RenderingDevice _renderingDevice;
private Rid _sampler;
private long _fullscreenVertexFormat = RenderingDevice.InvalidId;
private Rid _fullscreenVertexBuffer;
private Rid _fullscreenVertexArray;
private Rid _copyShader;
private Rid _avatarMaskShader;
private Rid _edgeLightShader;
private Rid _extractShader;
private Rid _downsampleShader;
private Rid _upsampleShader;
private Rid _glowCompositeShader;
private Rid _finalColorShader;
private PostProcessResources _resources;
private bool _missingRenderingDeviceWarningPrinted;
private bool _missingRenderBuffersWarningPrinted;
private bool _missingStencilWarningPrinted;
private bool _missingNormalRoughnessWarningPrinted;
private bool _missingResolvedDepthWarningPrinted;
private bool _avatarMaskResolveWarningPrinted;
private bool _sourceResolveWarningPrinted;
private bool _avatarMaskEnabled;
private bool _avatarEdgeLightEnabled = true;
private bool _finalColorMappingEnabled = true;
private StageRenderDebugView _debugView = StageRenderDebugView.Final;
private static readonly StageRenderPipelineStage[] PipelineStages =
{
StageRenderPipelineStage.SceneCopy,
StageRenderPipelineStage.AvatarMask,
StageRenderPipelineStage.AvatarEdgeLight,
StageRenderPipelineStage.AvatarGlow,
StageRenderPipelineStage.FinalColorMapping,
};
public StagePostProcessCompositorEffect(int stencilReference, Camera3D camera)
{
_stencilReference = (uint)stencilReference;
_camera = camera ?? throw new ArgumentNullException(nameof(camera));
AccessResolvedColor = true;
AccessResolvedDepth = true;
NeedsNormalRoughness = true;
EffectCallbackType = EffectCallbackTypeEnum.PostTransparent;
UpdateEnabled();
}
public bool AvatarMaskEnabled
{
get => _avatarMaskEnabled;
set
{
_avatarMaskEnabled = value;
UpdateEnabled();
}
}
public bool FinalColorMappingEnabled
{
get => _finalColorMappingEnabled;
set
{
_finalColorMappingEnabled = value;
UpdateEnabled();
}
}
public bool AvatarEdgeLightEnabled
{
get => _avatarEdgeLightEnabled;
set => _avatarEdgeLightEnabled = value;
}
internal StageRenderDebugView DebugView
{
get => _debugView;
set
{
_debugView = value;
UpdateEnabled();
}
}
private void UpdateEnabled()
{
Enabled = _finalColorMappingEnabled || _debugView != StageRenderDebugView.Final;
}
public override void _RenderCallback(int effectCallbackType, RenderData renderData)
{
if (!Enabled || (EffectCallbackTypeEnum)effectCallbackType != EffectCallbackType)
{
return;
}
if (!EnsureRenderingDevice())
{
return;
}
if (renderData.GetRenderSceneBuffers() is not RenderSceneBuffersRD buffers)
{
WarnOnce(
ref _missingRenderBuffersWarningPrinted,
"Stage post-process requires RenderSceneBuffersRD; compositor callback had no RD buffers."
);
return;
}
var fullSize = buffers.GetInternalSize();
if (fullSize.X < 2 || fullSize.Y < 2)
{
return;
}
var sceneColor = buffers.GetColorTexture(false);
if (!sceneColor.IsValid)
{
return;
}
bool includeAvatarMask = _avatarMaskEnabled;
var stencilDepth = new Rid();
var resolvedDepth = new Rid();
var normalRoughness = new Rid();
bool includeAvatarEdgeLight = false;
if (includeAvatarMask)
{
stencilDepth = SelectStencilDepthTexture(buffers);
if (!stencilDepth.IsValid)
{
includeAvatarMask = false;
}
else
{
var sceneDepthFormat = _renderingDevice.TextureGetFormat(stencilDepth).Format;
if (!HasStencil(sceneDepthFormat))
{
WarnOnce(
ref _missingStencilWarningPrinted,
$"Avatar mask needs a stencil depth texture; scene depth format is {sceneDepthFormat}."
);
includeAvatarMask = false;
stencilDepth = new Rid();
}
}
if (includeAvatarMask)
{
resolvedDepth = buffers.GetDepthTexture(false);
if (!resolvedDepth.IsValid)
{
WarnOnce(
ref _missingResolvedDepthWarningPrinted,
"Avatar edge light needs a resolved depth texture."
);
}
normalRoughness = GetNormalRoughnessTexture(buffers);
if (!normalRoughness.IsValid)
{
WarnOnce(
ref _missingNormalRoughnessWarningPrinted,
"Avatar edge light needs forward_clustered/normal_roughness."
);
}
includeAvatarEdgeLight =
_avatarEdgeLightEnabled
&& resolvedDepth.IsValid
&& normalRoughness.IsValid;
}
}
if (!EnsureResources(
fullSize,
sceneColor,
stencilDepth,
resolvedDepth,
normalRoughness,
includeAvatarMask,
includeAvatarEdgeLight,
_debugView
))
{
return;
}
RunPipeline();
}
public override void _Notification(int what)
{
if (what != NotificationPredelete)
{
return;
}
ReleaseRenderingResources();
}
public void ReleaseRenderingResources()
{
if (_renderingDevice != null && !RenderingServer.IsOnRenderThread())
{
RenderingServer.CallOnRenderThread(Callable.From(ReleaseRenderingResourcesOnRenderThread));
RenderingServer.ForceSync();
return;
}
ReleaseRenderingResourcesOnRenderThread();
}
private void ReleaseRenderingResourcesOnRenderThread()
{
ReleaseResources();
FreeOwnedRid(ref _sampler);
FreeOwnedRid(ref _fullscreenVertexArray);
FreeOwnedRid(ref _fullscreenVertexBuffer);
FreeOwnedRid(ref _copyShader);
FreeOwnedRid(ref _avatarMaskShader);
FreeOwnedRid(ref _edgeLightShader);
FreeOwnedRid(ref _extractShader);
FreeOwnedRid(ref _downsampleShader);
FreeOwnedRid(ref _upsampleShader);
FreeOwnedRid(ref _glowCompositeShader);
FreeOwnedRid(ref _finalColorShader);
_fullscreenVertexFormat = RenderingDevice.InvalidId;
_renderingDevice = null;
}
}
@@ -0,0 +1,68 @@
using System;
using Godot;
/// <summary>
/// Coordinates stage render-effect ownership for avatar source overlays and post-processing.
/// </summary>
public sealed class StageRenderEffectsRuntime : IDisposable
{
private const int AvatarStencilReference = 1;
private readonly StageCompositorOwner _compositorOwner;
private readonly StageMaterialOverlayOwner _overlayOwner;
private readonly StagePostProcessCompositorEffect _postProcessEffect;
private bool _disposed;
public StageRenderEffectsRuntime(Camera3D camera)
{
_postProcessEffect = new StagePostProcessCompositorEffect(AvatarStencilReference, camera);
_overlayOwner = new StageMaterialOverlayOwner(AvatarStencilReference);
_compositorOwner = new StageCompositorOwner(camera, _postProcessEffect);
}
public void UseAvatar(Node avatar)
{
if (_disposed)
{
return;
}
_overlayOwner.UseAvatarMask(avatar);
_postProcessEffect.AvatarMaskEnabled =
_overlayOwner.HasSource(StageMaterialOverlaySourceKind.AvatarMask);
}
public string CurrentDebugView => StageRenderDebugViewNames.ToTransportValue(_postProcessEffect.DebugView);
public string SetDebugView(string view)
{
if (!StageRenderDebugViewNames.TryParse(view, out var debugView))
{
throw new ArgumentException($"Unknown render debug view: {view}.", nameof(view));
}
_postProcessEffect.DebugView = debugView;
return StageRenderDebugViewNames.ToTransportValue(debugView);
}
public bool SetAvatarEdgeLightEnabled(bool enabled)
{
_postProcessEffect.AvatarEdgeLightEnabled = enabled;
return _postProcessEffect.AvatarEdgeLightEnabled;
}
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
_postProcessEffect.AvatarMaskEnabled = false;
_postProcessEffect.FinalColorMappingEnabled = false;
_overlayOwner.Dispose();
_compositorOwner.Dispose();
_postProcessEffect.ReleaseRenderingResources();
}
}
@@ -0,0 +1,241 @@
param(
[Parameter(Mandatory = $true)]
[int] $TargetProcessId,
[Parameter(Mandatory = $true)]
[string] $OutputPath,
[int] $TimeoutMs = 15000,
[int] $SettleMs = 1000
)
$ErrorActionPreference = 'Stop'
Add-Type -AssemblyName System.Drawing
Add-Type @"
using System;
using System.Runtime.InteropServices;
using System.Text;
public static class WindowCaptureNative
{
public delegate bool EnumWindowsProc(IntPtr hWnd, IntPtr lParam);
[DllImport("user32.dll")]
public static extern bool EnumWindows(EnumWindowsProc lpEnumFunc, IntPtr lParam);
[DllImport("user32.dll")]
public static extern bool IsWindowVisible(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint lpdwProcessId);
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
public static extern int GetWindowText(IntPtr hWnd, StringBuilder lpString, int nMaxCount);
[DllImport("user32.dll")]
public static extern int GetWindowTextLength(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern bool GetClientRect(IntPtr hWnd, out Rect lpRect);
[DllImport("user32.dll")]
public static extern bool ClientToScreen(IntPtr hWnd, ref Point lpPoint);
[DllImport("user32.dll")]
public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern bool SetWindowPos(
IntPtr hWnd,
IntPtr hWndInsertAfter,
int X,
int Y,
int cx,
int cy,
uint uFlags
);
[DllImport("user32.dll")]
public static extern bool SetProcessDPIAware();
[StructLayout(LayoutKind.Sequential)]
public struct Rect
{
public int Left;
public int Top;
public int Right;
public int Bottom;
}
[StructLayout(LayoutKind.Sequential)]
public struct Point
{
public int X;
public int Y;
}
public struct WindowInfo
{
public IntPtr Handle;
public int X;
public int Y;
public int Width;
public int Height;
public string Title;
}
public static WindowInfo FindBestWindow(int processId)
{
WindowInfo best = new WindowInfo();
EnumWindows(delegate (IntPtr hWnd, IntPtr lParam)
{
uint windowProcessId;
GetWindowThreadProcessId(hWnd, out windowProcessId);
if (windowProcessId != processId || !IsWindowVisible(hWnd))
{
return true;
}
WindowInfo candidate = GetWindowInfo(hWnd);
long bestArea = (long)best.Width * best.Height;
long candidateArea = (long)candidate.Width * candidate.Height;
if (candidateArea > bestArea)
{
best = candidate;
}
return true;
}, IntPtr.Zero);
return best;
}
public static WindowInfo GetWindowInfo(IntPtr hWnd)
{
Rect rect;
if (!GetClientRect(hWnd, out rect))
{
return new WindowInfo();
}
Point point = new Point();
if (!ClientToScreen(hWnd, ref point))
{
return new WindowInfo();
}
return new WindowInfo
{
Handle = hWnd,
X = point.X,
Y = point.Y,
Width = rect.Right - rect.Left,
Height = rect.Bottom - rect.Top,
Title = GetTitle(hWnd),
};
}
private static string GetTitle(IntPtr hWnd)
{
int length = GetWindowTextLength(hWnd);
if (length <= 0)
{
return "";
}
StringBuilder builder = new StringBuilder(length + 1);
GetWindowText(hWnd, builder, builder.Capacity);
return builder.ToString();
}
}
"@
try {
[WindowCaptureNative]::SetProcessDPIAware() | Out-Null
}
catch {
# The PowerShell host may already have a DPI awareness context.
}
$deadline = [DateTime]::UtcNow.AddMilliseconds($TimeoutMs)
$window = [WindowCaptureNative]::FindBestWindow($TargetProcessId)
while ($window.Handle -eq [IntPtr]::Zero -and [DateTime]::UtcNow -lt $deadline) {
Start-Sleep -Milliseconds 100
$window = [WindowCaptureNative]::FindBestWindow($TargetProcessId)
}
if ($window.Handle -eq [IntPtr]::Zero) {
throw "Could not find a visible top-level window for process $TargetProcessId."
}
[WindowCaptureNative]::ShowWindow($window.Handle, 9) | Out-Null
[WindowCaptureNative]::SetWindowPos(
$window.Handle,
[IntPtr]::new(-1),
0,
0,
0,
0,
0x0001 -bor 0x0002 -bor 0x0040
) | Out-Null
[WindowCaptureNative]::SetForegroundWindow($window.Handle) | Out-Null
Start-Sleep -Milliseconds $SettleMs
$window = [WindowCaptureNative]::GetWindowInfo($window.Handle)
if ($window.Width -le 0 -or $window.Height -le 0) {
throw "Godot window client area is empty."
}
$directory = [System.IO.Path]::GetDirectoryName($OutputPath)
if ($directory) {
New-Item -ItemType Directory -Force -Path $directory | Out-Null
}
$bitmap = [System.Drawing.Bitmap]::new(
$window.Width,
$window.Height,
[System.Drawing.Imaging.PixelFormat]::Format32bppArgb
)
$graphics = [System.Drawing.Graphics]::FromImage($bitmap)
try {
$graphics.CopyFromScreen(
$window.X,
$window.Y,
0,
0,
[System.Drawing.Size]::new($window.Width, $window.Height),
[System.Drawing.CopyPixelOperation]::SourceCopy
)
$bitmap.Save($OutputPath, [System.Drawing.Imaging.ImageFormat]::Png)
}
finally {
$graphics.Dispose()
$bitmap.Dispose()
[WindowCaptureNative]::SetWindowPos(
$window.Handle,
[IntPtr]::new(-2),
0,
0,
0,
0,
0x0001 -bor 0x0002
) | Out-Null
}
$resolvedPath = (Resolve-Path -LiteralPath $OutputPath).Path
@{
height = $window.Height
left = $window.X
path = $resolvedPath
title = $window.Title
top = $window.Y
width = $window.Width
} | ConvertTo-Json -Compress
@@ -0,0 +1,664 @@
import process from 'node:process'
import { Buffer } from 'node:buffer'
import { spawn } from 'node:child_process'
import { createHash, randomUUID } from 'node:crypto'
import { existsSync } from 'node:fs'
import { mkdir } from 'node:fs/promises'
import { createServer } from 'node:http'
import { dirname, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
const scriptDirectory = dirname(fileURLToPath(import.meta.url))
const projectDirectory = resolve(scriptDirectory, '..')
const repositoryRoot = resolve(projectDirectory, '..', '..')
const defaultModelPath = join(
repositoryRoot,
'packages',
'stage-ui',
'src',
'assets',
'vrm',
'models',
'AvatarSample-A',
'AvatarSample_A.vrm',
)
const artifactDirectory = join(projectDirectory, 'artifacts', 'render-stages')
const defaultLogPath = join(artifactDirectory, 'godot-stage.log')
const defaultRenderStageViews = [
'scene-copy',
'avatar-mask',
'avatar-edge-mask',
'after-avatar-edge-light',
'after-avatar-glow',
'final',
'final-edge-off',
]
const windowCaptureScriptPath = join(scriptDirectory, 'captureWindowClientPng.ps1')
const webSocketGuid = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'
function parseArgs(argv) {
const options = {
avatarEdgeLight: true,
godotPath: process.env.GODOT4,
headless: false,
height: 720,
logPath: defaultLogPath,
modelPath: defaultModelPath,
renderStageViews: null,
settleMs: 1000,
stageDumpDirectory: null,
viewPreset: 'default',
width: 1280,
}
for (let index = 0; index < argv.length; index++) {
const argument = argv[index]
switch (argument) {
case '--avatar-edge-light':
options.avatarEdgeLight = parseAvatarEdgeLight(
resolveRequiredValue(argv, ++index, argument),
argument,
)
break
case '--godot':
options.godotPath = resolveRequiredValue(argv, ++index, argument)
break
case '--dump-render-stages':
options.stageDumpDirectory = resolveRequiredValue(argv, ++index, argument)
break
case '--headless':
options.headless = true
break
case '--height':
options.height = parsePositiveInteger(resolveRequiredValue(argv, ++index, argument), argument)
break
case '--log-file':
options.logPath = resolveRequiredValue(argv, ++index, argument)
break
case '--model':
options.modelPath = resolveRequiredValue(argv, ++index, argument)
break
case '--render-stages':
options.renderStageViews = parseRenderStageViews(
resolveRequiredValue(argv, ++index, argument),
argument,
)
break
case '--settle-ms':
options.settleMs = parseNonNegativeInteger(
resolveRequiredValue(argv, ++index, argument),
argument,
)
break
case '--view-preset':
options.viewPreset = parseViewPreset(resolveRequiredValue(argv, ++index, argument), argument)
break
case '--width':
options.width = parsePositiveInteger(resolveRequiredValue(argv, ++index, argument), argument)
break
default:
throw new Error(`Unknown argument: ${argument}`)
}
}
if (!options.godotPath) {
throw new Error('GODOT4 is not set. Pass --godot or set GODOT4 to the Godot .NET executable.')
}
if (options.headless) {
throw new Error('Render-stage window capture requires a visible Godot window. Remove --headless.')
}
if (!options.stageDumpDirectory) {
throw new Error('Pass --dump-render-stages <directory>. Baseline comparison is not supported.')
}
options.godotPath = resolve(options.godotPath)
options.logPath = resolve(options.logPath)
options.modelPath = resolve(options.modelPath)
options.stageDumpDirectory = resolve(options.stageDumpDirectory)
options.renderStageViews ??= defaultRenderStageViews
return options
}
function resolveRequiredValue(argv, index, label) {
const value = argv[index]
if (!value || value.startsWith('--')) {
throw new Error(`${label} requires a value.`)
}
return value
}
function parsePositiveInteger(value, label) {
const parsed = Number.parseInt(value, 10)
if (!Number.isInteger(parsed) || parsed <= 0) {
throw new Error(`${label} must be a positive integer.`)
}
return parsed
}
function parseNonNegativeInteger(value, label) {
const parsed = Number.parseInt(value, 10)
if (!Number.isInteger(parsed) || parsed < 0) {
throw new Error(`${label} must be a non-negative integer.`)
}
return parsed
}
function parseRenderStageViews(value, label) {
const views = value.split(',').map(item => item.trim()).filter(Boolean)
if (views.length === 0) {
throw new Error(`${label} must include at least one render stage.`)
}
return views
}
function parseViewPreset(value, label) {
if (value === 'default' || value === 'upper-body') {
return value
}
throw new Error(`${label} must be "default" or "upper-body".`)
}
function parseAvatarEdgeLight(value, label) {
switch (value) {
case 'on':
case 'enabled':
case 'true':
return true
case 'off':
case 'disabled':
case 'false':
return false
default:
throw new Error(`${label} must be "on" or "off".`)
}
}
async function createStageHost() {
let peerSocket
let peerBuffer = Buffer.alloc(0)
const messages = []
const waiters = []
const token = randomUUID()
const server = createServer()
server.on('upgrade', (request, socket) => {
const requestUrl = new URL(request.url ?? '/', 'ws://127.0.0.1')
if (requestUrl.pathname !== '/ws' || requestUrl.searchParams.get('token') !== token) {
socket.destroy()
return
}
const key = request.headers['sec-websocket-key']
if (typeof key !== 'string') {
socket.destroy()
return
}
const accept = createHash('sha1').update(`${key}${webSocketGuid}`).digest('base64')
socket.write([
'HTTP/1.1 101 Switching Protocols',
'Upgrade: websocket',
'Connection: Upgrade',
`Sec-WebSocket-Accept: ${accept}`,
'\r\n',
].join('\r\n'))
peerSocket = socket
socket.on('data', (chunk) => {
peerBuffer = Buffer.concat([peerBuffer, chunk])
const result = readFrames(peerBuffer)
peerBuffer = result.remaining
for (const frame of result.frames) {
if (frame.opcode === 0x1) {
pushMessage(JSON.parse(frame.payload.toString('utf8')))
}
else if (frame.opcode === 0x8) {
socket.end()
}
else if (frame.opcode === 0x9) {
writeFrame(socket, 0xA, frame.payload)
}
}
})
socket.on('close', () => {
if (peerSocket === socket) {
peerSocket = undefined
}
})
})
function pushMessage(message) {
messages.push(message)
for (let index = waiters.length - 1; index >= 0; index--) {
const waiter = waiters[index]
if (waiter.predicate(message)) {
waiters.splice(index, 1)
clearTimeout(waiter.timeout)
waiter.resolve(message)
}
}
}
function waitFor(predicate, timeoutMs, label) {
const existingIndex = messages.findIndex(predicate)
if (existingIndex >= 0) {
const [message] = messages.splice(existingIndex, 1)
return Promise.resolve(message)
}
return new Promise((resolvePromise, rejectPromise) => {
const waiter = {
predicate,
resolve: resolvePromise,
timeout: setTimeout(() => {
const waiterIndex = waiters.indexOf(waiter)
if (waiterIndex >= 0) {
waiters.splice(waiterIndex, 1)
}
rejectPromise(new Error(`Timed out waiting for ${label}.`))
}, timeoutMs),
}
waiters.push(waiter)
})
}
await new Promise((resolvePromise, rejectPromise) => {
server.once('error', rejectPromise)
server.listen(0, '127.0.0.1', () => {
server.off('error', rejectPromise)
resolvePromise()
})
})
const address = server.address()
if (!address || typeof address === 'string') {
throw new Error('Failed to bind local WebSocket host.')
}
return {
close: () => new Promise((resolvePromise) => {
let resolved = false
let timeout
const resolveOnce = () => {
if (resolved) {
return
}
resolved = true
clearTimeout(timeout)
resolvePromise()
}
timeout = setTimeout(resolveOnce, 1000)
peerSocket?.destroy()
server.close(resolveOnce)
}),
send(type, payload) {
if (!peerSocket) {
throw new Error(`Cannot send ${type}; Godot WebSocket is not connected.`)
}
writeFrame(peerSocket, 0x1, Buffer.from(JSON.stringify({ type, payload }), 'utf8'))
},
url: `ws://127.0.0.1:${address.port}/ws?token=${token}`,
waitForType(type, timeoutMs) {
return waitFor(message => message?.type === type, timeoutMs, type)
},
waitForRequest(type, requestId, timeoutMs) {
return waitFor(
message => message?.type === type && message?.payload?.requestId === requestId,
timeoutMs,
`${type} ${requestId}`,
)
},
}
}
function readFrames(buffer) {
const frames = []
let offset = 0
while (buffer.length - offset >= 2) {
const first = buffer[offset]
const second = buffer[offset + 1]
const opcode = first & 0x0F
const masked = (second & 0x80) !== 0
let length = second & 0x7F
let headerLength = 2
if (length === 126) {
if (buffer.length - offset < 4) {
break
}
length = buffer.readUInt16BE(offset + 2)
headerLength = 4
}
else if (length === 127) {
if (buffer.length - offset < 10) {
break
}
const bigLength = buffer.readBigUInt64BE(offset + 2)
if (bigLength > BigInt(Number.MAX_SAFE_INTEGER)) {
throw new Error('WebSocket frame was too large.')
}
length = Number(bigLength)
headerLength = 10
}
const maskLength = masked ? 4 : 0
const totalLength = headerLength + maskLength + length
if (buffer.length - offset < totalLength) {
break
}
const payloadStart = offset + headerLength + maskLength
const payload = Buffer.from(buffer.subarray(payloadStart, payloadStart + length))
if (masked) {
const mask = buffer.subarray(offset + headerLength, offset + headerLength + 4)
for (let index = 0; index < payload.length; index++) {
payload[index] ^= mask[index % 4]
}
}
frames.push({ opcode, payload })
offset += totalLength
}
return {
frames,
remaining: buffer.subarray(offset),
}
}
function writeFrame(socket, opcode, payload) {
const length = payload.length
let header
if (length < 126) {
header = Buffer.from([0x80 | opcode, length])
}
else if (length <= 0xFFFF) {
header = Buffer.alloc(4)
header[0] = 0x80 | opcode
header[1] = 126
header.writeUInt16BE(length, 2)
}
else {
header = Buffer.alloc(10)
header[0] = 0x80 | opcode
header[1] = 127
header.writeBigUInt64BE(BigInt(length), 2)
}
socket.write(Buffer.concat([header, payload]))
}
function launchGodot(options, webSocketUrl) {
const args = [
'--path',
projectDirectory,
'--resolution',
`${options.width}x${options.height}`,
'--log-file',
options.logPath,
]
if (options.headless) {
args.push('--headless')
}
args.push('--', `--airi-ws-url=${webSocketUrl}`)
return spawn(options.godotPath, args, {
cwd: projectDirectory,
stdio: ['ignore', 'pipe', 'pipe'],
windowsHide: false,
})
}
function waitForProcessExit(processHandle, timeoutMs) {
return new Promise((resolvePromise) => {
const timeout = setTimeout(resolvePromise, timeoutMs, false)
processHandle.once('close', () => {
clearTimeout(timeout)
resolvePromise(true)
})
})
}
async function stopGodot(host, processHandle) {
try {
host.send('host.shutdown')
}
catch {}
const exited = await waitForProcessExit(processHandle, 3000)
if (!exited) {
processHandle.kill()
await waitForProcessExit(processHandle, 3000)
}
}
async function captureGodotWindowPng(processHandle, options) {
if (process.platform !== 'win32') {
throw new Error('Render-stage window capture currently requires Windows.')
}
if (!processHandle.pid) {
throw new Error('Godot process id was not available for window capture.')
}
const result = await runProcess(
process.env.PWSH ?? 'powershell.exe',
[
'-NoProfile',
'-ExecutionPolicy',
'Bypass',
'-File',
windowCaptureScriptPath,
'-TargetProcessId',
String(processHandle.pid),
'-OutputPath',
options.currentPath,
'-SettleMs',
String(options.settleMs),
],
)
const stdout = result.stdout.trim()
const lines = stdout.split(/\r?\n/).filter(Boolean)
const lastLine = lines[lines.length - 1]
if (!lastLine) {
throw new Error('Window capture did not return JSON metadata.')
}
try {
return JSON.parse(lastLine)
}
catch (error) {
throw new Error(`Failed to parse window capture metadata: ${error.message}\n${stdout}`)
}
}
async function setRenderDebugView(host, view) {
const requestId = randomUUID()
host.send('host.render.set_debug_view', {
requestId,
view,
})
const response = await host.waitForRequest('stage.render.debug_view', requestId, 10000)
if (response?.payload?.view !== view) {
throw new Error(`Godot applied unexpected render debug view: ${response?.payload?.view}`)
}
}
async function setAvatarEdgeLight(host, enabled) {
const requestId = randomUUID()
host.send('host.render.set_avatar_edge_light', {
requestId,
enabled,
})
const response = await host.waitForRequest('stage.render.avatar_edge_light', requestId, 10000)
if (response?.payload?.enabled !== enabled) {
throw new Error(`Godot applied unexpected avatar edge-light state: ${response?.payload?.enabled}`)
}
}
async function applyViewPreset(host, preset) {
if (preset === 'default') {
return
}
const snapshot = await requestViewSnapshot(host)
const patch = createViewPresetPatch(preset, snapshot)
const requestId = randomUUID()
host.send('host.view.patch', {
requestId,
patch,
})
await host.waitForRequest('stage.view.snapshot', requestId, 10000)
}
async function requestViewSnapshot(host) {
const requestId = randomUUID()
host.send('host.view.request_snapshot', {
requestId,
})
const response = await host.waitForRequest('stage.view.snapshot', requestId, 10000)
return response.payload
}
function createViewPresetPatch(preset, snapshot) {
if (preset !== 'upper-body') {
throw new Error(`Unknown view preset: ${preset}`)
}
const bounds = snapshot?.avatarBounds
if (!bounds) {
throw new Error('Upper-body view preset requires avatar bounds from Godot.')
}
const center = bounds.center
const size = bounds.size
const fovDeg = 35
const targetY = center.y + size.y * 0.22
const distance = Math.max(size.y * 0.95, 1.2)
const positionY = targetY + size.y * 0.02
const pitchDeg = Math.atan2(targetY - positionY, distance) * 180 / Math.PI
return {
camera: {
position: {
x: center.x,
y: positionY,
z: center.z + distance,
},
yawDeg: 0,
pitchDeg,
fovDeg,
},
}
}
async function captureRenderStageViews(host, processHandle, options) {
await mkdir(options.stageDumpDirectory, { recursive: true })
for (const view of options.renderStageViews) {
const isEdgeOffView = view === 'final-edge-off'
const edgeLightEnabled = isEdgeOffView ? false : options.avatarEdgeLight
await setAvatarEdgeLight(host, edgeLightEnabled)
await setRenderDebugView(host, isEdgeOffView ? 'final' : view)
const stageCapturePath = join(options.stageDumpDirectory, `${view}.png`)
const capture = await captureGodotWindowPng(processHandle, {
...options,
currentPath: stageCapturePath,
})
console.info(
`Captured render stage ${view}: ${capture.path} (${capture.width}x${capture.height})`,
)
}
await setAvatarEdgeLight(host, options.avatarEdgeLight)
await setRenderDebugView(host, 'final')
}
function runProcess(command, args) {
return new Promise((resolvePromise, rejectPromise) => {
const child = spawn(command, args, {
cwd: projectDirectory,
stdio: ['ignore', 'pipe', 'pipe'],
windowsHide: true,
})
const stdout = []
const stderr = []
child.stdout.on('data', chunk => stdout.push(chunk))
child.stderr.on('data', chunk => stderr.push(chunk))
child.once('error', rejectPromise)
child.once('close', (code) => {
const result = {
code,
stderr: Buffer.concat(stderr).toString('utf8'),
stdout: Buffer.concat(stdout).toString('utf8'),
}
if (code !== 0) {
rejectPromise(new Error(result.stderr.trim() || `${command} exited with code ${code}.`))
return
}
resolvePromise(result)
})
})
}
async function main() {
const options = parseArgs(process.argv.slice(2))
if (!existsSync(options.godotPath)) {
throw new Error(`Godot executable does not exist: ${options.godotPath}`)
}
if (!existsSync(options.modelPath)) {
throw new Error(`VRM model does not exist: ${options.modelPath}`)
}
await mkdir(dirname(options.logPath), { recursive: true })
await mkdir(options.stageDumpDirectory, { recursive: true })
const host = await createStageHost()
const godot = launchGodot(options, host.url)
godot.stdout.on('data', chunk => process.stdout.write(chunk))
godot.stderr.on('data', chunk => process.stderr.write(chunk))
try {
await host.waitForType('stage.ready', 20000)
host.send('host.scene.apply', {
format: 'vrm',
modelId: 'render-stage-observation-avatar-sample-a',
name: 'AvatarSample_A',
path: options.modelPath,
})
await host.waitForType('scene.applied', 45000)
await applyViewPreset(host, options.viewPreset)
await setAvatarEdgeLight(host, options.avatarEdgeLight)
await captureRenderStageViews(host, godot, options)
}
finally {
await stopGodot(host, godot)
await host.close()
}
}
main().catch((error) => {
console.error(error.message)
process.exitCode = 1
})
@@ -0,0 +1,213 @@
import json
import math
import os
from pathlib import Path
import bpy
from mathutils import Vector
def resolve_output_dir():
value = os.environ.get("AIRI_XIAOER_EDGE_OUT_DIR") or os.environ.get("AIRI_EDGE_LIGHT_OUT_DIR")
if not value:
raise RuntimeError("Set AIRI_XIAOER_EDGE_OUT_DIR to the reference stage output directory.")
output_dir = Path(value)
output_dir.mkdir(parents=True, exist_ok=True)
return output_dir
def configure_scene(scene):
scene.use_nodes = True
scene.render.use_compositing = True
scene.render.use_sequencer = False
scene.render.image_settings.file_format = "PNG"
scene.render.image_settings.color_mode = "RGBA"
scene.render.resolution_x = 1920
scene.render.resolution_y = 1080
scene.render.resolution_percentage = 100
def frame_upper_body(scene):
camera = scene.camera
if camera is None:
raise RuntimeError("Scene camera not found.")
min_v = Vector((math.inf, math.inf, math.inf))
max_v = Vector((-math.inf, -math.inf, -math.inf))
mesh_count = 0
for obj in scene.objects:
if obj.type != "MESH" or obj.hide_render:
continue
mesh_count += 1
for corner in obj.bound_box:
world = obj.matrix_world @ Vector(corner)
min_v.x = min(min_v.x, world.x)
min_v.y = min(min_v.y, world.y)
min_v.z = min(min_v.z, world.z)
max_v.x = max(max_v.x, world.x)
max_v.y = max(max_v.y, world.y)
max_v.z = max(max_v.z, world.z)
if mesh_count == 0:
raise RuntimeError("No render-visible mesh objects found.")
height = max_v.z - min_v.z
target = Vector((
(min_v.x + max_v.x) * 0.5,
(min_v.y + max_v.y) * 0.5,
min_v.z + height * 0.735,
))
frame_span = height * 0.48
camera.data.lens = 50
camera.data.clip_start = 0.01
camera.data.clip_end = 1000
distance = frame_span / (2 * math.tan(camera.data.angle_y * 0.5)) * 1.05
camera.location = Vector((target.x, target.y, target.z + 0.02)) + Vector((0, -1, 0)) * distance
camera.rotation_euler = (target - camera.location).to_track_quat("-Z", "Y").to_euler()
scene.camera = camera
return camera
def find_required_node(tree, label, predicate):
node = next((node for node in tree.nodes if predicate(node)), None)
if node is None:
raise RuntimeError(f"Missing compositor node: {label}.")
return node
def find_group_node(tree, group_name):
return find_required_node(
tree,
group_name,
lambda node: getattr(node, "node_tree", None)
and node.node_tree
and node.node_tree.name == group_name,
)
def ensure_group_output_socket(tree):
if not any(getattr(item, "in_out", None) == "OUTPUT" for item in tree.interface.items_tree):
tree.interface.new_socket(name="Image", in_out="OUTPUT", socket_type="NodeSocketColor")
tree.interface_update(bpy.context)
output = find_required_node(
tree,
"Group Output",
lambda node: node.bl_idname == "NodeGroupOutput",
)
real_inputs = [socket for socket in output.inputs if socket.type != "CUSTOM"]
if not real_inputs:
raise RuntimeError("Group Output has no real input socket.")
return output, real_inputs[0]
def clear_socket(tree, socket):
for link in list(tree.links):
if link.to_socket == socket:
tree.links.remove(link)
def link_once(tree, from_socket, to_socket):
clear_socket(tree, to_socket)
tree.links.new(from_socket, to_socket)
def render_path(scene, path):
scene.render.filepath = str(path)
bpy.ops.render.render(write_still=True)
print("AIRI_XIAOER_REFERENCE_RENDER", path)
def main():
output_dir = resolve_output_dir()
scene = bpy.context.scene
configure_scene(scene)
camera = frame_upper_body(scene)
main_tree = scene.compositing_node_group
if main_tree is None:
raise RuntimeError("Scene has no compositor node group.")
output, output_socket = ensure_group_output_socket(main_tree)
viewer = next((node for node in main_tree.nodes if node.bl_idname == "CompositorNodeViewer"), None)
render_layers = find_required_node(
main_tree,
"Render Layers",
lambda node: node.bl_idname == "CompositorNodeRLayers",
)
edge_node = find_group_node(main_tree, "边缘光")
bangs_node = find_group_node(main_tree, "刘海阴影")
glow_node = find_group_node(main_tree, "辉光")
def connect_final(socket):
link_once(main_tree, socket, output_socket)
if viewer is not None and len(viewer.inputs) > 0:
link_once(main_tree, socket, viewer.inputs[0])
def connect_edge_inputs():
link_once(main_tree, render_layers.outputs["Image"], edge_node.inputs["Image"])
link_once(main_tree, render_layers.outputs["Depth"], edge_node.inputs["深度"])
link_once(main_tree, render_layers.outputs["法向"], edge_node.inputs["法向"])
edge_group = edge_node.node_tree
edge_output, edge_output_socket = ensure_group_output_socket(edge_group)
mix_node = edge_group.nodes.get("Mix")
mask_node = edge_group.nodes.get("Map Range.001")
if mix_node is None or mask_node is None:
raise RuntimeError("Edge group is missing Mix or Map Range.001.")
def connect_edge_group_output(socket):
link_once(edge_group, socket, edge_output_socket)
# Raw render layer image.
connect_final(render_layers.outputs["Image"])
render_path(scene, output_dir / "reference_raw.png")
# Edge node output before bangs shadow and glow.
connect_edge_inputs()
connect_edge_group_output(mix_node.outputs["Result"])
connect_final(edge_node.outputs[0])
render_path(scene, output_dir / "reference_after_edge.png")
# Internal edge mask: Map Range.001.Result before Mix.
connect_edge_group_output(mask_node.outputs["Result"])
connect_final(edge_node.outputs[0])
render_path(scene, output_dir / "reference_edge_mask.png")
# Final enabled chain: edge -> bangs shadow -> glow.
connect_edge_group_output(mix_node.outputs["Result"])
connect_edge_inputs()
link_once(main_tree, edge_node.outputs[0], bangs_node.inputs["Input"])
link_once(main_tree, render_layers.outputs["Depth"], bangs_node.inputs["深度"])
link_once(main_tree, render_layers.outputs[""], bangs_node.inputs[""])
link_once(main_tree, bangs_node.outputs[0], glow_node.inputs["Input"])
connect_final(glow_node.outputs[0])
render_path(scene, output_dir / "reference_final_on.png")
# Final disabled chain: bypass only the edge-light node.
link_once(main_tree, render_layers.outputs["Image"], bangs_node.inputs["Input"])
connect_final(glow_node.outputs[0])
render_path(scene, output_dir / "reference_final_off.png")
metadata = {
"blend": bpy.data.filepath,
"cameraLocation": [round(value, 6) for value in camera.location],
"cameraRotationEuler": [round(value, 6) for value in camera.rotation_euler],
"viewTransform": scene.view_settings.view_transform,
"look": scene.view_settings.look,
"outputSocket": {
"main": output_socket.name,
"edge": edge_output_socket.name,
},
}
(output_dir / "reference_metadata.json").write_text(
json.dumps(metadata, ensure_ascii=False, indent=2),
encoding="utf-8",
)
main()
-1
View File
@@ -5,7 +5,6 @@ export default defineConfig({
projects: [
'apps/server',
'apps/ui-server-auth',
'apps/ui-admin',
'apps/stage-tamagotchi',
'packages/cap-vite',
'packages/core-agent',