feat(stage-tamagotchi-godot): avatar glow NPR effect (#1951)

## Summary

This PR adds the current Stage Godot avatar presentation path:
avatar-only same-frame glow plus tuned toon color mapping.

<img width="994" height="951" alt="屏幕截图 2026-06-02 015159"
src="https://github.com/user-attachments/assets/feb94b95-745f-4b47-aba2-0289dafecd1c"
/>

<img width="1769" height="1279" alt="屏幕截图 2026-06-02 021212"
src="https://github.com/user-attachments/assets/d4437dc2-e859-42ca-b3fd-0a90e55d641b"
/>

## Design

The implementation uses three layers:

1. Base avatar material rendering still comes from the vendored V-Sekai
MToon shader.
2. `StageAvatarGlowRuntime` marks visible avatar meshes through a
depth-tested `MaterialOverlay` stencil pass.
3. `StageAvatarGlowCompositorEffect` reads that stencil mask in the same
frame, extracts avatar pixels, builds the glow pyramid, composites
glare, and applies the current NAES/toon color mapping.

This avoids Godot Environment Glow because that path is global. Here,
source selection needs to be avatar-specific and controlled by the stage
runtime.

It also avoids an extra SubViewport path because prior testing showed
camera-motion lag. The compositor path keeps the effect in the same
render frame.

## Stage Baseline

`StageVisualPreset` now keeps the stage environment neutral for this
effect:

- skybox is visible as background only
- skybox does not drive avatar/ground ambient light
- reflected light is disabled
- Godot Environment Glow is disabled
- Godot tonemap/adjustment stay neutral
- custom color mapping runs in the compositor

## Notes

`StageAvatarGlowCompositorEffect` currently owns both avatar glow and
toon color mapping. That is acceptable for this PR because it is the
only custom stage compositor today.

Before adding rim light, screen-space outlines, or more post effects,
compositor ownership should be split from individual features so effects
can register passes instead of replacing `Camera3D.Compositor`
independently.
This commit is contained in:
Lilia_Chen
2026-06-06 08:20:32 +08:00
committed by GitHub
parent 87e9495667
commit 5b0568bbb4
8 changed files with 1814 additions and 22 deletions
+30
View File
@@ -27,6 +27,9 @@ Godot-native desktop stage runtime project for `stage-tamagotchi`.
- Current sidecar view state starts when the Godot stage process starts and ends
when that process exits. The retained store code is not wired into the active
runtime path yet.
- G1.3 fixed stage presentation baseline: sky background, neutral grid ground,
center marker, direct light rig, neutral Godot environment post settings, and
avatar-only same-frame glow with the current toon color mapping.
## Directory Layout
@@ -155,6 +158,33 @@ The grid ground is visual-only. It does not move the avatar root away from
`(0, 0, 0)`, and its shader fades distant grid lines into the horizon color
without enabling volumetric fog.
The sky is visible as the viewport background, but it is not used as avatar or
ground ambient/radiance input. `StageVisualPreset` sets ambient light to a fixed
color, sets sky ambient contribution to `0`, disables reflected light, disables
Godot Environment Glow, and leaves Godot tonemap/adjustment neutral. The current
custom color mapping is applied by the stage compositor instead of Godot's
Environment adjustment controls.
## Avatar Glow And Color Mapping
The current avatar presentation path uses a camera-local compositor:
1. `StageAvatarGlowRuntime` assigns a `Compositor` to the active `Camera3D`.
2. The runtime marks loaded avatar `GeometryInstance3D` nodes with a
depth-tested `MaterialOverlay` that writes stencil reference `1`.
3. `StageAvatarGlowCompositorEffect` runs at `PostTransparent`, extracts
stencil-marked avatar pixels from the resolved scene color, builds the bloom
pyramid, composites avatar glare, and applies the current NAES/toon color
mapping before Godot's neutral output path.
This is not Godot Environment Glow and does not use material emission as the
avatar-style glow source. The color mapping currently lives in the same
compositor effect as avatar glow; before adding more post effects such as rim
light or screen-space outlines, pass orchestration and shared transient render
textures should move behind a shared post-process owner.
Design notes live in [`docs/rendering-effects.md`](docs/rendering-effects.md).
## Material Rendering Check
G1.3 includes a focused runtime material verification scene for the committed
@@ -0,0 +1,122 @@
# Rendering Effect Placement
This note records the current design direction for custom stage rendering
effects. The goal is to keep vendored MToon shaders stable unless an effect
truly needs to participate in base material shading.
## Current Direction
Prefer this split for avatar and stage visual effects:
```text
MToon / base material
Owns the avatar's normal toon shading and material response.
GeometryInstance3D.MaterialOverlay
Owns non-invasive per-mesh auxiliary passes, such as stencil, mask, ID, or
simple extra visual sources.
CompositorEffect
Owns same-frame screen-space work, such as glow diffusion, compositing, and
final custom color mapping before display output.
```
For avatar glow, this means:
- `StageAvatarGlowRuntime` marks avatar meshes with an overlay material that
depth-tests against the scene and writes stencil reference `1` without writing
scene depth.
- `StageAvatarGlowCompositorEffect` reads the stencil-marked scene color,
extracts the glow source, diffuses it, composites it, and applies the current
toon color mapping.
- The vendored V-Sekai MToon shader remains responsible for the avatar's base
material look.
Current implementation details:
- `StageAvatarGlowRuntime` currently assigns a new `Compositor` directly to the
active `Camera3D`. This is acceptable while avatar glow is the only stage
compositor, but multiple camera effects need a shared compositor owner.
- `StageAvatarGlowCompositorEffect` currently owns both avatar glow and the
stage toon color mapping. That is an implementation coupling, not the desired
long-term feature boundary.
- The effect owns its transient bloom textures. It uses Godot's
`FramebufferCacheRD` and `UniformSetCacheRD` for derived framebuffer and
uniform-set RIDs, while texture ownership stays local to the effect until a
shared post-process resource context exists.
- Godot Environment Glow is disabled in `StageVisualPreset`; avatar-style glow
source selection comes from the stencil overlay, not from material emission.
## Why Overlay First
`MaterialOverlay` is instance-local and renders an additional material over the
same geometry. It can add semantic render data without replacing the imported
MToon material.
Use overlay first when an effect can be represented as an extra whole-geometry
pass:
- stencil or object-mask tagging;
- avatar-only glow source selection;
- silhouette, outline, selection, or interaction masks;
- depth-tested x-ray or rim/shell source passes;
- temporary verification passes that should not mutate imported materials.
This avoids patching every vendored MToon shader variant for effects that do
not belong to the material's base shading model.
## Where Overlay Stops
Overlay is not a general replacement for shader ownership.
Do not rely on overlay alone when the effect needs to change the base material
result before post-processing:
- MToon ramp, shade color, or lighting response;
- shadow attenuation or direct-light behavior;
- normal-dependent toon bands inside the material;
- texture-driven or UV-driven per-surface masks unavailable to the overlay
pass;
- skin, hair, cloth, or accessory behavior that must differ inside one imported
mesh.
Those cases need an owned shader contract, an importer/material patch, or extra
authoring data that the overlay pass can read.
## Known Constraints
- `MaterialOverlay` is a single slot on each `GeometryInstance3D`. Multiple
effects cannot assign it independently without an owner that arbitrates the
slot.
- `Camera3D.Compositor` is also a single owner slot in the current stage code.
Additional post effects should be registered through a shared compositor owner
instead of independently replacing the camera compositor.
- The overlay material applies to the whole geometry and all surfaces unless
the mesh is split or the overlay shader has reliable mask data.
- Overlay adds draw work for every marked geometry. This is acceptable for the
current avatar path, but it should be measured before expanding it to many
scene objects.
- Transparent occluders may not block overlay-derived masks when they do not
write depth. Opaque depth-tested occluders should block the current avatar
glow stencil source.
- Screen-space diffusion, blur pyramids, and color mapping still belong in a
same-frame compositor. Overlay should produce source information, not replace
the compositor pass graph.
- The current color mapping is tied to `StageAvatarGlowCompositorEffect`.
Splitting color mapping into a stage-level post-process pass should happen
together with the next multi-effect refactor.
## Decision Rule
Use the least invasive layer that has the data and timing required by the
effect:
1. Use MToon or an owned material shader when the effect changes base material
shading.
2. Use `MaterialOverlay` when the effect only needs an extra per-mesh pass or
semantic mask.
3. Use `CompositorEffect` when the effect needs same-frame screen-space image
processing.
Patch vendored MToon only when the required behavior cannot be expressed as an
overlay source plus compositor work.
@@ -0,0 +1,110 @@
# Runtime Verification Scenes
Use a verification scene when a renderer change needs isolated visual evidence
that unit tests cannot provide. Keep the main stage WebSocket flow as the final
acceptance path for avatar presentation work.
## When To Use
- Use a temporary verification scene to isolate one renderer question, such as
alpha cutout behavior, outline geometry, stencil masks, or glow source
extraction.
- Use the main stage plus a temporary WebSocket host before accepting any change
that affects the shipped stage view.
- Do not rely on a verification scene as CI evidence when it depends on local
fixtures that are not tracked by git.
## Scene Shape
For visual renderer checks, create a minimal scene under
`tests/<check-name>/<checkName>.tscn` with one `Node3D` root and a script that
uses the same runtime entry points as the feature under test. C# is preferred
when the check needs stage C# runtime objects. A GDScript scene is acceptable
for non-visual importer/material checks, such as
`tests/material-rendering-check/materialRenderingCheck.tscn`.
A visual renderer script should:
1. Set a deterministic viewport size with `DisplayServer.WindowSetSize`.
2. Apply the normal stage baseline with `StageVisualPreset.Apply(this)`.
3. Load the target avatar or fixture through the same runtime path being tested.
4. Create one active `Camera3D` and use fixed poses.
5. Install the renderer feature runtime being tested. For avatar glow, create
`StageAvatarGlowRuntime(camera)` and call `UseAvatar(avatar)` after loading
the avatar; `StageVisualPreset.Apply(this)` alone does not install the glow
compositor.
6. Wait several frames before each capture so imports, materials, and compositor
resources settle.
7. Save PNG captures to `Path.Combine(Path.GetTempPath(), "<check-name>")`.
8. Print compact diagnostics that can be compared across runs.
9. Quit with `GetTree().Quit(0)` after the final capture.
## Fixture Rules
- Prefer tracked fixtures when the check is meant to remain in the repository.
- If a check needs a private or large local VRM, keep the scene temporary and
document the required local path in the experiment notes.
- Do not commit a verification scene that cannot run from a clean checkout
unless the limitation is intentional and documented next to the scene.
## Renderer Caveats
- Avatar glow uses a depth-tested stencil overlay so opaque objects between the
camera and avatar occlude the glow source. Transparent occluders may not block
the mask if their materials do not write depth. This is a normal transparent
rendering limitation; verify those cases with a dedicated scene before relying
on them for shipped visuals.
## Capture Pattern
Use frame-number gates instead of timers so the script stays deterministic:
```csharp
public override void _Process(double delta)
{
_frame++;
if (_frame == 10)
{
SaveCapture("baseline");
return;
}
if (_frame == 22)
{
EnableRendererFeatureUnderTest();
SaveCapture("enabled");
GetTree().Quit(0);
}
}
```
The capture helper should create the output directory, call
`GetViewport().GetTexture().GetImage()`, save the image, and throw if
`SavePng` fails.
## Run Command
Launch the scene with the Godot mono binary:
```powershell
C:\Godot_v4.6.2-stable_mono_win64\Godot_v4.6.2-stable_mono_win64.exe `
--path "D:\TAworkspace\AIRIworkspace\airi\engines\stage-tamagotchi-godot" `
--scene "res://tests/<check-name>/<checkName>.tscn"
```
For shipped stage visuals, follow this with the main stage verification flow:
1. Start a local WebSocket host.
2. Launch the main Godot stage with `-- --airi-ws-url=ws://127.0.0.1:<port>/`.
3. Wait for `stage.ready`.
4. Send `host.scene.apply` with the target VRM.
5. Wait for `scene.applied`.
6. Bring the Godot window to the front and capture it.
7. Send `host.shutdown`.
## Cleanup
After the visual question is answered, either promote the scene into a stable
tracked check with tracked fixtures, or delete the temporary scene and keep only
the experiment result in the relevant design notes.
@@ -32,7 +32,8 @@ public partial class StageRoot : Node3D
private StageBridge _bridge = null!;
private StageSceneController _sceneController = null!;
private StageViewController _viewController = null!;
private StageCameraInputController _viewInputController = null!;
private StageCameraInputController _cameraInputController = null!;
private StageAvatarGlowRuntime _avatarGlowRuntime = null!;
private StageViewRuntime _viewRuntime = null!;
private string _activeSceneModelId;
private bool _shutdownRequested;
@@ -44,8 +45,10 @@ public partial class StageRoot : Node3D
StageVisualPreset.Apply(this);
var avatarRoot = ResolveAvatarRoot();
var camera = ResolveCamera();
_sceneController = new StageSceneController(avatarRoot, new VrmAvatarLoader());
InitializeViewRuntime(avatarRoot);
InitializeViewRuntime(avatarRoot, camera);
_avatarGlowRuntime = new StageAvatarGlowRuntime(camera);
var webSocketUrl = ResolveWebSocketUrl();
if (string.IsNullOrWhiteSpace(webSocketUrl))
@@ -78,13 +81,19 @@ public partial class StageRoot : Node3D
_bridge.Poll();
_viewRuntime?.Process(delta);
_viewInputController?.Process(delta);
_cameraInputController?.Process(delta);
}
/// <inheritdoc/>
public override void _ExitTree()
{
_avatarGlowRuntime?.Dispose();
}
/// <inheritdoc/>
public override void _Input(InputEvent @event)
{
_viewInputController?.HandleInput(@event);
_cameraInputController?.HandleInput(@event);
}
private void HandleBridgeOpened()
@@ -212,6 +221,7 @@ public partial class StageRoot : Node3D
// avatar already loaded.
var avatar = _sceneController.Apply(payload);
_viewController?.UseAvatar(avatar);
_avatarGlowRuntime?.UseAvatar(avatar);
_viewRuntime?.BootstrapForAvatar();
_activeSceneModelId = payload.ModelId;
}
@@ -292,15 +302,15 @@ public partial class StageRoot : Node3D
return string.Empty;
}
private void InitializeViewRuntime(Node3D avatarRoot)
private void InitializeViewRuntime(Node3D avatarRoot, Camera3D camera)
{
var cameraController = new StageCameraPoseController(ResolveCamera());
var cameraController = new StageCameraPoseController(camera);
_viewController = new StageViewController(avatarRoot, cameraController);
_viewRuntime = new StageViewRuntime(_viewController);
_viewRuntime.SnapshotReady += payload =>
_bridge.SendEnvelope("stage.view.snapshot", payload);
_viewRuntime.ErrorReady += payload => _bridge.SendEnvelope("stage.view.error", payload);
_viewInputController = new StageCameraInputController(_viewRuntime, cameraController);
_cameraInputController = new StageCameraInputController(_viewRuntime, cameraController);
}
private void SendSceneError(string message)
@@ -0,0 +1,119 @@
using System;
using System.Collections.Generic;
using Godot;
/// <summary>
/// Owns avatar-only stencil masking and the camera-local glow compositor.
/// </summary>
public sealed class StageAvatarGlowRuntime : IDisposable
{
private const int AvatarStencilReference = 1;
private readonly Camera3D _camera;
private readonly StageAvatarGlowCompositorEffect _avatarGlowEffect;
private readonly StandardMaterial3D _avatarGlowMaskMaterial;
private readonly Dictionary<GeometryInstance3D, Material> _previousOverlays = new();
private bool _disposed;
public StageAvatarGlowRuntime(Camera3D camera)
{
_camera = camera ?? throw new ArgumentNullException(nameof(camera));
_avatarGlowEffect = new StageAvatarGlowCompositorEffect(AvatarStencilReference);
_avatarGlowMaskMaterial = CreateAvatarGlowMaskMaterial();
// NOTICE:
// Camera3D owns compositor effects. Avatar glow is the only stage compositor today;
// replace this direct assignment with a shared owner before adding more camera passes.
var compositor = new Compositor();
compositor.CompositorEffects = new Godot.Collections.Array<CompositorEffect>
{
_avatarGlowEffect,
};
_camera.Compositor = compositor;
}
public void UseAvatar(Node avatar)
{
if (_disposed)
{
return;
}
ClearAvatarMask();
if (avatar == null)
{
_avatarGlowEffect.Enabled = false;
return;
}
MarkAvatarMask(avatar);
_avatarGlowEffect.Enabled = _previousOverlays.Count > 0;
}
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
_avatarGlowEffect.Enabled = false;
ClearAvatarMask();
if (_camera.Compositor?.CompositorEffects.Contains(_avatarGlowEffect) == true)
{
_camera.Compositor = null;
}
_avatarGlowEffect.ReleaseRenderingResources();
}
private StandardMaterial3D CreateAvatarGlowMaskMaterial() => new()
{
AlbedoColor = new Color(0.0f, 0.0f, 0.0f, 0.0f),
CullMode = BaseMaterial3D.CullModeEnum.Disabled,
// NOTICE:
// Keep the mask depth-tested but non-writing: occluded avatar fragments do not mark
// stencil, and the mask pass does not mutate scene depth.
DepthDrawMode = BaseMaterial3D.DepthDrawModeEnum.Disabled,
DisableFog = true,
NoDepthTest = false,
RenderPriority = (int)Material.RenderPriorityMax,
ShadingMode = BaseMaterial3D.ShadingModeEnum.Unshaded,
StencilCompare = BaseMaterial3D.StencilCompareEnum.Always,
StencilFlags = (int)BaseMaterial3D.StencilFlagsEnum.Write,
StencilMode = BaseMaterial3D.StencilModeEnum.Custom,
StencilReference = AvatarStencilReference,
Transparency = BaseMaterial3D.TransparencyEnum.Alpha,
};
private void MarkAvatarMask(Node node)
{
if (node is GeometryInstance3D geometry)
{
// NOTICE:
// MaterialOverlay is a single instance-level extra pass. Save and restore it while
// this runtime uses the slot as a temporary avatar mask producer.
_previousOverlays[geometry] = geometry.MaterialOverlay;
geometry.MaterialOverlay = _avatarGlowMaskMaterial;
}
foreach (Node child in node.GetChildren())
{
MarkAvatarMask(child);
}
}
private void ClearAvatarMask()
{
foreach (var (geometry, previousOverlay) in _previousOverlays)
{
if (GodotObject.IsInstanceValid(geometry))
{
geometry.MaterialOverlay = previousOverlay;
}
}
_previousOverlays.Clear();
}
}
@@ -91,12 +91,14 @@ public static class StageVisualPreset
}
""";
private static readonly Color GroundColor = new(0.27f, 0.34f, 0.37f, 1.0f);
private static readonly Color HorizonMistColor = new(0.62f, 0.73f, 0.76f, 0.82f);
private static readonly Color GroundColor = new(0.31f, 0.31f, 0.31f, 1.0f);
private static readonly Color HorizonMistColor = new(0.68f, 0.68f, 0.68f, 0.82f);
private static readonly Color MinorGridColor = new(1.0f, 1.0f, 1.0f, 0.20f);
private static readonly Color MajorGridColor = new(1.0f, 1.0f, 1.0f, 0.38f);
private static readonly Color CenterGridColor = new(1.0f, 1.0f, 1.0f, 0.72f);
private static readonly Color CenterMarkerColor = new(0.72f, 0.94f, 1.0f, 0.92f);
private static readonly Color AmbientLightColor = new(0.06f, 0.06f, 0.06f, 1.0f);
private static readonly Color StageBackgroundColor = new(0.015f, 0.014f, 0.018f, 1.0f);
/// <summary>
/// Applies the stage visual preset under the provided root node.
@@ -117,6 +119,7 @@ public static class StageVisualPreset
throw new ArgumentNullException(nameof(stageRoot));
}
ConfigureViewport(stageRoot.GetViewport());
RemoveExistingPreset(stageRoot);
var visualRoot = new Node3D
@@ -131,6 +134,17 @@ public static class StageVisualPreset
visualRoot.AddChild(CreateLightingRig());
}
private static void ConfigureViewport(Viewport viewport)
{
if (viewport == null)
{
return;
}
viewport.UseHdr2D = true;
viewport.UseDebanding = true;
}
private static void RemoveExistingPreset(Node stageRoot)
{
var existingPreset = stageRoot.GetNodeOrNull<Node>(VisualPresetRootNodeName);
@@ -157,24 +171,25 @@ public static class StageVisualPreset
};
var environment = new GodotEnvironment
{
AmbientLightColor = AmbientLightColor,
AmbientLightEnergy = 0.26f,
AmbientLightSkyContribution = 0.48f,
AmbientLightSource = GodotEnvironment.AmbientSource.Sky,
BackgroundEnergyMultiplier = 1.06f,
AmbientLightSkyContribution = 0.0f,
AmbientLightSource = GodotEnvironment.AmbientSource.Color,
AdjustmentBrightness = 1.0f,
AdjustmentColorCorrection = null,
AdjustmentContrast = 1.0f,
BackgroundColor = StageBackgroundColor,
BackgroundEnergyMultiplier = 1.0f,
// Show the sky as a background while keeping ambient and reflections neutral.
BackgroundMode = GodotEnvironment.BGMode.Sky,
ReflectedLightSource = GodotEnvironment.ReflectionSource.Sky,
ReflectedLightSource = GodotEnvironment.ReflectionSource.Disabled,
Sky = sky,
// Keep MToon/NPR avatars out of filmic tone mapping, then apply a small
// stylized display grade. The VRM materials already use source_color
// texture inputs; the remaining gap to three-stage is presentation color,
// not importer-side texture conversion.
// Three-stage disables tone mapping per MToon material; Godot applies
// environment tone mapping globally, so filmic curves wash avatar colors out.
AdjustmentContrast = 1.03f,
AdjustmentEnabled = true,
AdjustmentSaturation = 1.24f,
AdjustmentSaturation = 1.0f,
AdjustmentEnabled = false,
GlowEnabled = false,
TonemapExposure = 1.0f,
TonemapMode = GodotEnvironment.ToneMapper.Linear,
TonemapWhite = 1.0f,
};
return new WorldEnvironment
@@ -9,6 +9,7 @@
<GodotProjectDirBase64>$([MSBuild]::ConvertToBase64('$(GodotDir)'))</GodotProjectDirBase64>
<ImplicitUsings>false</ImplicitUsings>
<Nullable>disable</Nullable>
<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
</PropertyGroup>
<ItemGroup>
@@ -16,6 +17,7 @@
<CompilerVisibleProperty Include="GodotProjectDirBase64" />
<CompilerVisibleProperty Include="GodotSourceGenerators" />
<CompilerVisibleProperty Include="IsGodotToolsProject" />
<Compile Remove="..\..\.godot\**\*.cs" />
<ProjectReference Include="..\..\stage-tamagotchi-godot.csproj" />
</ItemGroup>
</Project>