The Wayback Machine - https://web.archive.org/web/20201119080420/https://github.com/mrdoob/three.js/pull/20421
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NodeMaterial: WebGPU #20421

Draft
wants to merge 11 commits into
base: dev
from
Draft

NodeMaterial: WebGPU #20421

wants to merge 11 commits into from

Conversation

@sunag
Copy link
Collaborator

@sunag sunag commented Sep 26, 2020

@Mugen87 @mrdoob Very, very draft to facilitate the understanding... this is a step implement to WebGL too, is my new approach, merger node slots directly in ShaderLiib... The next step in this PR is implement uniforms.

I fork the example webgpu_sandbox.html and added just some lines code:

import FloatNode from './jsm/renderers/nodes/inputs/FloatNode.js';
import Vector3Node from './jsm/renderers/nodes/inputs/Vector3Node.js';
import OperatorNode from './jsm/renderers/nodes/math/OperatorNode.js';

materialBox.colorNode = new OperatorNode( '+', new FloatNode( .5 ).setConst( true ), new Vector3Node( new THREE.Vector3( 0, 0, 1 ) ) );
materialBox.opacityNode = new FloatNode( .5 );
materialBox.transparent = true;
@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 28, 2020

@Mugen87 folow ( #20421 (comment) ) ( f738e6a )
/jsm/renderers/nodes agnostic renderer node system
/jsm/renderers/webgpu/WebGPUNodeBuilder WebGPU node builder

@sunag sunag mentioned this pull request Sep 28, 2020
@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 28, 2020

I don't think it is necessary to make a new example for nodes anymore. Do you mind modifying the existing webgpu_sandbox_nodes example?

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 28, 2020

@Mugen87 You have any idea for that error message I am getting:
If I add a single vec3 in nodeUniforms it works done. 24 bytes seems the correct size for two vec3.

image

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 28, 2020

You have any idea for that error message I am getting:

This message did not pop up in my tests so far but I assume that vec3s must start on a 16-byte boundary. Can you please change the byteLength of Vector3Uniform to 16 for now?

/cc @Kangz @takahirox

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 28, 2020

@mrdoob This PR looks really great! In context of design, I would highlight one important aspect since it will accompany node material from the very beginning.

With the current implementation, nodes have the responsibility to control their build/generation. This process is mainly done with a node builder with is injected as a parameter. One could consider a different architecture such that nodes do not have any build/generate methods but just describe themselves. You then have a centralized entity that traverses through the node composition and performs the build.

I personally favor the current approach since it's similar to having toJSON() and fromJSON() on object level instead of a single complex component that handles the entire serialization/deserialization. It's easier to encapsulate code and hide component-specific logic (in this case node specific).

@Kangz
Copy link

@Kangz Kangz commented Sep 28, 2020

@Mugen87 You have any idea for that error message I am getting:
If I add a single vec3 in nodeUniforms it works done. 24 bytes seems the correct size for two vec3.

image

Mmmmh it should work with the std140 layout, but WebGPU seems to think it is 28 bytes probably because of a bad array stride? Could you share permalink of the GLSL->SPIRV compilation in http://shader-playground.timjones.io/ ?

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 28, 2020

I don't know how to share a permalink. Here is some sample code:

#version 450
layout(set = 0, binding = 2) uniform MaterialUniforms {
    float opacity;
    vec3 color1;
    vec3 color2;
} materialUniforms;

layout(set = 0, binding = 3) uniform sampler mySampler;
layout(set = 0, binding = 4) uniform texture2D myTexture;

layout(location = 0) in vec2 vUv;
layout(location = 0) out vec4 outColor;

void main() {
    outColor = texture( sampler2D( myTexture, mySampler ), vUv );
    outColor.a *= materialUniforms.opacity;
}

It seems that even if a single float only takes 4 bytes, it's necessary to put 12 bytes as padding when starting with color1. And after defining color1, you need 4 additional bytes before defining color2.

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 28, 2020

I guess we need the logic from the UBO PR that determines the total buffer size according to the STD140 layout.

https://github.com/mrdoob/three.js/pull/15562/files#diff-cba50499b84f8d3f3690b625a907ba74R209

@Kangz
Copy link

@Kangz Kangz commented Sep 28, 2020

Ah yes, I misread. It is expect that the buffer would use 28 bytes in std140 layout because things are mostly padded to 16 bytes all the time. Like you said, the logic in the UBO PR should help.

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 28, 2020

Okay, added: 1cf4431

@sunag After rebasing the mentioned warnings should be gone.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 28, 2020

@sunag After rebasing the mentioned warnings should be gone.

@Mugen87 It work now. Thanks guys for quick fix.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 29, 2020

I think in changing the properties to material.colorNode and material.opacityNode because the implementation will be easier including in WebGL context.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 29, 2020

I add Nodes support for PointsMaterial ( 99cced5) in some minutes, very simple... This can be used for any material of threejs too.


}

let uniformGroup;

This comment has been minimized.

@Mugen87

Mugen87 Sep 29, 2020
Collaborator

This variable should be called just uniform. Instances of WebGPUUniform are not related to group.

@@ -19,6 +21,8 @@ class WebGPURenderPipelines {
this.glslang = glslang;
this.sampleCount = sampleCount;

this.nodeBindings = new WeakMap();

This comment has been minimized.

@Mugen87

Mugen87 Sep 29, 2020
Collaborator

This can be renamed to just bindings.

@@ -200,6 +212,12 @@ class WebGPURenderPipelines {

}

getNodeBindings( object ) {

This comment has been minimized.

@Mugen87

Mugen87 Sep 29, 2020
Collaborator

And this to getBindings().

import NodeSlot from '../nodes/core/NodeSlot.js';
import NodeBuilder from '../nodes/core/NodeBuilder.js';

class WebGPUNodeUniformsGroup extends WebGPUUniformsGroup {

This comment has been minimized.

@Mugen87

Mugen87 Sep 29, 2020
Collaborator

Can you also use just WebGPUUniformsGroup? Eventually, bindings should only be managed over the node material system.

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 29, 2020

Nice! I think we should manage now all existing uniforms over the node material system so the following code block can be eliminated:

if ( material.isMeshBasicMaterial ) {
bindings = this._getMeshBasicBindings();
} else if ( material.isPointsMaterial ) {
bindings = this._getPointsBasicBindings();
} else if ( material.isLineBasicMaterial ) {
bindings = this._getLinesBasicBindings();
} else {
console.error( 'THREE.WebGPURenderer: Unknwon shader type.' );
}

Instead of this code, we just have this:

const bindings = this.pipelines.getBindings( object );
@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 29, 2020

When you look at the shader code in WebGPURenderPipelines, you can see that the UBOs CameraUniforms and ModelUniforms are required in all programs so far. It's essentially a replacement for the pre-defined code in WebGLProgram.

When managing all bindings over the node material system, I suggest to finally review and then merge this PR since it is a proper first step.

createUniformFromNode( node, shaderStage, type ) {

const uniformNode = super.createUniformFromNode( node, shaderStage, type )
const nodeData = this.createDataFromNode( node, shaderStage );

This comment has been minimized.

@Mugen87

Mugen87 Sep 29, 2020
Collaborator

This call irritated me a bit. You call createDataFromNode() here although this method was already called inside of createUniformFromNode(). I understand that you only call it to get the (already existing) node data.

This code will be more clear when it is align to WebGPUProperties or WebGLProperties. Meaning to transform it to a getter. The implementation of createDataFromNode() and createDataFromNode () is already similar to:

function get( object ) {
let map = properties.get( object );
if ( map === undefined ) {
map = {};
properties.set( object, map );
}
return map;
}

The method creates a cached object if necessary. But this creation is not something the caller has not know about. Ideally, the code would be:

const uniformNode = super.getUniformNode( node, shaderStage, type ) 
const nodeData = this.getUniformNodeData( node, shaderStage );

In my mind create*() methods are factory methods that always return a new instance when called. This is currently not true for both uniform methods.

This comment has been minimized.

@sunag

sunag Sep 29, 2020
Author Collaborator

This was get before getNodeData I will keep and I have that change the logic of createUniformFromNode... This code block changed a lot from yesterday to today because of the implementation of texture2d uniform...

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 29, 2020

When managing all bindings over the node material system, I suggest to finally review and then merge this PR since it is a proper first step.

I am already thinking about cleaning and Ready for review this PR after implement TextureNode.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 29, 2020

Instead of this code, we just have this:
const bindings = this.pipelines.getBindings( object );

if ( material.isMeshBasicMaterial ) { 
  
 	bindings = this._getMeshBasicBindings(); 
  
 } else if ( material.isPointsMaterial ) { 
  
 	bindings = this._getPointsBasicBindings(); 
  
...

It is not clear to me where you want to move these functions, it is for WebGPURenderPipelines ?

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 29, 2020

It is not clear to me where you want to move these functions, it is for WebGPURenderPipelines ?

I thought we could remove them and add the common bindings (CameraUniforms and ModelUniforms) with the node material system.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 29, 2020

I thought we could remove them and add the common bindings (CameraUniforms and ModelUniforms) with the node material system.

This would change a lot, a very simple implementation would mix the logic of the procedural with natives... I don't think it's appropriate for the moment.

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 29, 2020

No problem. We can experiment later with this. But the goal should be that all bindings are defined by the node material system at some point (and not be hard-wired in the renderer). In general, the renderer should not perform any checks for materials anymore (e.g. test for isMeshBasicMaterial). Only then it can be implemented material agnostic. We should focus on that after merging this PR.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 30, 2020

If so, I see no reason not to leave the properties hybrid right now because code block like this don't make much sense... thinking ahead.

layout(set = 0, binding = 2) uniform OpacityUniforms {
	float opacity;
} opacityUniforms;

layout(set = 0, binding = 3) uniform sampler mySampler;
layout(set = 0, binding = 4) uniform texture2D myTexture;
@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Sep 30, 2020

Yeah, the current shader code was more or less developed for testing. If we can consolidate this in order to avoid unnecessary bindings, we should do so.

We just have to be aware that UBOs might be shared across multiple shader programs. So e.g. CameraUniforms should be a unique UBO which is reused.

@sunag
Copy link
Collaborator Author

@sunag sunag commented Sep 30, 2020

We just have to be aware that UBOs might be shared across multiple shader programs. So e.g. CameraUniforms should be a unique UBO which is reused.

Yes, this feature already exists in the current node material system we should only bring it to the new node system: ( https://github.com/mrdoob/three.js/blob/dev/examples/jsm/nodes/accessors/CameraNode.js#L100 )

@Mugen87
Copy link
Collaborator

@Mugen87 Mugen87 commented Nov 15, 2020

@sunag Do you think you can finish the PR in the next time by adding support for TextureNode? Otherwise I would give it a try since I've made myself a little bit more familiar with the node material system.

It would be good to have a basic version of the node material in place so we can move on with developing the renderer at other places.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.
Morty Proxy This is a proxified and sanitized view of the page, visit original site.