Rendering a model with different materials

Hi,

In my game, I’m using the ModelComponent, with each Model having a single mesh representing a small chunk of my game level. These chunks are 4x4 tiles. Now the problem I see is that my SharpDX code would have used Vector3 for texcoord { U, V, textureIndex }, the textureIndex being a look-up into a Texture2DArray. And I would have a custom hlsl shader to fetch the approprirate texture for each part of the mesh. Using Paradox Model system however, I don’t know how to setup this kind of multi-textured mesh, while still keeping the draw calls at 1 / chunk.

So I need some guidance with how to approach this problem.

Thanks,

Gavin

Edit : Until I can work out how to use TextureArrays for mesh texturing, I suppose i could employ a texture sheet solution, and my U,V’s can map into a single texture sheet.

I assume you are building your own VertexArrayObject from VertexPositionNormalTexture already. You will first have to define your custom vertex layout and use that instead:

[StructLayout(LayoutKind.Sequential)]
public struct MyCustomVertex: IVertexWindable
{
    public Vector3 Position;
    public Vector3 Normal;
    public Vector2 TextureCoordinate;
    public float TextureIndex; // You could also combine these into a Vector3 again, if you like

    public static readonly int Size = 36;

    public static readonly VertexDeclaration Layout = new VertexDeclaration(new VertexElement[]
    {
        VertexElement.Position<Vector3>(),
        VertexElement.Normal<Vector3>(),
        VertexElement.TextureCoordinate<Vector2>(),
        new VertexElement("MY_SEMANTIC", PixelFormat.R32_Float),
    });

    public void FlipWinding()
    {
        TextureCoordinate.X = (1.0f - TextureCoordinate.X);
    }
}

Next you need to create a custom effect. I assume you are currently using the effect from the default game template (MyGameEffectMain.pdxfx). If not let me know. That effect has a variable diffuseAlbedo of type ComputeColor which defines the final color.
The easiest way to change that is to first define our own ComputeColor:

// ComputeMyCustomColor.pdxsl
class ComputeMyCustomColor : ComputeColor
{
    // EDIT: Forgot stage modifier. Needed if used inside 'mixin compose'
    stage Texture2DArray TextureArray // This will be available as 'ComputeMyCustomColorKeys.TextureArray'

    stage stream float2 TexCoord : TEXCOORD0;
    stage stream float TextureIndex : MY_SEMANTIC;

    override float4 Compute()
    {
        return TextureArray.Sample(Texturing.Sampler, float3(streams.TexCoord, streams.TextureIndex));
    }
};

That will probably be similar to your SharpDX shader. Next, modify MyGameEffectMain to use that color computation. Replace

    if (MaterialParameters.AlbedoDiffuse != null)
        mixin compose albedoDiffuse = MaterialParameters.AlbedoDiffuse;

with

    mixin compose albedoDiffuse = ComputeMyCustomColor;

Finally, add your texture array to each ModelComponent.Parameters (or to RenderSystem.Pipeline.Parameters, for simplicity):

    modelComponent.Parameters.Set(ComputeMyCustomColorKeys.TextureArray, myTextureArray);

That should be it!

There are multiple ways to do this of cause. You could dump the MyGameEffectMain and use a much simpler one, closer to SimpleEffect. You could also make ComputeMyCustomColor a bit more generic, so it would be easier to configure from Paradox Studio.

5 Likes

OK, thanks for this, I’ve worked with custom vertex formats in SharpDX so that part is clear. And yes I’m using the default GameEffectMain as you guessed. I was trying to avoid Paradox’s shader language, because it looks completely alien to me, it has a few unusual semantics such as stage, stream, mixin, compose. And I’m yet to work out how all this get’s turned into hlsl. I’ll have to spend some time getting to grips with it over the evening and tomorrow.

@jwollen answers is great for handling such a case with current Paradox.

Also, in a future release, we will provide a way to pack textures for models into one or multiple textures at build time (while rewriting UVs), and we could definitely support for using Texture2DArray in this case. Note that we have already a texture packer that will come for sprite and image groups in the next release.

1 Like

The ModelComponent ParameterKey is not available. In my case it is ‘ComputeColorUVIndexKeys’, I’m assuming that after compiling the assets in GameStudio, the EffectShader should be available in code.

I also tried

modelComponent.Parameters.Set( (ParameterKey)ParameterKeys.FindByName("ComputeColorUVIndexKeys.TextureArray"), game.SurfaceTextureArray);

which didn’t work.

Also, do I need to build the TextureArray in code. I have some SharpDX code to build a texture array. Should I duplicate that or is there an easier way ?

Edit : Reading through the materials tutorials to see if I’m missing something with linking up the nodes and shaders.

When you save the shader file, a .cs file with the same name should be generated, which defines the parameter keys. Could you check if that’s there? If so, resave it in Visual Studio and see if there is a red squiggle below the file name. There may be a syntax error in your shader.

I had created the effect shader in GameStudio, and tried to link it to the material there without success.

But it sounds like you meant write the pdxsl file in Visual Studio so I was doing that wrong. In VS, in the Effects folder, I have created a file called: ComputeColorTest.pdxsl, and that autogenerates a cs file as you say. And after adding:

using SiliconStudio.Paradox.Effects.Modules;

i can see the shader constant in my Chunk : Entity class code. So I can now do this (note that I’m using Color as test):

modelComponent.Parameters.Set(ComputeColorTestKeys.Color, new Vector4(1, 0, 0, 1));

Now I’m trying to attach what i assume is an Effect (ComputeColorTest) to the mesh.

Edit : So I have finally got a simple custom shader running through the ModelRenderer, by using the edit you recommended in EffectMain, which is passed to ModelRenderer constructor. However, I can’t seem to to set the parameter (cbuffer variable) from code, so the following produced transparent geometry:

    class ComputeColorTest : ComputeColor
    {
        float4 Color;
        override float4 Compute()
        {
            return Color;
        }
    };

// and in Chunk : Entity class:
modelComponent.Parameters.Set(ComputeColorTestKeys.Color, new Vector4(1, 1, 1, 1));

But this works:

 
override float4 Compute()
{
    return float4(1,0,0,1);
}

So I finally got cbuffer values updating by using ‘stage’ keyword before my variable. I don’t know what stage does, even after reading documentation, but it seems to be required.

stage float4 Color;

Ah yes, i overlooked that. Sorry!

I’m not sure of every implication of stage yet, either. From my understanding, it declares the variable globally to the final effect. Otherwise, it’s local to the current shader class. If you use a shader class as a mixin directly, they behave similar, but using mixin compose albedoDiffuse will declare non-stage variables in the scope of albedoDiffuse.
It’s useful to think in terms of ‘compositions’, the final effect being the ‘stage composition’.

Maybe a staff member can clarify further.

1 Like

So I’ve worked out how to load a texture array. I’ll put the code up here, because I think it’s quite difficult to get working. The function copies the format of the first texture, and i have no checking for mixed source formats at the moment. It will load mip-maps, again based upon the number of mip-map levels in the first texture. There seems to be an issue with format, as I was getting BGRA while the files were RGBA, I will have to look into this later. There may be one or two optimizations as well, that i need to consider (such as the duplication of textures).

/// 
        /// Load the given named texture assets into a Texture2DArray, using the format of the first texture.
        /// TODO : Provide error checking and resolution for format inconsistencies.
        /// 
        /// The names of the texture assets.
        /// The texture array.
        public Texture2D LoadTextureArray(params String[] textureAssetNames)
        {
            // the individual textures as loaded from the AssetManager
            List singles = new List();

            // load each texture through AssetManager
            for (int i = 0; i < textureAssetNames.Length; i++)
            { 
                singles.Add((Texture2D)Asset.Load(textureAssetNames[i]));
            }
            TextureDescription singleDesc = singles[0].Description;

            // copy textures into staged textures (required for loading data into array)
            Texture[] stagedSingles = new Texture2D[textureAssetNames.Length];
            TextureDescription stagedDesc = singleDesc;
            stagedDesc.Usage = GraphicsResourceUsage.Staging;
            
            for (int i = 0; i < textureAssetNames.Length; i++)
            {
                stagedSingles[i] = Texture2D.New(GraphicsDevice, singles[i].GetDataAsImage(), TextureFlags.None, GraphicsResourceUsage.Staging);
            }
            
            // create the texture array (which is a form of Texture2D)
            TextureDescription arrayDesc = new TextureDescription()
            {
                ArraySize = singles.Count,
                Depth = 1,
                Dimension = TextureDimension.Texture2D,
                Flags = TextureFlags.ShaderResource,
                Format = singleDesc.Format,
                Height = singleDesc.Height,
                Level = singleDesc.Level,
                MipLevels = singleDesc.MipLevels,
                Usage = GraphicsResourceUsage.Default,
                Width = singleDesc.Width
            };
            Texture2D textureArray = Texture2D.New(GraphicsDevice, arrayDesc);
            
            // copy each texture (and each mip level ) into the texture array
            for (int texIndex = 0; texIndex < singles.Count; texIndex++)
            {
                for (int mipLevel = 0; mipLevel < arrayDesc.MipLevels; mipLevel++)
                {
                    int srcSubResourceIndex = stagedSingles[texIndex].GetSubResourceIndex(0, mipLevel);
                    int desSubResourceIndex = textureArray.GetSubResourceIndex(texIndex, mipLevel);
                    MappedResource mappedTexture2D = GraphicsDevice.MapSubresource(stagedSingles[texIndex], srcSubResourceIndex, MapMode.Read);
                    GraphicsDevice.CopyRegion(mappedTexture2D.Resource, srcSubResourceIndex, null, (GraphicsResource)textureArray, desSubResourceIndex); 
                    GraphicsDevice.UnmapSubresource(mappedTexture2D);
                }
            }

            // dispose single textures
            for (int i = 0; i < singles.Count; i++)
            {
                singles[i].Dispose();
                stagedSingles[i].Dispose();
            }
            
            return textureArray;
        }

A bit more concise, adding some format checks and keeping the packing on CPU:

public Texture2D LoadTextureArray(params string[] textureAssetNames)
{
    Image imageArray = null;
    Texture2D result;

    try
    {
        for (int slice = 0; slice < textureAssetNames.Length; slice++)
        {
            Image image = Asset.Load<Image>(textureAssetNames[slice]);

            try
            {
                var description = image.Description;

                if (description.ArraySize > 1 || description.Dimension != TextureDimension.Texture2D)
                    throw new InvalidOperationException("Invalid texture dimension");

                description.ArraySize = textureAssetNames.Length;

                if (imageArray == null)
                {
                    imageArray = Image.New2D(description.Width, description.Height, description.MipLevels, description.Format, description.ArraySize);
                }
                else
                {
                    if (!description.Equals(imageArray.Description))
                        throw new InvalidOperationException("Image descriptions do not match");
                }

                for (int mipIndex = 0; mipIndex < description.MipLevels; mipIndex++)
                    image.PixelBuffer[0, mipIndex].CopyTo(imageArray.PixelBuffer[slice, mipIndex]);
            }
            finally
            {
                Asset.Unload(image);
            }
        }

        result = Texture2D.New(GraphicsDevice, imageArray);
    }
    finally
    {
        Utilities.Dispose(ref imageArray);
    }

    return result;
}
1 Like