Running Microsoft DXC in Docker
in Pipeline on Docker, Shaders, Linux
Microsoft’s new DirectX shader compiler (DXC) is based on LLVM and Clang, which has been traditionally a cross platform codebase, but became Windows-centric through COM, SAL, etc. used to support DirectX 12 shaders. Recently, Google listened (thank you!) to a number of requests from myself and others (here and here) to refactor the codebase to support Linux and macOS compilation.
I have been pursuing cloud based shader compilation for a while, in order to scale our very slow shader compilation pipelines to greatly improve developer iteration time. Running content pipelines on Windows-based virtual machines is not a feasible approach due to concerns of cost, maintainability, and robustness.
With Linux compilation support, it is now possible to run the DXC compiler within a Docker container, and scale it out in a Kubernetes cluster (like GKE).
FROM ubuntu:18.04 as dxc_builder
RUN apt-get update && \
apt-get install -y \
software-properties-common \
build-essential \
git \
cmake \
ninja-build \
python
ENV DXC_BRANCH=master
ENV DXC_REPO=https://github.com/google/DirectXShaderCompiler.git
ENV DXC_COMMIT=f45c5766277627d2b9c24b3e265701c961d75557
WORKDIR /dxc
RUN mkdir -p /dxc && \
git clone --recurse-submodules -b ${DXC_BRANCH} ${DXC_REPO} /dxc && \
git checkout ${DXC_COMMIT} && \
git reset --hard
RUN mkdir -p /dxc/build && \
cd /dxc/build && \
cmake ../ -GNinja -DCMAKE_BUILD_TYPE=Release $(cat ../utils/cmake-predefined-config-params) && \
ninja
ENTRYPOINT ["/dxc/build/bin/dxc"]
The above Dockerfile has been published to Docker Hub as gwihlidal/dxc.
The published image can be invoked with:
$ docker run --rm gwihlidal/dxc -help
The host machine file system can also be bind mounted into the container so that dxc can be used like a regular command line application on any machine:
$ docker run --rm -v $(pwd):$(pwd) -w $(pwd) gwihlidal/dxc -T <target> -E <entry-point-name> <input-hlsl-file>
Example output (SPIR-V):
% docker run --rm -v $(pwd):$(pwd) -w $(pwd) gwihlidal/dxc -spirv -T ps_6_0 -E main simple.hlsl
; SPIR-V
; Version: 1.0
; Generator: Google spiregg; 0
; Bound: 12
; Schema: 0
OpCapability Shader
OpMemoryModel Logical GLSL450
OpEntryPoint Fragment %main "main" %out_var_SV_TARGET0
OpExecutionMode %main OriginUpperLeft
OpSource HLSL 600
OpName %main "main"
OpName %out_var_SV_TARGET0 "out.var.SV_TARGET0"
OpDecorate %out_var_SV_TARGET0 Location 0
%void = OpTypeVoid
%4 = OpTypeFunction %void
%float = OpTypeFloat 32
%v4float = OpTypeVector %float 4
%_ptr_Output_v4float = OpTypePointer Output %v4float
%float_0 = OpConstant %float 0
%float_1 = OpConstant %float 1
%10 = OpConstantComposite %v4float %float_0 %float_1 %float_0 %float_1
%out_var_SV_TARGET0 = OpVariable %_ptr_Output_v4float Output
%main = OpFunction %void None %4
%11 = OpLabel
OpStore %out_var_SV_TARGET0 %10
OpReturn
OpFunctionEnd
Example output (DXIL):
% docker run --rm -v $(pwd):$(pwd) -w $(pwd) gwihlidal/dxc -T ps_6_0 -E main simple.hlsl
;
; Input signature:
;
; Name Index Mask Register SysValue Format Used
; -------------------- ----- ------ -------- -------- ------- ------
; no parameters
;
; Output signature:
;
; Name Index Mask Register SysValue Format Used
; -------------------- ----- ------ -------- -------- ------- ------
; SV_Target 0 xyzw 0 TARGET float xyzw
;
;
; Pipeline Runtime Information:
;
; Pixel Shader
; DepthOutput=0
; SampleFrequency=0
;
;
; Output signature:
;
; Name Index InterpMode DynIdx
; -------------------- ----- ---------------------- ------
; SV_Target 0
;
; Buffer Definitions:
;
;
; Resource Bindings:
;
; Name Type Format Dim ID HLSL Bind Count
; ------------------------------ ---------- ------- ----------- ------- -------------- ------
;
;
; ViewId state:
;
; Number of inputs: 0, outputs: 4
; Outputs dependent on ViewId: { }
; Inputs contributing to computation of Outputs:
;
target datalayout = "e-m:e-p:32:32-i1:32-i8:32-i16:32-i32:32-i64:64-f16:32-f32:32-f:64:64-n8:16:32:64"
target triple = "dxil-ms-dx"
define void @main() {
call void @dx.op.storeOutput.f32(i32 5, i32 0, i32 0, i8 0, float 0.000000e+00) ; StoreOutput(outputSigId,rowIndex,colIndex,value)
call void @dx.op.storeOutput.f32(i32 5, i32 0, i32 0, i8 1, float 1.000000e+00) ; StoreOutput(outputSigId,rowIndex,colIndex,value)
call void @dx.op.storeOutput.f32(i32 5, i32 0, i32 0, i8 2, float 0.000000e+00) ; StoreOutput(outputSigId,rowIndex,colIndex,value)
call void @dx.op.storeOutput.f32(i32 5, i32 0, i32 0, i8 3, float 1.000000e+00) ; StoreOutput(outputSigId,rowIndex,colIndex,value)
ret void
}
; Function Attrs: nounwind
declare void @dx.op.storeOutput.f32(i32, i32, i32, i8, float) #0
attributes #0 = { nounwind }
!llvm.ident = !{!0}
!dx.version = !{!1}
!dx.valver = !{!2}
!dx.shaderModel = !{!3}
!dx.typeAnnotations = !{!4}
!dx.viewIdState = !{!8}
!dx.entryPoints = !{!9}
!0 = !{!"clang version 3.7 (tags/RELEASE_370/final)"}
!1 = !{i32 1, i32 0}
!2 = !{i32 1, i32 3}
!3 = !{!"ps", i32 6, i32 0}
!4 = !{i32 1, void ()* @main, !5}
!5 = !{!6}
!6 = !{i32 0, !7, !7}
!7 = !{}
!8 = !{[2 x i32] [i32 0, i32 4]}
!9 = !{void ()* @main, !"main", !10, null, null}
!10 = !{null, !11, null}
!11 = !{!12}
!12 = !{i32 0, !"SV_Target", i8 9, i8 16, !13, i8 0, i32 1, i8 4, i32 0, i8 0, null}
!13 = !{i32 0}
warning: DXIL.dll not found. Resulting DXIL will not be signed for use in release environments.
NOTE: The DXIL produced by DXC must be validated by the signed dxil.dll library, which can only (currently) occur with a sneaky LoadLibrary call during compilation. As this call is only possible on Windows, any DXIL produced by a Linux version will only allow for usage with Windows configured in development / testing mode. Please see this blog post for information on how to solve this!
