Implement texture support in the compiler. Exposing all of this is a lot, and needs a decent amount of UX thought, so posting this issue so people can chime in with ideas in addition to tracking the implementation.
Here's an overview of how textures work in the SPIR-V spec:
- OpTypeImage: an opaque data blob that contains some form of image. The type contains:
- the underlying sampled type
- if it's a depth image (as in depth testing), can be yes/no/unknown
- whether it's multisampled
- whether it can be used without a sampler (I'm guessing capabilities are required here?), values are "unknown, with sampler, without sampler"
- image format, can be unknown
- if it's read, write, or readwrite.
- OpTypeSampler/OpConstantSampler: does not contain image data, merely information on how to access an image. Data is in the OpConstantSampler, not the type (the type has no parameters at all).
- if it's normalized
- sampler filter mode: Nearest/Linear
- OpTypeSampledImage/OpSampledImage: combines an image and OpConstantSampler into one meta-object.
- The result of OpSampledImage has absolutely wild restrictions: it can only be referenced in the same BB that it's declared in, can't be used in OpSelect/OpPhi, if the image is in a composite then all indices getting the image must be dynamically-uniform, etc.
Then, lots of image read/write instructions (not sure what Dref is used for, "depth-comparison reference value"):
and edit, to be clear, nearly all of these (except for ImageRead/ImageWrite) take the combined TypeSampledImage. There is no avoiding using the combined version. The combined type has insane restrictions that pretty much guarantees that it will not be exposed to users as a struct (e.g. must be used in the same basic block it is declared in), but we cannot get around using it, it is how texture reads work in SPIR-V.
- OpImageFetch - directly on OpTypeImage with sampled=1, not OpTypeSampledImage, integer indices
- OpImageRead - directly on OpTypeImage with sampled=0 or 2, integer indices
- OpImageWrite - directly on OpTypeImage with sampled=0 or 2, integer indices
- OpImageGather - fetches one color from four pixels I think? weirdly, takes OpTypeSampledImage, despite name. float indices
- OpImageDrefGather - fetches depth from four pixels I think? weirdly, takes OpTypeSampledImage, despite name. float indices
- OpImageSampleDrefExplicitLod - idk, "Sample an image doing depth-comparison using an explicit level of detail."
- OpImageSampleDrefImplicitLod - idk, like explicit, but implicit derivative for LOD
- OpImageSampleFootprintNV - spec literally just says "TBD"
- OpImageSampleExplicitLod - LOD sample, float indices, except in kernel mode which allows integer
- OpImageSampleImplicitLod - like explicit, but implicit derivative for LOD
- OpImageSampleProjDrefExplicitLod - idk, "Sample an image with a project coordinate, doing depth-comparison, using an explicit level of detail."
- OpImageSampleProjDrefImplicitLod - idk, like explicit, but implicit derivative for LOD
- OpImageSampleProjExplicitLod - Sample image but divide coordinate by w
- OpImageSampleProjImplicitLod - like explicit, but implicit derivative for LOD
- OpImageSparseDrefGather - assuming all these sparse ones are equivalent to their non-sparse names. Needs SparseResidency capability.
And some misc instructions:
- OpImageTexelPointer - like OpAccessChain, but for pixels. "Use of such a pointer is limited to atomic operations."
- OpImageQueryFormat - kernel capability (??)
- OpImageQueryLevels - kernel, ImageQuery capability
- OpImageQueryLod - ImageQuery capability (??)
- OpImageQueryOrder - kernel capability (??)
- OpImageQuerySamples - kernel, ImageQuery capability
- OpImageQuerySize - kernel, ImageQuery capability
- OpImageQuerySizeLod - kernel, ImageQuery capability
- OpImageSparseTexelsResident - "Result is false if any of the texels were in uncommitted texture memory, and true otherwise."
but like, the ImageQuery capability implicitly declares the Shader capability, so the queries that require both the ImageQuery and the Kernel capability are, uuuh, a program needs to be both a Shader and a Kernel? what.
So, if we were to support sampled_type that is f64, or any integer type, we would need vector types for f64 and integers. Glam currently doesn't support those. So, do we want to put in that work eventually? Probably not now, probably not soon, but should it be on our roadmap to do eventually?
There's a already an issue on glam for it, so we could just track that for now. https://github.com/bitshifter/glam-rs/issues/70