The news of Warp3D for PiStorm is very exciting for me, as I have spent a lot of effort on creating my own RTG 3D engine (and hopefully releasing a game soon, but not holding my breath for that yet).

Few bullet points:
- written from scratch in ASM for full performance (I'm a professional C++ graphics (DX 11/OpenGL/Mesa/XNA/CUDA) engineer, not currently working in industry to work on my own games)
- vasm (vasm -m68040 -Fhunkexe) / Notepad++ / WinUAE
- testing against V4SA (that's the only Amiga HW I have at home)
- target ops: 68040 (no FPU, everything is FixedPoint math)
- RTG (using only CGX's function to select the target video mode, the rest is SW rasterizer)
- support for all resolutions from 320x200 up to 1920x1080 (via requester)
- have some texturing functionality, but for now focusing on flatshading

Q1: I want to be a Beta Tester, but looks like the IndieGogo campaign only has the $10 EUR pledge (I suspect there were more pledges before it met the target?)
Did I just miss the window for that on day two of the campaign?

Q2: Is this the API doc (initial googling) ? https://wiki.amigaos.net/amiga/autodocs/warp3d.doc.txt

Q3: I don't see any 3D Transform&Lighting functionality. That'd imply I still keep my 3D Projection/Clipping from my SW rasterizer then. Does it mean that Warp3D really only does the rasterizer part ?

Q4: Is Warp3D/W3D_DrawElements the core rendering method for Indexed Batch Rendering ? How does it handle off-screen coordinates ?

Q5: Does it handle Z-Buffer automatically or do I have to fill it in a manual pre-pass ?

Q6: Do I only get the following pipeline stages from Warp3D ? FrameBuffer Clear, BackFace Culling, Z-Buffer, Batch Rasterizing of triangles

Here's some screenshots:



img tag isn't working, posting the links
https://imgur.com/3KQMD0l
https://imgur.com/q53TqS9
https://imgur.com/OY4ELme

Last edited by Heimdall on 21-Jan-2025 at 09:47 AM.

@Heimdall

Warp3D (legacy) is purely a rasterizer. You get hardware rendering of primitives but that's it. All the transformation, clipping, lighting etc is your responsibility.

Coordinates are floating point types, screen space, Z is expected to be normalised in the range 0.0-1.0 (near - far). Your fixed point data will need to be converted here.

You can make use of some hardware scissor clipping and back face culling at the driver level. However any culling you do before getting there is obviously more efficient.

There is a w coordinate, but this *is not* the 4th component of your GL style homogeneous coordinate system (i.e. coordinate v normal) since the system has no 3D transformation. It's a reciprocal value used for perspective correction of texture coordinates, independent of the actual depth value. This gives you some flexibility in how you manage the Z buffer values which is good, because a traditional reciprocal z value would cluster around the far plane and lose precision for depth testing. You can thus have a rather more linear depth function (less z fighting at a distance) while retaining the expected perspective corrections to texture coordinates.

You manage the Z buffer. You can fill it with a known depth that will typically be implemented by a hardware fill.

Querying, reading and plotting the Z buffer is a bit hit and miss and can be very slow since it involves double precision floating point normalised values and the ZBuffer may not be linear. It's common for chips to have various tiling / hierarchical structures. Avoid if possible.

DrawElements is used for indexed primitives. DrawArray expects linear data.

Last edited by Karlos on 21-Jan-2025 at 12:06 PM.
Last edited by Karlos on 21-Jan-2025 at 11:55 AM.
Last edited by Karlos on 21-Jan-2025 at 11:54 AM.
Last edited by Karlos on 21-Jan-2025 at 11:44 AM.

_________________
Doing stupid things for fun...

One more thing. Your images look like you are also using some line rendering too. This is a very hit and miss feature of the old drivers. Some implement it, some don't. Of those that do, it may be implemented as a single pixel wide triangle strip.

_________________
Doing stupid things for fun...

@Karlos

Thanks for your insights. This is very helpful.

Quote:

Karlos wrote: @Heimdall Warp3D (legacy) is purely a rasterizer. You get hardware rendering of primitives but that's it. All the transformation, clipping, lighting etc is your responsibility.

I was hoping I actually looked up a wrong library (I'm confused about all the variants - Wazp, Warp, Nova, etc.). Alas, I was correct - and there's no HW T&L - oh, well :(



Quote:

Coordinates are floating point types, screen space, Z is expected to be normalised in the range 0.0-1.0 (near - far). Your fixed point data will need to be converted here.

That begs the question - because I handle screen-space clipping during actual rasterization call (I have 2 methods - one without and one with clipping).

Can Warp handle off-screen coordinates and clip accordingly ? Or do all triangles HAVE to be in valid screenspace coords ?


Quote:

You can make use of some hardware scissor clipping and back face culling at the driver level. However any culling you do before getting there is obviously more efficient.

Backface culling would be very beneficial. Not for this game, but for future ones.


Quote:

You manage the Z buffer. You can fill it with a known depth that will typically be implemented by a hardware fill. Querying, reading and plotting the Z buffer is a bit hit and miss and can be very slow since it involves double precision floating point normalised values and the ZBuffer may not be linear. It's common for chips to have various tiling / hierarchical structures. Avoid if possible. DrawElements is used for indexed primitives. DrawArray expects linear data.

Right now I am doing back to front sorting by Z-coord, so I don't have to handle Z-Buffer at all.


Double floats ? Ouch !
I went to great lengths to develop and debug a fully Integer pipeline. I am not throwing out all that work so that I would now backconvert it to floating point just to appease the API...


Doesn't look like Wazp would bring anything other than HW texturing to the table (which I don't need right now anyway - as I could easily create dozen great-looking games with this 3D engine).
Not to mention I have multiple codepaths for axis-aligned texturing (just not the generic 6 DOF texturing yet)...

@Heimdall

Quote:

I was hoping I actually looked up a wrong library (I'm confused about all the variants - Wazp, Warp, Nova, etc.). Alas, I was correct - and there's no HW T&L - oh, well :(

So I can't say for certain about Nova - it was designed for more modern hardware and I am sure it supports shaders and other feaures. It's also presently only availabe for OS4 machines with appropriate hardware (Radeon HD) as far as I know.

Wazp3D is an open source replacement for Warp3D that supports software rendering and various other backends. It is API compatible with the original Warp3D up to v5.

There's no reason to abandon your integer pipeline but you would have to be prepared to convert them to floating point when passing it to the API.

Double precisions for Z was likely chosen as there are varying z-buffer formats from device to device, 15 bit, 16-bit, 24 bit and 32 bit are known among the supported hardware so in order to support normalised depth with no loss of precision 64-bit float was chosen. This format only applies to legacy V3 Z coordinate data and to the Z buffer read and write pixel/span operations which are rately used. One thing I started working on but didn't finish was a Z format query (that would return the specific data format) and methods for direct sampling/plotting/rectange copy.

_________________
Doing stupid things for fun...

On the specific topic of the integer versus floating point, floating point multiplication on the 040 is faster than integer - I think. IIRC, it's around 18 cycles for an integer multiply and 5 for a floating point.

_________________
Doing stupid things for fun...

@Karlos

TC's 1st post has PiStorm with 3D in mind. Emu68 self-reports as a 68040, a hyper-fast 68040.

_________________
Amiga 1200 (rev 1D1, KS 3.2, PiStorm32/RPi CM4/Emu68)
Amiga 500 (rev 6A, ECS, KS 3.2, PiStorm/RPi 4B/Emu68)
Ryzen 9 7950X, DDR5-6000 64 GB RAM, GeForce RTX 4080 16 GB

@Hammer

Understood but I assume he also wants to make it viable on real silicon, otherwise why worry about assembler and fixed point arithmetic?

Almost all real 68040 Amiga machines had FPU and moreover, those that can run Warp3D require it.

To interface with W3D, the fixed point output of the software transformation can be converted to floating point and passed along. The 68040 can do this directly. fmove.l the fixed point value to a register, which will now be some power of 2 too big, multiply by a scaling factor (2.0^-precison size) to convert back to the correct floating point representation and push that to the array you're building.

There are other hacks you can do to convey fixed point to floating point without using FPU, but it depends on the range, tolerance of denormalised values etc.

_________________
Doing stupid things for fun...

@Karlos

Quote:

Karlos wrote: @Heimdall Quote: I was hoping I actually looked up a wrong library (I'm confused about all the variants - Wazp, Warp, Nova, etc.). Alas, I was correct - and there's no HW T&L - oh, well :( So I can't say for certain about Nova - it was designed for more modern hardware and I am sure it supports shaders and other feaures. It's also presently only availabe for OS4 machines with appropriate hardware (Radeon HD) as far as I know. Wazp3D is an open source replacement for Warp3D that supports software rendering and various other backends. It is API compatible with the original Warp3D up to v5. There's no reason to abandon your integer pipeline but you would have to be prepared to convert them to floating point when passing it to the API. Double precisions for Z was likely chosen as there are varying z-buffer formats from device to device, 15 bit, 16-bit, 24 bit and 32 bit are known among the supported hardware so in order to support normalised depth with no loss of precision 64-bit float was chosen. This format only applies to legacy V3 Z coordinate data and to the Z buffer read and write pixel/span operations which are rately used. One thing I started working on but didn't finish was a Z format query (that would return the specific data format) and methods for direct sampling/plotting/rectange copy.

I'm familiar with the Z-Buffer precision/range distribution issues as I have been working with 3D since OpenGL, then DX 8, DX9, DX11. So, I understand their decision to go with double precision. I just don't like it when I already have an integer/FxP pipeline as it took so much bloody work to debug it.

I'm surprised API allows me to read the actual Z-Buffer values. That opens up a whole range of new effects that are possible (especially shadows).

When talking about HW - I mean the rPi4. Is there even any other accelerator that supports Warp3D that isn't on OS4/PPC ? I genuinely don't know.

@Karlos

Quote:

Karlos wrote: On the specific topic of the integer versus floating point, floating point multiplication on the 040 is faster than integer - I think. IIRC, it's around 18 cycles for an integer multiply and 5 for a floating point.

My whole reason for trying to avoid FP on Amiga is that I recently became aware of all the FP issues - there's many library versions, patches, patches of patches, some 060s (LC, I think) don't even have FP unit, etc.

So, having an Integer pipeline (FxP really) helps avoiding all that mess.

I did, on V4, implement a second version of my scanline traversal that was using FP registers alongside Integer registers and the speed-up on V4 was quite significant, as the entire loop (computing both endpoints) worked just with registers.

And that was without AMMX, just basic FP code.

@Hammer

Quote:

Hammer wrote: @Karlos TC's 1st post has PiStorm with 3D in mind. Emu68 self-reports as a 68040, a hyper-fast 68040.

Correct, but one of the reasons for writing whole engine in ASM was so that eventually I will get it working on 68040-060.

Right now, it's using RTG (CGX, really). But if I am confident that if I spent about 2 weeks, I could write a pretty fast C2P routine (not as fast as Kalms's but good enough for me).

Once I have C2P, I can get it running on 060, 040, and probably 030 (though I am using long divide now which wouldn't probably compile on 030). Technically, all AGA should work (though I don't foresee butchering my 3D meshes down to the level of A1200).


First, however, I need to bloody finish the thing, release it (even if it's playable just on Vampire and rPi due to being 24-bit). Then I can worry about C2P and lower configs.

@Heimdall

Don't depend on the software ZBuffer reading for anything that's not a still life project, it's extraordinarily slow. You can only safely call it while in the locked state, then there's the whole internal conversion to/from 64-bit float. Finally, some cards require a bunch of address permutations to calculate the actual position in the buffer for a given coordinate because the buffer is not a simple rectangle.

_________________
Doing stupid things for fun...

@Heimdall

Quote:

Can Warp handle off-screen coordinates and clip accordingly ? Or do all triangles HAVE to be in valid screenspace coords ?

So, in my experience, Warp3D is generally OK with rendering primitives that are partially off screen, though strictly what happens can vary from driver to driver.

Generally, if no user defined scissor is set, the scissor is set to the limits of the draw area. However, the effect of having vertices far outside the scissor can vary. For some drivers, it may make no difference and performance is limited only by the area filled. For other drivers, performance mat suffer depending on how things like vertex colours are interpolated and whether or not there's still "outside scissor" calculation for them, but no plotting until within the boundary. For some other drives there may be additional checks, e.g. guardband culling that just skips the primitive all together if a vertex is sufficiently outside the render area.

Last edited by Karlos on 22-Jan-2025 at 11:26 AM.

_________________
Doing stupid things for fun...

@Karlos

Quote:

Karlos wrote: @Heimdall Quote: Can Warp handle off-screen coordinates and clip accordingly ? Or do all triangles HAVE to be in valid screenspace coords ? So, in my experience, Warp3D is generally OK with rendering primitives that are partially off screen, though strictly what happens can vary from driver to driver. Generally, if no user defined scissor is set, the scissor is set to the limits of the draw area. However, the effect of having vertices far outside the scissor can vary. For some drivers, it may make no difference and performance is limited only by the area filled. For other drivers, performance mat suffer depending on how things like vertex colours are interpolated and whether or not there's still "outside scissor" calculation for them, but no plotting until within the boundary. For some other drives there may be additional checks, e.g. guardband culling that just skips the primitive all together if a vertex is sufficiently outside the render area.

Well, partially. Due to the way Math works, the vertices close to the front plane get ridiculous values after transformation (e.g. -8192,+8192) or more.

And it's exactly those vertices that are part of the clipped polygons, and their scanlines still need to be processed.

My first version of clipping code was computing all endpoints in that case, but my recent upgraded routine skips all out-of-screen scanline and only processes scanlines that are visible.

Which begs the question - is the Wapr3D implementation doing the first or second approach ?

Because while it's trivial to avoid that scenario for polygons crossing bottom screen edge (via early out), it's not the case for top screen edge - and you often get polygons that have 3,000 scanlines above the top screen edge (due to the front-plane math).
Then you either introduce per-scanline condition (very, very bad) or double the amount of render paths :)

@Heimdall

Quote:

Which begs the question - is the Wapr3D implementation doing the first or second approach ?

As I said, it's somewhat driver specific what the behaviour is for grossly out of range vertices with scissoring. Scissoring may not even be implemented at a hardware level in some driver, but AFAIR, all the chips that had drivers supported it in hardware.

How each chip implemented it, well, as I said, that varies. Some are doubtless able to correctly compute the value of any interpolant for a given screen coordinate that's "inside" the triangle. Others may do it only for the ends of a span that they are rendering and rely on per pixel addition of precalculated deltas to a total.

As a general rule of thumb I'd advise relying on scissoring as your only method of dealing with oversized polygons because you aren't guaranteed a consistent performance behaviour across multiple drivers. If you clip and interpolate your own polygons and maybe rely on scissoring to deal with vertices that are only a small amount outside the render area, that's a more reliable and performance predictable method.

_________________
Doing stupid things for fun...