Click Here
home features news forums classifieds faqs links search
6071 members 
Amiga Q&A /  Free for All /  Emulation /  Gaming / (Latest Posts)
Login

Nickname

Password

Lost Password?

Don't have an account yet?
Register now!

Support Amigaworld.net
Your support is needed and is appreciated as Amigaworld.net is primarily dependent upon the support of its users.
Donate

Menu
Main sections
» Home
» Features
» News
» Forums
» Classifieds
» Links
» Downloads
Extras
» OS4 Zone
» IRC Network
» AmigaWorld Radio
» Newsfeed
» Top Members
» Amiga Dealers
Information
» About Us
» FAQs
» Advertise
» Polls
» Terms of Service
» Search

IRC Channel
Server: irc.amigaworld.net
Ports: 1024,5555, 6665-6669
SSL port: 6697
Channel: #Amigaworld
Channel Policy and Guidelines

Who's Online
23 crawler(s) on-line.
 122 guest(s) on-line.
 0 member(s) on-line.



You are an anonymous user.
Register Now!
 OneTimer1:  8 mins ago
 amigakit:  24 mins ago
 Troels:  1 hr 9 mins ago
 Gunnar:  1 hr 24 mins ago
 Rob:  1 hr 30 mins ago
 zipper:  1 hr 53 mins ago
 NutsAboutAmiga:  2 hrs 59 mins ago
 kolla:  3 hrs 11 mins ago
 Comi:  3 hrs 40 mins ago
 vox:  4 hrs 26 mins ago

/  Forum Index
   /  General Technology (No Console Threads)
      /  Virtual machines, instruction sets, that sort of thing...
Register To Post

Goto page ( 1 | 2 | 3 | 4 | 5 | 6 Next Page )
PosterThread
Karlos 
Virtual machines, instruction sets, that sort of thing...
Posted on 9-May-2022 19:20:25
#1 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

Just in case anyone else likes shooting the breeze about this sort of thing.

Last edited by Karlos on 10-May-2022 at 10:04 AM.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 4:59:38
#2 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@Karlos: continuing here is better.

Quote:

Karlos wrote:
@cdimauro

Quote:
OK, I see. And I think you have no plans to further expand it: looks like an old project now.


One of the things about being "on the spectrum" is that you can't let go of ideas easily. So as unmoving as it is, the day will surely come when I decide I want to work on it. It wasn't a terrible design and I think it has potential as a generalised scripting target.

I don't find it terrible; rather, the contrary. I think that it's a good design, which can be improved to become a more general architecture.

As additional suggestions, you might consider to remove the more complex FP instructions, which you don't find on modern ISA: they could be emulated in software, when needed.

And you can support to FP16 and FP128 data types.

I don't know how is your vector extension, but you if it's fixed-length (SIMD) you might consider to make it length-agnostic (mine is "hybrid": it's also fixed-length. But because I had enough encoding space for it; otherwise it's expensive to add bits on an already restricted opcode space).
Quote:
Quote:
Understood, thanks. It's still difficult to get an idea of how good can be the code density, but this looks VM highly specialized, and maybe a comparison with some ISA doesn't make sense.

Is the the opcode granularity in bytes or words (16-bits)?

StackGVM was a pure bytecode machine. Instructions vary between 1-6 bytes in length, most arithmetic and logic instructions are three operand.

My original idea was that it would be a register based machine but after a first design pass, I became dissatisfied with it given the need to support 3 component vectors which would tend to be accessed indirectly too. The register machine did support some local stack operations too by which point I had an epiphany that a much simpler "stack frame" model was possible. In the end, it was.

So, it's an hybrid machine: stack and register based.

I did the same with my CPython VM (WPython), because I didn't like the purely stack-based VM which Python used.
Quote:
The code density is not as high as other designs, however, the types of operations implemented in it are largely short functions that benefit from having the shortest sequence of discrete operations. With effectively up to 128 function-scope variable slots to play with (that includes your own parameters, return, locals and the return/parameter space of any child calls) and three operand instructions, you can get a lot done in just a few steps. And as you say, it's highly specialised for a particular use case.

128 operands is enough for general-purpose use.
Quote:
Quote:
Well, I also saw your YT channel, and this is even more evident.


Feel free to like and subscribe I make no apologies for torturing PHP and terminals...

Done! Albeit I don't like PHP (I contributed a lot to the now dead "Your Language Sucks!" PHP section ).

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 5:40:49
#3 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

By terrible I was referring to how I'd used it. Today nobody in their right mind would argue for data files that have embedded executable sections. The idea came from datatypes, sort of. I had high level types for different types of media in my framework and loaders that could use datatypes and such for common formats. But as I wrote more and more of my own custom data formats I started using it as a means of not having to care at all how a given platform prefers to represent the serialised data, just embed a decoder so that if the application sees this is an image type but doesn't have a native decoder to load it, it can fall back to invoking the VM on the embedded decoder.

With the benefit of hindsight, I'd probably create a separate FPU register set too rather than the completely unified set that exists now. As a load/store this would reduce register pressure.

The more complex maths operations, like vector operations exist in an extension set, so they don't sit in the hot path anyway. I added support for basic complex numbers, 2 and 3D vectors, transformation and such when started playing with it for use in a game engine. That experiment ultimately led to the dedicated stack gvm, because I love reinventing wheels.

Vector wise, the opcodes essentially do block operations on varying length arrays in memory. Stream processing is probably a better description really and it's definitely not as flexible as a proper vector unit. This comes from the aforementioned data processing workloads which tended to do simple operations in bulk. As such they are much faster than doing such operations imperatively within the VM.

As with the rest of the machine, there are three basic implementation levels (x64 would expected to implement all 3), but a flavour can be found here: https://github.com/0xABADCAFE/exvm/blob/master/source/include/opcodes/vector/enum.hpp.bak

The implementation typically use register operands. Two source pointers, a destination pointer and a count.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 5:46:02
#4 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

As for Stack GVM, it's not register based per se, it's just that operands are byte offsets from the stack pointer. Every invocation of a function has it's own frame and pair of indirection pointers all of which are "on the stack". But this is semantics. From the purpose of how an instruction works, it's as if there are up to 128 local registers (plus up to 128 more from the caller scope) it can access.

If you are interested, the instruction opcode definition is here: https://github.com/0xABADCAFE/random-junk/blob/master/stackgvm/source/include/gvm_opcode_instruction.hpp

You will note the multiple variations based on the use of locals (_L) and/or indirections (_I0 / _i1).

Most of the time, these operations are used and are typically 4 bytes (opcode plus operand bytes) for 3 operand instructions.

Conditionals are handled in a slightly odd fashion. To save opcode space for the main instructions, I made the conditional test itself a parameter to a singe conditional test and branch opcode (_BOC): https://github.com/0xABADCAFE/random-junk/blob/master/stackgvm/source/include/gvm_opcode_condition.hpp


I appreciate this doesn't really help understand the opcode layout. So a simple example would be add.i local, local2, local3, which would perform an integer addition as local3 = local1 + local2

The opcode bytes are:

_ADD_LLL, local1_stack_pos, local2_stack_pos, local3_stack_pos

The indirection pointers follow a convention that dictates which is the destination when both are in use. Loading a structure pointer into these registers allows operations to modify elements in the structure directly. So when you pick up an ammo pack, or take damage, we don't have to copy data to and from the stack to manipulate it.

Eventually I decided to add a more complex indirection capability as an extension set that can be used when wanting to avoid constantly switching indirection pointers which turned out to be a more common occurrence than I would like. So every opcode enumeration that has an _X variant can use any of the additional addressing modes here: https://github.com/0xABADCAFE/random-junk/blob/master/stackgvm/source/include/gvm_opcode_address.hpp

So yeah, pretty specialised but ideal for the target use case.

I decided that I'd take all the stuff I'd thought about 20 odd years ago and begin to factor it out into separate chunks to implement them in isolation, just to see how doable they were. And if I do enough of them, maybe I can complete it. I notice that retro FPS are currently vogue and this being a 2.5D game engine with "quake style" builtin game logic interpreter would be pretty retro.

If it is of any interest, there are some braindumps
here and here

I did work on some of the level compiling prototype here: https://github.com/0xABADCAFE/os25d-level-compiler

Last edited by Karlos on 10-May-2022 at 09:58 AM.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 6:16:06
#5 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

Quote:
Albeit I don't like PHP (I contributed a lot to the now dead "Your Language Sucks!" PHP section)


Lol. Yes it's definitely a love/hate thing for me. I love the rapidity with which things can be prototyped. I hate the inconsistency and traps caused by implicit type juggling. The language is definitely improving though. For me the syntactical familiarity is a plus. Performance is acceptable for the sorts of workloads it's normally used for but the JIT has been a big disappointment. I did some analysis on that here: https://github.com/0xABADCAFE/php8-playground

In short, idiomatic PHP doesn't benefit much from it. Which is probably why the demos to show it off have been based on highly imperative looping code (e.g fractals) with almost no function calls. I did manage to torture it into producing code about as fast as equivalent C, but it would fail every semblance of good practise in the language

As an aside, I have to say, the biggest challenge to writing demos in PHP wasn't PHP itself, it was the decision to use the terminal as the display. It's a trifle to slow to just pump out full RGB ANSI every frame, unless speed isn't important. Consequently there's quite a lot of delta analysis and RLE encoding of space and other characters. When I decided to use some UTF8 blockmode characters to effectively double the resolution vertically, it got even trickier, lol.

Last edited by Karlos on 10-May-2022 at 10:08 AM.
Last edited by Karlos on 10-May-2022 at 10:03 AM.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
kolla 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 9:58:15
#6 ]
Elite Member
Joined: 20-Aug-2003
Posts: 2859
From: Trondheim, Norway

Life has taught me that I am better avoiding programming languages/interpreters which names are starting with the letter “p”.

_________________
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 10:05:54
#7 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

I have to apologise for all the weird typos. It's when I post from my phone, lol.

@kolla

So you're a Ruby man, then?

Hey, at least I didn't say Javascript.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 11:24:24
#8 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

Quote:

cdimauro wrote:

I did the same with my CPython VM (WPython), because I didn't like the purely stack-based VM which Python used.


Tell me more.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
NutsAboutAmiga 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 16:16:13
#9 ]
Elite Member
Joined: 9-Jun-2004
Posts: 12795
From: Norway

@Karlos

Love Ruby, but find it hard to install, not sure why you are using web languages for this, I’m scratching my head. Don’t laud too loudly about JavaScript, node.js blows PHP out of the water.

if write bad C code, even C programs can perform badly.
compilers are smart, but does not replace a good developer.

Last edited by NutsAboutAmiga on 10-May-2022 at 04:23 PM.
Last edited by NutsAboutAmiga on 10-May-2022 at 04:20 PM.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 10-May-2022 16:42:57
#10 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@NutsAboutAmiga

Quote:
not sure why you are using web languages for this


I'm not sure what you are referring to here. Can you be more specific? I wouldn't touch Node.js with a bargepole; even the author has distanced himself from it.

I've used PHP to implement the commandline MC64K's assembler for a number of reasons:

1) It's extremely easy to get to a working prototype from an idea and then to continuously iterate and refine.
2) It has extremely powerful string manipulation, maps, etc.
3) It supports procedural, imperative, functional and object oriented programming styles well and with a syntax that's reasonably comfortable to use.
4) It's more than fast enough for the task at hand.

Naturally, the runtime for MC64K is written in C++ as it's the right tool for that job.

Unless you are referring to PDE. In which case I'm using it because commandline PHP is underrepresented in the demo scene

Last edited by Karlos on 10-May-2022 at 04:44 PM.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 5:05:51
#11 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@Karlos

Quote:

Karlos wrote:
@cdimauro

By terrible I was referring to how I'd used it. Today nobody in their right mind would argue for data files that have embedded executable sections. The idea came from datatypes, sort of. I had high level types for different types of media in my framework and loaders that could use datatypes and such for common formats. But as I wrote more and more of my own custom data formats I started using it as a means of not having to care at all how a given platform prefers to represent the serialised data, just embed a decoder so that if the application sees this is an image type but doesn't have a native decoder to load it, it can fall back to invoking the VM on the embedded decoder.

Got it. I don't like either to embed code on data that way.
Quote:
With the benefit of hindsight, I'd probably create a separate FPU register set too rather than the completely unified set that exists now. As a load/store this would reduce register pressure.

Absolutely. You've 16 registers, which are a good number, but L/S architectures need spare / temporary registers compared to CISCs, so there's much more register spilling. And if you have a unified register set for everything then this exacerbated.
Quote:
The more complex maths operations, like vector operations exist in an extension set, so they don't sit in the hot path anyway. I added support for basic complex numbers, 2 and 3D vectors, transformation and such when started playing with it for use in a game engine. That experiment ultimately led to the dedicated stack gvm, because I love reinventing wheels.

Vector wise, the opcodes essentially do block operations on varying length arrays in memory. Stream processing is probably a better description really and it's definitely not as flexible as a proper vector unit. This comes from the aforementioned data processing workloads which tended to do simple operations in bulk. As such they are much faster than doing such operations imperatively within the VM.

As with the rest of the machine, there are three basic implementation levels (x64 would expected to implement all 3), but a flavour can be found here: https://github.com/0xABADCAFE/exvm/blob/master/source/include/opcodes/vector/enum.hpp.bak

The implementation typically use register operands. Two source pointers, a destination pointer and a count.

LOL That's exactly what I've done on my ISA, and that I call "cheap vector/block extension".

I did it because I wanted to extend the REP prefix/mechanism of x86/x64 to a more general way.

In fact, I've introduced dedicated opcodes (not prefixes: I've no prefixes on my ISA. Only bigger opcodes) that allow to "repeat" any instruction, with a flexible method to define the sources (up to 3 with some SIMD/vector instructions) and the destination.

This allowed me to reduce the famous daxpy function (a classical one that it's used on ISA studies) to just one instruction:
rep sfmadd231 [r7]+,d0,[r6]+
However I was just lucky, with this function which has only one operation to be iterated.

More information on the following paper:
SIMD Instructions Considered Harmful: https://www.sigarch.org/simd-instructions-considered-harmful/
and article:
Three fundamental flaws of SIMD ISA: https://www.bitsnbites.eu/three-fundamental-flaws-of-simd/

BTW, those studies are biased: AVX-2 & AVX-512 could do MUCH better. IMO they are comparing the code generated by a compiler (for x86/x64) to the one written by assembly coders (for all other ISAs). But I haven't investigated further.

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 5:08:37
#12 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@Karlos

Quote:

Karlos wrote:
As for Stack GVM, it's not register based per se, it's just that operands are byte offsets from the stack pointer. Every invocation of a function has it's own frame and pair of indirection pointers all of which are "on the stack". But this is semantics. From the purpose of how an instruction works, it's as if there are up to 128 local registers (plus up to 128 more from the caller scope) it can access.

If you are interested, the instruction opcode definition is here: https://github.com/0xABADCAFE/random-junk/blob/master/stackgvm/source/include/gvm_opcode_instruction.hpp

You will note the multiple variations based on the use of locals (_L) and/or indirections (_I0 / _i1).

Most of the time, these operations are used and are typically 4 bytes (opcode plus operand bytes) for 3 operand instructions.

Conditionals are handled in a slightly odd fashion. To save opcode space for the main instructions, I made the conditional test itself a parameter to a singe conditional test and branch opcode (_BOC): https://github.com/0xABADCAFE/random-junk/blob/master/stackgvm/source/include/gvm_opcode_condition.hpp


I appreciate this doesn't really help understand the opcode layout. So a simple example would be add.i local, local2, local3, which would perform an integer addition as local3 = local1 + local2

The opcode bytes are:

_ADD_LLL, local1_stack_pos, local2_stack_pos, local3_stack_pos

The indirection pointers follow a convention that dictates which is the destination when both are in use. Loading a structure pointer into these registers allows operations to modify elements in the structure directly. So when you pick up an ammo pack, or take damage, we don't have to copy data to and from the stack to manipulate it.

Eventually I decided to add a more complex indirection capability as an extension set that can be used when wanting to avoid constantly switching indirection pointers which turned out to be a more common occurrence than I would like. So every opcode enumeration that has an _X variant can use any of the additional addressing modes here: https://github.com/0xABADCAFE/random-junk/blob/master/stackgvm/source/include/gvm_opcode_address.hpp

So yeah, pretty specialised but ideal for the target use case.

I decided that I'd take all the stuff I'd thought about 20 odd years ago and begin to factor it out into separate chunks to implement them in isolation, just to see how doable they were. And if I do enough of them, maybe I can complete it. I notice that retro FPS are currently vogue and this being a 2.5D game engine with "quake style" builtin game logic interpreter would be pretty retro.

If it is of any interest, there are some braindumps
here and here

I did work on some of the level compiling prototype here: https://github.com/0xABADCAFE/os25d-level-compiler

It took some time for checking all links, but I've no knowledge on that game code.

Regarding strictly the stackVM, it looks perfectly suited for the specific needs. Which is obvious.

The code density should also be good, because you packed "more useful work" on the single instructions.

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 5:13:12
#13 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@Karlos

Quote:

Karlos wrote:
@cdimauro

Quote:
Albeit I don't like PHP (I contributed a lot to the now dead "Your Language Sucks!" PHP section)


Lol. Yes it's definitely a love/hate thing for me. I love the rapidity with which things can be prototyped. I hate the inconsistency and traps caused by implicit type juggling. The language is definitely improving though. For me the syntactical familiarity is a plus. Performance is acceptable for the sorts of workloads it's normally used for but the JIT has been a big disappointment. I did some analysis on that here: https://github.com/0xABADCAFE/php8-playground

In short, idiomatic PHP doesn't benefit much from it. Which is probably why the demos to show it off have been based on highly imperative looping code (e.g fractals) with almost no function calls. I did manage to torture it into producing code about as fast as equivalent C, but it would fail every semblance of good practise in the language

Indeed. I've to say that PHP maybe BIG improvements to the performance with the JIT: quite impressive!

However the JIT has the usual problems that happens for dynamic languages. PyPy (which the best Python alternative which is based on a very advanced JIT) does more or less the same, but PHP has advantage (sic!) because you can specify the type of data, whereas Python has only type annotations (which is just metadata attached on data).
Quote:
As an aside, I have to say, the biggest challenge to writing demos in PHP wasn't PHP itself, it was the decision to use the terminal as the display. It's a trifle to slow to just pump out full RGB ANSI every frame, unless speed isn't important. Consequently there's quite a lot of delta analysis and RLE encoding of space and other characters. When I decided to use some UTF8 blockmode characters to effectively double the resolution vertically, it got even trickier, lol.

Well, this is like the ANSI art which we saw with our Amiga, but... made it animated.

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 5:14:34
#14 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@kolla

Quote:

kolla wrote:
Life has taught me that I am better avoiding programming languages/interpreters which names are starting with the letter “p”.

Nah. Python is wonderful: try it!

@Karlos

Quote:

Karlos wrote:
@cdimauro

Quote:

cdimauro wrote:

I did the same with my CPython VM (WPython), because I didn't like the purely stack-based VM which Python used.


Tell me more.

https://code.google.com/archive/p/wpython2/

Please check the download page.

And now my time is over...

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 10:01:18
#15 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

Quote:
It took some time for checking all links, but I've no knowledge on that game code.


Very little of the game engine was implemented when I wrote those documents. However, at the time, Quake had just been ported and I felt that the classic 2.5D era was over before the limits of what was possible had been reached. I wanted to avoid the "path well trodden" of just another DOOM style 2D BSP (which is an excellent algorithm, but there's nothing new to learn) as I was a big fan of the Alien Breed 3D games and their quirky "room over room" capabilities.

My plan, then, was to design an engine optimised for 040/060/WarpOS, using Warp3D for rasterisation and provide a custom PVS solution that I felt could be optimised for lower FPU intensity. Like the AB3D games, this resulted in a zone based approach, with some lessons learned from the issues those engines had. Unlike AB3D, there was no effective limit to how many vertical zones could overlap or share an edge with a common zone. So creating a lift shaft with a dozen floors via the same wall segment was no problem. This led to some interesting data structure considerations. I had a basic prototype of a level compiler that produced the binary format and PVS connection data.

One thing that became clear pretty quickly is that there was more than one solution, each of which had better performance characteristics for a given level size/complexity and CPU target. So I thought that I might just make the whole runtime PVS a strategy pattern and then map files could contain the same PVS data in each format. By running a few hundred queries through the data on loading a level (while displaying the story so far), the engine could choose the variant that was faster for the combo and discard the rest.

I also wanted the game to be fully scriptable without recompiling or being restricted to simple table-driven behaviours. I'd had a lot of fun modding Quake (more fun than playing it) by then, hence the VM.

One day... I may just get the time to pick up where I left off.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 10:11:36
#16 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

Quote:
Indeed. I've to say that PHP maybe BIG improvements to the performance with the JIT: quite impressive!


It still has some significant overhead. For idiomatic, highly procedural/object oriented PHP, the gains are quickly lost. If it were possible to guarantee that a function call doesn't throw an exception, for example, it's clear from the JIT asm output that code could be a lot tighter.

Quote:
However the JIT has the usual problems that happens for dynamic languages. PyPy (which the best Python alternative which is based on a very advanced JIT) does more or less the same, but PHP has advantage (sic!) because you can specify the type of data, whereas Python has only type annotations (which is just metadata attached on data).


PHP has type hinting for function parameters, return and member variables of classes. It still doesn't have them for local variables or constants. Constants are likely not a problem as their type is inferred once and, well, they're constant so it can't change. However, local variables can be anything at any time.

This leads to some very annoying behaviours. For example:


function square(int $x): int {
return $x * $x;
}


As watertight as this looks at first glance, the temporary that is created by the multiplication is not guaranteed to be an int. If I pass a value for x greater than the square root of PHP_INT_MAX, it does the type juggling dance and produces a double. That then violates the return type hint and results in a TypeError exception. All of which is runtime evaluated.

Quote:
Well, this is like the ANSI art which we saw with our Amiga, but... made it animated.


I definitely remember there being ANSI demos for the amiga terminal. What I did here was to create a number of display abstractions with different capabilities and switch between them for various effects. Everything from just plain text to fully emulated RGB pixel buffer. The latter, which uses UTF8 blockmode characters to split a character cell into two "pixels" gives the higher resolution used in the plasma fire and voxelspace effects. Even with a number of optimisations, it's still megabytes of text being output.

I also got massively sidetracked implementing audio. Yes, audio. I'd previously made an experimental "modular" synth architecture to better learn the problem domain.

The music you hear in this short homage to Bad Gear is 100% realtime produced by PHP https://www.youtube.com/watch?v=qr4CvcSNHtM

I can also take pride in having the only PHP cowbell implementation on github lol https://github.com/0xABADCAFE/php-demo-engine/blob/main/src/audio/machine/percussion/AnalogueCowbell.php

Last edited by Karlos on 11-May-2022 at 10:21 AM.
Last edited by Karlos on 11-May-2022 at 10:17 AM.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 10:14:45
#17 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

Quote:
Please check the download page.


The slides look interesting! I'll have to have a proper read later.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 11-May-2022 21:19:38
#18 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@Karlos

Quote:

Karlos wrote:
@cdimauro

Quote:
Indeed. I've to say that PHP maybe BIG improvements to the performance with the JIT: quite impressive!


It still has some significant overhead. For idiomatic, highly procedural/object oriented PHP, the gains are quickly lost. If it were possible to guarantee that a function call doesn't throw an exception, for example, it's clear from the JIT asm output that code could be a lot tighter.

For Python it's even worse, because you also have the tracing mechanism that allows to control code execution in real time:
https://docs.python.org/3/library/sys.html?highlight=settrace#sys.settrace

You can image how so difficult can be a high-performance JIT for Python.
Quote:
Quote:
However the JIT has the usual problems that happens for dynamic languages. PyPy (which the best Python alternative which is based on a very advanced JIT) does more or less the same, but PHP has advantage (sic!) because you can specify the type of data, whereas Python has only type annotations (which is just metadata attached on data).


PHP has type hinting for function parameters, return and member variables of classes.

I don't follow PHP from long time, but I've just checked now and it seems that what it's called type hinting is type enforcing in reality. So, an exception is raised if you pass data which aren't the expected.

That's completely different form Python, where type annotation means that some metadata is just attached to the parameter / variable definition, and nothing else is done with it (but it's used by external programs to make static code analysis).
Quote:
It still doesn't have them for local variables or constants. Constants are likely not a problem as their type is inferred once and, well, they're constant so it can't change. However, local variables can be anything at any time.

That's very strange. I don't understand why they didn't implemented it for local vars as well.

Python has type annotations for everything.
Quote:
This leads to some very annoying behaviours. For example:


function square(int $x): int {
return $x * $x;
}


As watertight as this looks at first glance, the temporary that is created by the multiplication is not guaranteed to be an int. If I pass a value for x greater than the square root of PHP_INT_MAX, it does the type juggling dance and produces a double. That then violates the return type hint and results in a TypeError exception. All of which is runtime evaluated.

OK, I see it now. At first sight it looked correctly to me.

This resembles what happened to Python up to v2.7, where you had 2 different integer types, the short one (fixed length: machine word size) and the long one ("unlimited" / variable length). When two short integers exceeded the word length, then the VM automatically and transparently converted the result to long integer.

On Python 3 there's only one integer type (the long one), so it doesn't happen anymore (but execution is slower, of course).
Quote:
Quote:
Well, this is like the ANSI art which we saw with our Amiga, but... made it animated.


I definitely remember there being ANSI demos for the amiga terminal. What I did here was to create a number of display abstractions with different capabilities and switch between them for various effects. Everything from just plain text to fully emulated RGB pixel buffer. The latter, which uses UTF8 blockmode characters to split a character cell into two "pixels" gives the higher resolution used in the plasma fire and voxelspace effects. Even with a number of optimisations, it's still megabytes of text being output.

I also got massively sidetracked implementing audio. Yes, audio. I'd previously made an experimental "modular" synth architecture to better learn the problem domain.

The music you hear in this short homage to Bad Gear is 100% realtime produced by PHP https://www.youtube.com/watch?v=qr4CvcSNHtM

I can also take pride in having the only PHP cowbell implementation on github lol https://github.com/0xABADCAFE/php-demo-engine/blob/main/src/audio/machine/percussion/AnalogueCowbell.php

LOL I see... and hear.

Impressive. And it's quite fluid. It recalls the Amiga demos, but it's fascinating to see that everything was done in PHP.
Quote:

Karlos wrote:
@NutsAboutAmiga

Quote:
not sure why you are using web languages for this

I've used PHP to implement the commandline MC64K's assembler for a number of reasons:

1) It's extremely easy to get to a working prototype from an idea and then to continuously iterate and refine.
2) It has extremely powerful string manipulation, maps, etc.
3) It supports procedural, imperative, functional and object oriented programming styles well and with a syntax that's reasonably comfortable to use.
4) It's more than fast enough for the task at hand.

Naturally, the runtime for MC64K is written in C++ as it's the right tool for that job.

Absolutely, it makes sense: high-level languages are way much more productive.

I did the same with an (unfinished) 8086 emulator, which I've written in Python (test suite included), and the same with an ISA for a ternary system (assembler + emulator + test suite).
In particular for the latter I've used Python's decorators and docstrings to define tests for each implemented instruction. For example:


class TernaryTests(BaseTest):
@expected_ternary('++----0+00+-000---++')
@expected_output('')
@init(r2=2, r3=3)
@postconditions(r2=2, r3=3, exception=WriteToR0)
def test_write_to_r0(self):
"""add r0,r2,r3"""

The source code in the doc string is automatically compiled by my test infrastructure, then the emulator is started with some registers defined with proper values (by @init), and after execution of the instruction then emulator status (registers, exception) is checked, as well as the generated ternary (object code) and the stdout of the assembler.

Try to do it with a classic language like Java, C++, etc, and it'll take much more time to have something working.

 Status: Offline
Profile     Report this post  
Karlos 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 12-May-2022 9:41:15
#19 ]
Elite Member
Joined: 24-Aug-2003
Posts: 4394
From: As-sassin-aaate! As-sassin-aaate! Ooh! We forgot the ammunition!

@cdimauro

Quote:
For Python it's even worse, because you also have the tracing mechanism that allows to control code execution in real time:
https://docs.python.org/3/library/sys.html?highlight=settrace#sys.settrace

You can image how so difficult can be a high-performance JIT for Python.


Absolutely. I think this is ultimately the trade off between performance and expressive power. Dynamically typed languages, particularly those that have no truly enforced notions of type make this trade off somewhat knowingly. JavaScript is something of an exception to the rule as the V8 JIT was a remarkable piece of performance engineering.

I suspect that languages ideally need to be designed for JIT. I haven't implemented one but I had intended that exvm and MC64K should be AOT recompilable, though the latter may be trickier as it's more open to abuse (just like the real thing, I guess). My knowledge of x64 assembler is not up to the task yet, but it's an excuse to learn.

Quote:
Impressive. And it's quite fluid. It recalls the Amiga demos, but it's fascinating to see that everything was done in PHP.


That was the intention, really - to be keep things interesting I think you need some limits to work against and to do things that are outside the expected. People expect PHP to serve web pages and run tedious cron jobs. They don't expect it to turn their terminal into a rudimentary RGB display and play music at them (provided sox or aplay are found).

Quote:
Absolutely, it makes sense: high-level languages are way much more productive.

I did the same with an (unfinished) 8086 emulator, which I've written in Python (test suite included), and the same with an ISA for a ternary system (assembler + emulator + test suite).
In particular for the latter I've used Python's decorators and docstrings to define tests for each implemented instruction. For example:

class TernaryTests(BaseTest):
@expected_ternary('++----0+00+-000---++')
@expected_output('')
@init(r2=2, r3=3)
@postconditions(r2=2, r3=3, exception=WriteToR0)
def test_write_to_r0(self):
"""add r0,r2,r3"""

The source code in the doc string is automatically compiled by my test infrastructure, then the emulator is started with some registers defined with proper values (by @init), and after execution of the instruction then emulator status (registers, exception) is checked, as well as the generated ternary (object code) and the stdout of the assembler.

Try to do it with a classic language like Java, C++, etc, and it'll take much more time to have something working.


100% agreed. C++11 onwards has inbuilt support for Regular Expressions and lots of other stuff that make a PHP implementation strictly unnecessary, but I have no doubt that if I'd started writing the assembler in it, I'd not be at a useful point by now. I thought about porting the whole lot back to C++ once it was at a functional stage of development but the truth is it's just not necessary; even without JIT (which is the default even in 8.1) it's perfectly fast enough. I created assembler test cases that were thousands of lines long and it could assemble them in milliseconds.

_________________
Doing stupid things for fun...

 Status: Offline
Profile     Report this post  
cdimauro 
Re: Virtual machines, instruction sets, that sort of thing...
Posted on 13-May-2022 4:54:09
#20 ]
Elite Member
Joined: 29-Oct-2012
Posts: 3621
From: Germany

@Karlos

Quote:

Karlos wrote:
@cdimauro

I suspect that languages ideally need to be designed for JIT.

Sort of. At least it's what I expect from for modern languages.
Quote:
I haven't implemented one but I had intended that exvm and MC64K should be AOT recompilable, though the latter may be trickier as it's more open to abuse (just like the real thing, I guess).

Well, exvm and MC64K are architectures, so JITs for them should be totally transparent.

What helps a JIT/AOT compiler, in general, is the metadata that you embedded on the binaries generated.
For example entry points for functions (with prologue/epilogue) and lines or even (this is especially useful) instructions debug info. This helps understanding how the code / instructions workflow is made / connected, and generate better code (or generate the entire code if the goal is AOT).

The problem comes with pure binaries (like 68K JITs for our Amiga), where usually you have no metadata and you need to build it at runtime...
Quote:
My knowledge of x64 assembler is not up to the task yet, but it's an excuse to learn.

It's quite simple for people coming from 68K: it's a CISC, with some similarities (stack pointer as regular reg, base / indexed memory modes, mem instructions). You need to get used to the slightly different syntax, but this you'll do quickly.

 Status: Offline
Profile     Report this post  
Goto page ( 1 | 2 | 3 | 4 | 5 | 6 Next Page )

[ home ][ about us ][ privacy ] [ forums ][ classifieds ] [ links ][ news archive ] [ link to us ][ user account ]
Copyright (C) 2000 - 2019 Amigaworld.net.
Amigaworld.net was originally founded by David Doyle