i have successfully borrowed 2^46 bytes from the address space for mirrored addressing. the pinned address seems to be outside of the typical ASLR / dynamic mmap ranges, but if it fails for a particular program layout, the user can specify a different constant address. no idea how portable this is though, under windows we might have to do something else. #UVM
... but it's probably better to abstract the stack/heap in the IR; interpreter and compiler can then choose a faster implementation, and static analysis can pick a slower one. as long as the operations are linearly connected, it should remain pure.
decided against abstracting a heap in the IR, since it really just exists to facilitate C interfacing; otherwise heaps are just another form of index table based referencing, and for that i want to develop techniques that remove indices where possible and reconstruct direct object relationships.
this is what the value <-> address mapping instructions look like in the interpreter. the mim state is a partially updateable 128-bit rolling hash of the entire 46-bit mapping.
i renamed the MIM (memory interface map) to MEM (memory export map - a recursive acronym, too!) so it is easier to understand even without context, and expanded on the instructions.
i realized any MEM is equivalent to a sparse deduplicated image of stack and heap, and as such our UVM is much closer to VM's as people know it, but with much better caching where possible.
hmhm. seems like i need to emulate one more memory section in addition to heap and stack: globals; i already use bucket 0 for the stack, but i could just put all globals at the bottom of the stack.
wow, hello world works! this is the first time we use the memory export mapping; though here the locations are premapped since the pointers need to be valid before we even enter the program.
the function that syncs our mapping changes to memory got pretty involved. but now we can do it all safely and sparsely, and the code wll port to windows later.
i love the synergies of this btw. we're practically turning our programs into (mostly) pure functional continuations that do all their memory allocation management manually. sounds weird, but this is extremely helpful on the GPU, which doesn't expose an address space. we could provide our own r/w mapping on the GPU, and then alloca/malloc/free based CPU code will "just run" there too.
next feature: safely catching when C functions modify memory, so we can save the change for return values and also restore our immutable state.
a simple (and very slow way) could be to track which memory might be reachable by a C call, & then compare that memory afterwards to find differences.
OR we could do this a bit quicker and a lot more safely and employ userfaultfd() to catch all writes. outside of ccalls, writes wouldn't be permitted at all, otherwise we log & handle it afterwards.
intermezzo to get printf support. this is what a function signature looks in UVM btw. we only have them for libffi, effectively (and they're actually enough to lower the code to C as well) so they're just a bunch of vaguely nested enums.
one special thing here is that for variadic calls, the signature must also contain the extra argument types.
this is pretty much the entire type system you need to interface with the C ABI. everything else is just by convention.
i'm playing with page fault handling and getting the hang of it. these write-protect handlers work like mousetraps - they snap once, and then have to be primed again. which is optimal; once the page is tagged dirty i can wait until after the C call, figure out which addresses have been affected, do the rollback, then rearm the traps.
the handler thread can rearm the traps immediately, but then everything comes to a crawl, as every little write waits for permission.
phew. implemented a whole heap today (malloc/free/alloca) for the interpreter. these are the bookkeeping fields i ended up with.
free() was particularly tricky; when you punch holes in the heap, you have to make sure you merge with adjacent holes so the space is free again. (for that i needed freed_by_addr) - i suppose this job is what allocators thread.
malloc() wants the smallest free pointer with the smallest fitting size; for that, freed_by_size.
Add comment