▶️ #CTRAN: A drop-in FOSS replacement for CTRAN.EXE, the #Psion SIBO/EPOC16 OO C preprocessor on #DOS. Almost feature complete, but still work to do!
▶️ #ObjectPascal with #FreePascal: Used for CTRAN for easy development and portability. Honestly, I'm really enjoying it; it fits my needs and makes my brain happy.
▶️ RAM upgrade for 3mx to 4MB: Trying to source old DRAM isn't easy.
▶️ RAM upgrade for 5mx: Got the DRAM. Just need to solder it in place.
Just tried compiling Wari, a game written in #Psion OO C. The project uses #Borland Make 3.6.
Got it to compile first time with my SIBO SDK setup - all good!
However... Borland Make uses 16-bit DPMI, and its extender won't load 32-bit DPMI binaries. If I pre-load the 32-bit extender, it won't load 16-bit DPMI binaries, so Make won't run!
TL;DR: I can't use the new #ctran with Borland Make 3.6.
Looks like I'll be converting that Makefile to GNU Make or a #TopSpeed project.
Why can't I recompile #ctran for 16-bit #DOS? Because many of #FreePascal's libraries are too big to fit into 64 KiB data blocks, and won't compile no matter which memory model I use.
There is a chance I'll rewrite ctran in the future. #FreeVision (the #Borland#TurboVision "clone") with #ObjectPascal is certainly an option. I could also rewrite the lot in C or C++.
But today is not that day. For now, I'd rather just rewrite a Makefile.
I wonder if it's worth making a little interface using #FreeVision (the #TurboVision-compatible library that comes with #FreePascal) to display information about #Psion OO category (class definition) files?
Yes, I realise this is feature creep. But currently I'm outputting a lot of information to the terminal that the original CTRAN.EXE doesn't do. How much do I leave in as a "verbose" option, and how much to I move to a shiny TUI?
Done a lot of work on #ctran tonight. In fact, I think we might have a working parser! I definitely need to check over what I've done, but it's looking promising.
I need to add more checks to make sure that tokens don't appear in files that aren't meant to have them (e.g. DECLARE should only be in .EXT files). Then I need to test that the tree is being built the way I think it is.
After that, it's time to do something with EXTERNALs and REQUIREs to make MANY trees.
✅ Expose the "tree" as read-only properties, as 5 separate arrays.
I thought this was going to be awkward. #FreePascal won't let you create a property that is an "array of" some type. I assumed I'd have to do a load of rewriting to accommodate new custom types.
Turns out I just needed to create the new types and put them in the relevant public/private declarations. And for the three "array of string" variables, I just used TStringArray. No other changes. It Just Worked.
I'm starting to wonder if there's any point in having the lexer and parser as two separate classes.
Other than testing, the lexer is only ever going to be called by the parser, and only once during the process.
It might be better to just have a lexer-parser class that grabs a file, tokenises it, then (if it's happy with the file it's tokenised) immediately turns it into a tree.
Is there a really good reason why they should be separate classes?
About 30 lines of code later, and I can now pull lines of tokens, one by one. I've also been able to remove the generation of newline tokens, as they're no longer needed.
I didn't realise in #FreePascal that Result is preserved in a function! I'm having to nil one of the dynamic arrays every time, otherwise it remembers what it generated last time.
#ctran's parser has now been disabled because it relied on newlines, but that's fine. It's about to be torn apart.
Been thinking about #ctran and ASTs today. I've realised that I might be approaching this wrong.
#Psion OO C files are definition files. They're glorified mark-up, wrapping around snippets of C code that just gets copied to new files. There's no logic in them at all.
I basically need two arrays per file: a list of "classes" and their contents, and a list of "includes" and how they're, uh, included (EXTERNAL, INCLUDE, or REQUIRE).
It's still a tree, but it's got two distinct parts. Wonky tree.
Actually, I wonder if variant records would be useful for building a tree, or at least an array containing elements of different types? Need to read up on them.
I say "maybe", because it looks like variant records have a fixed size. If that's the case, the #FreePascal might not like me having a dynamic array of variant records, where some of the variant records contain dynamic arrays of AnsiStrings. Unless Free Pascal does some clever memory management in the background.
Another option is to add an "item number" variable to each record, then searching for the next one so that they're processed in order.
Interesting... I'd assumed that #FreePascal's 16-bit #DOS cross-compiler was just ignoring the memory model I was setting. Turns out it wasn't, and I hadn't been reading the error messages properly. They're all failing for different reasons!
Surely I must be doing something wrong?
I'm struggling to find any information on issues like this, largely because hardly anyone uses it.
I might end up switching to the 32-bit DOS compiler, just to get something working.
Well, the #FreePascal community have kindly responded on the Lazarus forum, and it seems that the Classes unit just won't fit into 64K and there's no way around it.
So, we are basically where I (and others on Mastodon) thought I'd be. Either extract individual parts from the Free Pascal libraries, or roll my own solutions.
There's also Free Vision (a FOSS API-compatible replacement for Borland's Turbo Vision framework), which I've been told includes some alternative lists.
#CTRAN currently has a lot of repetitive code in the #tokeniser. I really want to refactor it so that it's both tidier and more readable. I also know that I'm going to need to reuse a lot of the tokeniser code to process #Psion's .EXT files, which are a sort of cut-down version of the class/category files.
I'm in a position where the tokeniser works well, so I think I can afford to shift things around. I think it's time to put the tokeniser in a class.
Well, the tokeniser is now in a class and still works! Procedures and functions have been pulled in and simplified, and private things are strict private. Tidying the code will now be much easier.
Have a look at the code so far on GitHub. Feel free to critique - I know there's much to be improved, but I could have missed something.
Currently trying to decide whether I should work on getting the #FreePascal 16-bit DOS cross-compiler to work with the medium memory model, or if I should just keep ploughing on with coding without really knowing if the code I'm writing works on DOS.
The cross-compiler should just work. I'm using the binaries, fpc.cfg matches the wiki, and I've checked my command line switches (-WmMedium). It sticks stubbornly with the small memory model.
Hmm, adding uses Classes to get TStringList has made the executable even bigger. And for some reason the #FreePascal 16-bit DOS cross-compiler is stuck in small memory model, no matter what switches I provide. fpc.cfg looks right when compared to the wiki, and I have all the necessary units installed from the binary distribution.
I'll have to come back to that.
I guess if I wanted something small, I'd have written it in C from the start and rolled my own nice-to-have libraries.
Thank you to @rvr for sending me this. I've had a quick read through and it looks to be exactly what I need. Especially the section "7.2. Containers (lists, dictionaries) using generics".
Turns out that Free Pascal already has a very good HTTP server and JSON de/serialization library (with JSON<>Object serialization) as part of their stdlib.
I played around with them a bit and I'm impressed with the performance. The compiler is also amazingly fast.
Don't be fooled, Object Pascal is not a "students" language anymore. They all moved to "React" now. ;-)
#FreePascal compiles for almost everything (See the picture), including 16-bit DOS.
It's also still maintained, with an active community.
Using #ObjectPascal (which is basically what Free Pascal is), I can get the logic written for a lexer, an AST, and then a few code generators, without worrying about how I'm going to allocate memory for an array, let alone an entire tree.
With #FreePascal I can write code on Linux, compile it on Linux, and then cross-compile for DOS with no change to the code so that it can be used with the existing SDK.