@ELLIOTTCABLE@mcc@glowcoil my recommendation is to first see if there's a high level rendering library that can do what you need before reaching for a low level one. The things you want to decide early and try to stick to as best you can are 1) what sort of things do you want to render, what do they look like, and how do they change over time, 2) how many of them do you need to be able to draw at once, and 3) how many of them need to be able to move.
@ELLIOTTCABLE@mcc@glowcoil if you can know roughly all three of these things ahead of time then you can evaluate relatively easily if an existing library or engine will do what you need, because you can see from the docs and screenshots what your limits are for the first thing, and you can set up some basic stress test experiments to see if a given engine can handle the other two
@ELLIOTTCABLE@mcc@glowcoil I don't recommend taking on low level graphics programming, VR interaction, and UI programming all at once for the first time (unless you don't mind spending a few years learning and experimenting) without using a high level library or engine to help on at least some of these, because all three of these are extremely deep fields with tons of complexity and nuance
@ELLIOTTCABLE@mcc@glowcoil so start with this: write out a short design document for the basic constraints you intend to follow (do you need just basic models with simple textures, a full PBR lighting model with sophisticated emulation of camera optics, point cloud rendering, do you need to animate anything, how many things need to be drawn, how many things need to move, what kind of information needs to be communicated via the UI, what sort of interaction verbs the user has, etc)
more context: have Theories (bouncing around in my head for a decade from langdev work) about programmmer productivity, visible context, and how stupid plaintext really is … and i kinda feel like AR has small-but-genuine potential to change the math on “plaintext vs. structured code.” i want to fiddle with an AR VScode extension over this coming year to validate my theories.
thing is, I know myself, and I’m a much better bottom-up learner — I strongly prefer a solid foundation, even if some of the foundational knowledge turns out to be irrelevant. (it helps motivate me, keep me focused)
so, kinda looking for 1. basic “graphics programming bootstrap”, even if i’m eventually going to 2. definitely pick a high-level, platform-friendly toolkit, and do my level best to avoid needing anything i learned. 😅
@ELLIOTTCABLE@mcc@glowcoil so I think mcc is going to have the most helpful answers here regarding what you want to do. I would recommend starting with building out a very basic prototype in LOVR first to help you understand the nuance of your problem space better. This might seem like a diversion from fundamentals, but it isn't, because this exercise will help you focus on which fundamentals are most relevant.
@ELLIOTTCABLE@mcc@glowcoil as for learning a low level API at the same time, is there a particular platform or range of platforms you would like to be able to target?
@ELLIOTTCABLE@mcc@glowcoil Something that might not be obvious from the outside, but all graphics APIs are high level APIs for submitting command packets to the GPU. Commands mostly boil down to moving memory into and out of the GPU and "run this shader program in a particular way with these resources attached". The really sensitive granular performance work is mostly shaders and data flow, and everything else in rendering is math.
@aeva@ELLIOTTCABLE Actually I think you should read my blog post about WebGPU just because it incidentally explains the relationships between the different low level APIs (and maybe watch enough of my LOVR video to see the part where I explain the various high-level VR engine options as of last year)
@mcc@ELLIOTTCABLE oh I have, it's a great blog post. I was holding back from saying "go check out WebGPU" because I haven't used it and I figured I'd let you have the honor
@mcc@aeva 10,000% on my todo list now within the next couple of days.
currently hunting down podcasts for the interim, as I’m driving across the U.S., and videos/reading are rather less accessible; but oof am i glad i knew y’all
@ELLIOTTCABLE@mcc@glowcoil and virtually all modern engines expose at least some amount of shader functionality, so in a way low level vs high level is a false dichotomy
@aeva@ELLIOTTCABLE I think Aeva is basically correct here. Here are my takes:
The market dynamics of games right now are such even for small personal projects, Unity is heavily favored/required. Especially with VR, the platforms support Unity far better than any other API including raw C; on Vision you may be required to use Unity. You can do low level gfx programming in Unity using command buffers (although this may not be allowed on Apple Vision) or URP.
@ELLIOTTCABLE 2. The VR platform I personally use is called LOVR. It currently has a next-gen graphics API (IE based on Vulkan). I will be trying to make it run on the Vision but I might fail. I made an extended intro video to it: https://www.youtube.com/watch?v=u9cYW6ffkPM
This lets you write VR apps as small demo-like Lua scripts. It's great for rapid prototyping, however, it may not work as well for complex VR UI as you will have to write that UI "by hand" whereas in Unity u use VRTK https://forum.unity.com/threads/vrtk-unity-and-vr.1046245/
@ELLIOTTCABLE 3. Aeva is correct you should not attempt to learn VR and learn low-level GPU programming at the same time.
If you decide you want to learn low-level GPU programming to learn low-level GPU programming, I recommend WebGPU. I also wrote an intro to that here: https://cohost.org/mcc/post/1406157-i-want-to-talk-about and am currently writing an actual tutorial. TLDR: WebGPU is a low-level wrapper that exposes the full power of Vulkan/DirectX/Metal AND it's easier to use AND it is now built into web browsers.
@ELLIOTTCABLE So WebGPU is the lowest level API available on web, and it is a good way of learning how low level graphics APIs "work", and it talks natively to all of JavaScript, TypeScript, C++ and Rust. And if you are interested in VR, you can integrate it with WebXR (but possibly not on Apple Vision, whose requirements re: WebXR are not yet clear to me). This is all compelling enough it would be what I flat-out recommend you use, EXCEPT—
@ELLIOTTCABLE —EXCEPT, if your goal is to FINISH a VR project, you will be slower if you use WebGPU, especially if you're learning as you go, especially if you're working without my tutorial (which I am still writing it). You'll bootstrap far quicker if you use a higher-level API that lets you think about "what am I drawing" rather than thinking about buffers and pipelines, such as LÖVR, Unity or any number of things.
There are several new and possibly compelling high level graphics APIs, some of which I haven't evaluated, such as Meta IGL (released yesterday!), Godot (Unity but open source, supports many languages, has improved SO much lately), and RedGPU (???)
If you want to Actually Make Something and not write code and draw cubes, the most important thing to do is ignore APIs and learn Blender. Seriously.
PS have u opinions re language(s) u can be happy writing in
@mcc i’m pretty language-agnostic at the moment; trying to stay true to a goal of “proof out a Human Concept,” not get muddled down on which tech is funner (which is. lol. so, so easy.)
generally OCaml, maybe Rust. JavaScript/TypeScript are trivially familiar to me, and thus good alternatives when i need to move fast but OCaml is a poor choice for some reason.
platform-wise, though, it probably doesn’t matter to me, largely because i have the distinct impression that no other VR platform than Apple Vision is about to matter … (not sure how true that is, just a low-key fear) … and thus i should just shut up and use whatever supports that? ¯_(ツ)_/¯
@ELLIOTTCABLE Even on MacOS. Unity wrote their own C# compiler and they are 100% hard all-in on it. That's the point of Unity. They were originally The C# Game Engine.
I think Godot just uses normal Mono. This works 100% fine on mac, literally no problems. I'm not sure if it works on iPhone, however maybe if you use it in conjunction with Microsoft's "Xamarin" project (which I've got familiarity with but not recently).
@ELLIOTTCABLE Also, bad news. When I say C# I mean C#. Godot's C# support will I feel certain support F# fine, since I assume they're using Mono, but Unity may not. You can check Google to be sure but the thing is Unity actually did write their own C# compiler and although it ingests CIL and therefore you CAN write a plugin that compiles F# and feeds it into Unity, however clumsily, you are "outside the warranty" and cannot expect it to work.
@mcc mehhhhh no big deal. again, humans and how they write software are the interesting bit here; I’m sure C#’ll be fine.
hell, if anything, it’s always nice to fall back into an Extremely Mainstream toolkit; makes it so easy to believe you’ll be able to welcome external contributors / OSS participation in the future. gives me some Good Citizen feels, even if i’m personally bored. 🤣
okay! thank you so much for all the advice! a lot to work with here.
@mcc@ELLIOTTCABLE don't quote me on this but I vaguely recall that Godot does something really diabolical to support C# specifically, and you don't actually get the benefit of being able to use other dot net languages
@aeva@ELLIOTTCABLE Checking and… god damn it Microsoft really did something nasty to us when they decided to have the C# VM specification technically be named the "CLI"
"Since Godot uses .NET 6, in theory, you can use any third-party .NET library or framework in Godot, as well as any Common Language Infrastructure-compliant programming language, such as F#, Boo, or ClojureCLR. However, C# is the only officially supported .NET option."
@mcc@aeva@ELLIOTTCABLE some users managed to use F# in Godot 3, and it seems that 4 needs .net 8 to solve some limitations.
In the part of graphics it may not be easy to enter there right now but you can ask in chat.godotengine.org
Raylib could be a good place to start, has a small codebase and it was made originally for educational purposes.
@eons@mcc@ELLIOTTCABLE it might have been godot 4. I know it was godot because my friend is making a game in godot and is a huge f# enthusiast and was disappointed he couldn't use it, but I don't remember the exact reason why
@mcc i love the idea of that, but that definitely sounds like scope-creep :P
I think I’ll indeed stick to seeking out resources that are either A. hella general, or B. specific to a set of higher-level constructs, lol.
my only signal for “low-level” was my innate awareness (i.e. series of partners and friends) of how much even the slightest rendering latency in VR seems to induce nausea, or at least low-key annoyance — i’ve got a vague notion of the sorts of nice abstractions i’m used to, inducing unacceptable amounts of latency, for a tool someone would theoretically use 8+ hours a day …
i think y’all’ve somewhat disabused me of that fear, though — apparently VR toolkits do, in fact, exist, and are unlaggy enough that not every single person wanting to do any UIs in VR needs to obtain low-level optimization knowledge first to avoid nauseating people? that’s relieving tbh. 😵💫
@ELLIOTTCABLE modern vr headsets have something called "asynchronous time warp" where even if the program is turning in frames at 20 fps they can "reproject" the last frame repeatedly and present 90 fps to the user.
in addition, "full engine" platforms like Unity or Godot have the potential to be drawing the scene at 90 FPS even if your mechanics code is slow and runs at 10 FPS. But I don't know how they perform in practice.
Add comment