Posit, The sparklyr package and friends have been getting some important updates in the past few months!
sparklyr is a package that allows you to interact with Spark using familiar R interfaces, such as dplyr, broom, and DBI. You can also gain access to Spark's distributed Machine Learning libraries, Structure Streaming, and ML Pipelines from R.
Read more in the blog post: https://blogs.rstudio.com/ai/posts/2024-04-22-sparklyr-updates/