@jorisvandenbossche@fosstodon.org avatar

jorisvandenbossche

@jorisvandenbossche@fosstodon.org

Open source #python software developer and teacher. Pandas core developer. GeoPandas and Shapely maintainer. Working on Apache Arrow at Voltron Data Labs

This profile is from a federated server and may be incomplete. Browse more on the original instance.

kylebarron, to random
@kylebarron@mapstodon.space avatar

From @jorisvandenbossche :

GeoArrow enables a 30x speedup of reading GeoPackage and FlatGeobuf from GDAL into @geopandas !

jorisvandenbossche,
@jorisvandenbossche@fosstodon.org avatar

@geospacedman @kylebarron @geopandas It's not thàt big data ;) The geometries (3.3 million polygons) take a bit more than half a gigabyte in memory, but for example calculating the area of all those polygons with geopandas takes less than 0.2s for me.

jorisvandenbossche, to random
@jorisvandenbossche@fosstodon.org avatar

GeoPandas 1.0 is around the corner! @martinfleis and I just presented about this at the GeoPython conference in Basel, you can find the slides here: https://docs.google.com/presentation/d/1SphOTBLPV8HGcqzfjLkGja-3NlSAB_6NyID1Ky2mvrM/edit?usp=sharing

EvenRouault, to random
@EvenRouault@mastodon.social avatar

Is geospatial Parquet story going to be messy? So up to now GeoParquet was regular Parquet + a bunch of metadata telling "oh this binary column holds WKB geometries in case you care" (like GeoPackage). And now there's this proposal https://github.com/apache/parquet-format/pull/240 about adding a native Parquet geometry type.

jorisvandenbossche,
@jorisvandenbossche@fosstodon.org avatar

@EvenRouault I don't think it has to be a mess, if we try to ensure that what is standardized in Parquet itself is a natural evolution of what we have been doing within the GeoParquet community (e.g. AFAIK we should be able to ensure that such a new logical data type is fully compatible with existing readers, and it's only when you want to make use of the statistics that you will need custom handling of it).

jorisvandenbossche,
@jorisvandenbossche@fosstodon.org avatar

@EvenRouault At that point, for example a potential future GeoParquet 1.x spec release could say that a geometry column encoded as WKB can use the GEOMETRY logical type in addition to a plain BYTE_ARRAY physical type.

(that said, I also not entirely sure of the added value compared to what GeoParquet already offers)

sgillies, to random
@sgillies@mastodon.social avatar

Reminded myself that I tried to write a zero-copy GEOS adapter for coordinates stored in Numpy arrays in 2011, but gave up because I didn't know enough C++ and knew even less about how to package such a thing. https://web.archive.org/web/20110610174600/http://lists.gispython.org/pipermail/community/2011-January/002868.html

jorisvandenbossche,
@jorisvandenbossche@fosstodon.org avatar

@sgillies A more recent reincarnation of this: https://github.com/libgeos/geos/pull/747

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • megavids
  • thenastyranch
  • rosin
  • GTA5RPClips
  • osvaldo12
  • love
  • Youngstown
  • slotface
  • khanakhh
  • everett
  • kavyap
  • mdbf
  • DreamBathrooms
  • ngwrru68w68
  • provamag3
  • magazineikmin
  • InstantRegret
  • normalnudes
  • tacticalgear
  • cubers
  • ethstaker
  • modclub
  • cisconetworking
  • Durango
  • anitta
  • Leos
  • tester
  • JUstTest
  • All magazines
    •