Astro Hacker News - Protomaps: Self-hosting map tiles has gotten cheap and easy

boramalper |next [-]

OpenFreeMap is a similar project:

> What is OpenFreeMap?

> OpenFreeMap lets you display custom maps on your website and apps for free.

> Using our public instance is completely free: there are no limits on the number of map views or requests. There’s no registration, no user database, no API keys, and no cookies. We aim to cover the running costs of our public instance through donations.

> What is the tech stack?

> There is no tile server running; only nginx serving a Btrfs image with 300 million hard-linked files. This was my idea; I haven’t read about anyone else doing this in production, but it works really well.

bcye |root |parent [-]

I didn't know they just served it straight from disk, that's pretty cool!

neomantra |next |previous [-]

We're doing something like this behind Cloudflare R2, same concept of owning the map infra (for privacy and access-control) and reducing load on public infrastructure.

If you are into this stuff, you can view PMTiles from the CLI with "gloss", or try to drag-and-drop on its web interface [1]. It seems large tiles (>128M) and some raster formats are failing; plus it would be nice to load from URL there. Will fix that...

(AI warning). ntmaps [2] is a Golang library that lets you use different tile providers, deployable in terminal or browser. ntplaygrounds [3] uses that and nt3d and ntactor to make a city simulation. Data is pulled from OpenStreetMap in those.

[1] https://nimbleterminal.com/gloss/app.html [2] https://nimbleterminal.com/ntmaps [3] https://nimbleterminal.com/ntplaygrounds

jamessb |root |parent |next [-]

> If you are into this stuff, you can view PMTiles from the CLI with "gloss", or try to drag-and-drop on its web interface

The official PMTiles viewer [1, 2, 3] is also pretty good.

[1]: https://pmtiles.io/

[2]: https://github.com/protomaps/PMTiles/tree/main/app

[3]: https://protomaps.com/blog/new-pmtiles-io/

cldellow |root |parent |next |previous [-]

R2 is amazing for this.

Cloudflare, generally, is amazing for this, actually. It serves my personal hiking map for about $2/mo, using R2, D1 and Workers.

Building the map itself is free via GitHub Actions.

It's bonkers how the map world has changed over the last 20 years.

is_true |root |parent |next [-]

That's mostly for the storage not the bandwidth right?

cldellow |root |parent [-]

It's pretty much all storage.

kellpossible2 |root |parent |previous [-]

how are you using D1 and Workers with this?

cldellow |root |parent [-]

Two small uses:

1. The app is a remix app, workers do server-side rendering, so that a vanilla `curl` shows the actual payload, not just a script tag that loads the app.

2. I'm a great big nerd, so I recently added the ability to post reviews/photos/GPX traces to your AT protocol PDS. The app has a Worker cron job that subscribes to Bluesky's Jetstream for events in its namespace, and indexes these into D1 in order to answer certain API calls like "get traces in this area" or "get reviews for this hike". So the app itself isn't the primary source of truth, a person's PDS (which they control) is. The app just maintains a cache/index that can be rebuilt.

tjwebbnorfolk |root |parent |previous [-]

same, we use pmtiles on R2 for landstats.com parcel map. it's very easy to set up and cheap to serve

and we have a CF worker than turns them into rasters on the fly if that's how people want to get them

eruci |next |previous [-]

I deployed self-hosted protomaps on geocode.xyz and it works like a charm. I wish it had more customization options though ( https://geocode.xyz/415%20C'WEALTH%20AVE%20WEST%20Singapore )

motoroco |next |previous [-]

yeah, basemaps are small enough to store and host cost effectively, but high res satellite imagery gets big and expensive fast as you go up in zoom level. I built my own satellite mosaic of North America for my app so I can offer users offline access to the tiles and had to stop at z17 because of projected storage requirements. high res global coverage can get you into the hundreds of TB range which is no longer in the "stick it on r2 and serve it with a worker" or "download and upload the entire dataset sequentially over your residential internet" territory

the fun/hard parts of the project weren't really scale dependent, although downloading source imagery and building the tile pyramids did take literally weeks to run. actually, the hardest part was getting demosaicing right, e.g. making the quilt seamless. filtering out cloudy and smokey/hazy tiles while picking alternatives from the same time of year so groundcover & crops are in a consistent growth stage so you don't end up with checkerboard patterns or cloud silhouettes randomly across your map. and there were other problems that came up more or less just because of how & when the satellite took the images that require exposure correction and more filtering

I also have a pelias index with 170M records for search and self-host valhalla for routing. the open source mapping ecosystem has really impressed me the more I learn with each project

maelito |root |parent [-]

Have you check Mapterhorn's next project ? Funded by Nlnet.

motoroco |root |parent [-]

yeah, I heard about them for the first time a few weeks ago. they have a fantastic data set just judging with my bare eyes. I built my topo map from terrain tiles on aws + gebco et al for the bathymetry data which is just personal preference (I love me an old globe with the tactile mountains and seafloor)

nchmy |next |previous [-]

I was very grateful to find protomaps recently for a project where I want to allow people to download custom regional subsections of the global map for offline usage.

After evaluating maplibre vs the somewhat-abandonned protomaps-leaflet library, I decided to go with the latter as I don't really need the fancy 3d gpu and custom layers functionality of maplibre. Plus leaflet is releasing a long-awaited v2 right now.

joshstrange |next |previous [-]

I use Protomaps and absolutely love it. After getting hit with a Google Maps bill that was larger than I ever would have expected, I swapped it out for Protomaps instead and I’ve loved it ever since.

Dirt cheap to run, theme-able, easy to use.

perks_12 |next |previous [-]

Deploying this as I am typing this. The problem I have with Mapbox is that there is no way to limit spending. When the public key gets abused, it's a potentially enormous bill.

maelito |next |previous [-]

For Cartes.app we used to pay Maptiler. While great, recommanded to start, without client revenue it was impossible to scale, we've hit 150 €/month bills way too fast.

Hence Protomaps. Hosting costs almost nothing.

The main problem though is that once you host your tiles, why not put what you want inside ?

Tune Tilemaker's processing.lua to your envy. Spend hours designing maps and building a Typescrpt dynamic Maplibre style.

maelito |next |previous [-]

Checkout the new PRs submitted to Tilemaker. It appears building the planet could soon take less than 5 minutes.

rafram |root |parent [-]

Any in particular? That’s pretty cool. I switched my own workflow to Planetiler because Tilemaker had too many bugs and quirks for my liking, but I might be tempted to switch back if it’s that much faster.

doodlebugging |next |previous [-]

I visited the link and Malwarebytes warned of a potentially malicious site:

p - proxy - tp8ez dot bunny dot run

so I closed that window.

What else you got?

bcye |root |parent [-]

It's my API endpoint for Plausible (private website analytics), not sure why it would get flagged

doodlebugging |root |parent [-]

Yeah, me neither. I opened the link along with several other article links from the front page so it sat idle in a separate private tab waiting for me to get to it and just as I clicked on it MWB popped the warning. I'm using LibreOffice with container tabs and MWB Pro under Win10 Pro. Probably nothing.

Thanks for the explanation.

Protomaps sounds interesting.

edumucelli |next |previous [-]

Really cool, did not know about it!

pluto_modadic |previous [-]

okay, but what clients work with this approach?

perrygeo |root |parent |next [-]

You can use the pmtiles:// protocol in Maplibre GL, requires protomaps.js and a few lines of setup [1]

My experience with it is ... ok. It works. But going directly to cloud storage (s3 in my case) is unpredictable latency and neither the browser nor most servers really cache HTTP range requests granularly, so every tile is another high-latency round trip. Not worth it - I ended up putting Martin tile server in front of it.

I assume someone could write a protomaps adapter in WASM with some smart caching? Until then, Standard GET requests are just easier to cache and perform much better in practice. The downside being you have to run a server or statically generate files.

[1]- https://docs.protomaps.com/pmtiles/maplibre

wkrp |root |parent |next [-]

I've had success with using a cache in front of my buckets for great performance. In my case I have raster tiles so they get turned into z/x/y tile filenames and a cloudflare worker pulls that from the full file and caches the result in the region.

bcye |root |parent |previous [-]

For what it's worth I think latency/reliability varies a lot with provider. Bunny Storage has quite good latency in my experience (100-300ms) and their CDN supports Range caching (from my understanding Cloudflare does too they just limit the size of the file regardless of the Range)

cr125rider |root |parent |next |previous [-]

Protomaps, pmtiles, are actually pretty ubiquitous. Even has a GDAL driver https://gdal.org/en/stable/drivers/vector/pmtiles.html

bcye |root |parent |next |previous [-]

There is an npm library that integrates the format into MapLibre (fork of Mapbox) and Leaflet.

pstuart |root |parent |previous [-]

A quick search showed that this should work with HTTP range requests: https://maplibre.org/