GrapheneOS Server Infrastructure Changes Involving New ASN, DNS and New Servers Away From France
We host our own authoritative DNS servers to provide DNS resolution for our services. Authoritative DNS are the servers queried by DNS resolvers run by your ISP, VPN or an explicitly user chosen one such as Cloudflare or Quad9 DNS. We now have our own AS and IP space for this.
You can see information about our AS and IP space here:
We received a free ASN, IPv6 /40 and IPv4 /24 from ARIN. We use one IPv6 /48 for our ns1 anycast DNS network and one for our anycast ns2 network. We're using the IPv4 /24 for ns2 and need another.
Our ns1 network currently has 10 locations: New York City, Miami, Los Angeles, Seattle, London, Frankfurt, Singapore, Mumbai, Tokyo and Sydney. We're considering moving London to Amsterdam. We plan to add a South American location and perhaps Warsaw. ns2 isn't as scaled out yet.
Our ns2 network currently has New York City, Miami, Las Vegas and Bern.
Here's latency to ns1:
ping6.ping.pe/ns1.grapheneos.o…
ping.pe/ns1.grapheneos.org
Here's latency to ns2:
ping6.ping.pe/ns2.grapheneos.o…
ping.pe/ns2.grapheneos.org
We plan to add more locations to ns2 via another provider.
When we begin a reboot of a server, the change propagates across all internet backbone routers within a few seconds. This provides high availability for server downtime too. We have 2 networks so routing/transit issues or a malfunctioning server don't break using our services.
For ns1, there's a mix of different upstream transit providers. We've done traffic engineering with BGP communities configuration to get traffic routed to the right places. We prioritize Arelion and NTT since nearly all locations have both and we can configure their routing well.
We make the routes announced by our servers deprioritized when propagated into other continents for Arelion, Cogent and NTT. We deprioritize transit ruining global routing (GTT, Lumen) and block some peering (RETN, Bharti). We deprioritize Cogent since only 3 locations have it.
Our authoritative DNS server setup is largely in a public Git repository:
github.com/GrapheneOS/ns1.grap…
Here's our BGP communities setup ns1 New York City as an example:
github.com/GrapheneOS/ns1.grap…
Here's ns1 Miami with different handling for South America:
github.com/GrapheneOS/ns1.grap…
We have two main groups of servers around the world:
1) website and OS network services
github.com/GrapheneOS/ns1.grap…
github.com/GrapheneOS/ns1.grap…
2) update mirrors, which are currently 3x sponsored dedicated servers with 10Gbps
github.com/GrapheneOS/ns1.grap…
We'll have more of both soon.
We're in the process of our website and OS network services away from OVH due to the threats from French law enforcement. We're going to add nodes in South America, India, Japan and Australia as part of this. We also have 5 non-static-content servers in Canada to move off OVH.
The servers with more than static content are our discussion forum and attestation service for our users along with our email, Matrix and Mastodon servers for our project. These will move to colocated servers in Toronto long term but short term we'll just switch providers for it.
vas
in reply to KindnessInfinity • • •Personally, I'd feel safer if it were in Europe.
For example, in the Netherlands, which quickly and clearly was against ChatControl.