SCION - A Novel Internet Architecture

The SCION BoF at RIPE 81

“This is something different than the usual IP protocol suite that we are going to talk about today”

This is how I welcomed participants at the first SCION BoF at RIPE 81, which I organised together with Victor Reijs (SIDN Labs) and Adrian Perrig (ETH Zurich). SCION, which stands for Scalability, Control, and Isolation on Next-Generation Networks, is a clean-slate Internet architecture that has been implemented and is in operation. SCION has been designed to fundamentally solve many security issues of today's Internet through route control, failure isolation, and explicit trust information for end-to-end communication.

The central reason we proposed this BoF was to discuss with the RIPE community how RIRs (and potentially LIRs) could be involved in the novel SCION Internet architecture that matured over the recent years and is now ready for wide-spread deployment.

“So, this session is very much about you as well, we’d like to hear your ideas on how we can roll this out on a large scale.”

“What is your experience with SCION?” our BoF secretary, Caspar Schutijser (SIDN Labs), then asked in an initial audience poll to begin the session. 38% answered “I have no experience”, 32% said “I heard about it”, while 27% had either played with it, had practical experience, or even considered themselves an expert. All in all a good mixture of SCION-savvy people and newcomers to have a fruitful discussion.

SCION Overview

If anyone is a SCION expert, then it must be Adrian Perrig, professor at ETH Zurich in the Network Security area. He’s been involved with the design of SCION for the last decade. To give a flavour of the topic that the audience was going to discuss about, he provided a high level overview on SCION at the beginning of the BoF.

“So, essentially SCION is an architecture to provide an alternative to BGP”, Adrian started his presentation. “There are some principles which are quite different from the current Internet.” For example, the SCION architecture follows a stateless packet forwarding approach - i.e., there's no per-flow state in routers and thus no opportunity for inconsistent state.

Also, the routing system is essentially instant-convergence, therefore it is always in a consistent state. In contrast to today's Internet, where all routing decisions are taken by the network nodes, SCION is a path-aware networking approach that gives control to end hosts - i.e., end hosts learn about available network path segments, combine them into end-to-end paths according to their own preferences, and embed the corresponding forwarding information into the packet headers.

In addition, SCION's path awareness directly enables multi-path communication, which can be used to provide high availability and rapid failover in case of network failures. As Adrian said: “Instead of just having 1 or 2 paths, in some cases one has dozens or even over 30 to 50 paths one can select amongst”.

The whole system has been designed from a security perspective and all aspects of the architecture have been designed securely. Adrian elaborated: “We’ve started several years ago with formal security analysis to formally prove the properties of the architecture including the implementation, providing validation of high-level properties all the way down to the code.”

A core concept of SCION are its isolation domains (ISDs), which organise multiple autonomous systems (ASes) into independent routing planes and substantially increase both scalability and security properties of the network. On the one hand, ISDs enable a separation of the routing protocol into an intra-ISD and an inter-ISD process, which reduces the overall complexity, similar to the separation of the Internet into ASes or a division of an AS into areas by existing intra-domain routing protocols (e.g., OSPF or IS-IS). On the other hand, by isolating the routing process within an ISD from all external actions, ISDs limit the effect of misconfigurations and routing attacks. Furthermore, all routing messages are authenticated based on a secure but flexible public-key infrastructure (PKI) in which each ISD can independently define its own roots of trust.

As a result, the SCION architecture provides strong resilience and security properties as an intrinsic consequence of its design.

Fig.1 provides a SCION overview on a single slide. In the control plane, the routing system establishes path segments. The example shows two orange path segments across ISDs, which are established between core ASes, and several paths inside an ISD represented by blue and green paths segments. The control plane discovers these path segments and redistributes them using a pull-based path server infrastructure.

In the data plane, end-hosts fetch path segments and combine them to end-to-end paths. In the example shown, there are two sets of path combinations from a host in AS F to a host in AS S. The combined paths are then embedded in the packet header to send the packet end-to-end. Moreover, all the operations are protected in the control plane with digital signatures, similar to BGPsec, while in the data plane high speed crypto is being used in packet forwarding to validate the correctness of the forwarding information, so that a malicious entity cannot change or use paths that are not allowed by policy.

The SCION architecture provides not only high security for communication, but also higher performance than traditional Internet approaches thanks to multiple paths that are available enabling latency or bandwidth optimisation. Moreover, SCION's symmetric key derivation system enables high-speed packet authentication at routers and firewalls at less than 100ns on commodity hardware.

Of course, an important question is how all this can be deployed. Adrian claims that for ISPs it is relatively straightforward to deploy: “There’s essentially no change to the internal infrastructure of an ISP, or a domain, so SCION packets are encapsulated in whatever local forwarding fabric is used internally, typically IP or MPLS”. Since these internal headers are again removed, SCION is not an overlay architecture on BGP or IP -- it’s really an independent architecture, an “upgrade” to an AS that reuses the internal communication fabric to enable a simple deployment. It is envisioned that end domains will use the same approach, but to enable a smooth transition there is a SCION IP gateway (SIG) which encapsulates standard IP packets into SCION packets. Adrian explained: “The SIG can make use of multi-path and other properties that SCION offers fully transparent to hosts. Of course, additional benefits can be gained if end-hosts are upgraded, but for the time being the majority of deployments are leveraging a SIG.”

Besides SCIONLab -- a global research infrastructure enabling users to set up their own AS for experimentation -- there is already global availability of SCION connectivity with seven ISPs offering native SCION connections to local domains, mostly in Switzerland, but now also in South Korea. Also with SwissIX an IXP is already offering SCION peering. Adrian provided more details on the deploying entities: “Several Swiss banks are already using SCION in production, the first bank for over three years now and thanks to Anapaya Connect, native BGP-free connectivity is now available to a hundred data centres in over 10 countries.”

An introduction to SCION is given in the SIDN Labs Blog. More details can also be found in the SCION book and on the SCION website.

What Role Should RIRs and LIRs Play in SCION?

Next, I introduced the SCION-related questions that we wanted to discuss with the RIPE community in the BoF:

• Should RIRs assist with the establishment of ISDs? We expect 10-100 times more SCION-ASNs than ASes in today’s Internet.
• Should RIRs hand out SCION-AS numbers? How to coordinate with traditional Internet AS numbers? In particular, owners of an Internet AS would often want to obtain multiple SCION-AS numbers for different domains to take advantage of SCION’s features.
• What about AS certificates, should RIRs take on a similar role as in RPKI?
• For current IP prefixes, could RIRs provide certificates that enable the owner to prove ownership? This could be based on RPKI, but require the active use of the private key for authentication / key establishment purposes.

Discussion around these questions was moderated by Victor Reijs (SIDN Labs). It was kicked off by short presentations from several organisations (SWITCH, SIX, AMS-IX and SIDN Labs) who discussed their possible role in a SCION Internet:

Simon Leinen is a network engineer at SWITCH who runs the backbone for the Swiss university network. They are involved in two projects where they use SCION. “Looking at registration,” Simon stated, “it’s maybe like the Internet at the end of the 1980s. Right now we have Anapaya who hands out resources to end networks, much like Jon Postel used to do before the InterNIC and the RIRs were created.” Therefore, Simon hopes that RIRs such as the RIPE NCC would be willing to help this new SCION Internet, even if it may still take a few years until that would actually be needed. After all, RIPE NCC was the first RIR on the Internet as well. “I would be thrilled since you have so much experience with how to get this up and running,” Simon concluded.

Fritz Steinmann is a senior network and network security architect at SIX, a Swiss financial market infrastructure provider. For about 10 years SIX has been a LIR for their clients. Together with the Swiss National Bank, SWITCH and a couple of other ISPs, they built the first SCION industry field implementation for the Swiss financial community called Swiss Secure Finance Network (SSFN), replacing multiple existing MPLS networks. Fritz reported: “The role of RIRs and LIRs was brought forward especially by those parties who do not have their own BGP AS number already and who asked, who would assign SCION AS numbers to them.” And their idea is that RIRs and LIRs should have their role similar to what they do currently for BGP ASNs.

Stavros Konstantaras is a senior network engineer at AMS-IX, an IXP based in Amsterdam. He believes “our role in SCION is similar to our current ones, since IXPs are good at grouping ASNs and provide traffic isolation, and this is what SCION is doing as well.” As neutral and independent organisations, IXPs could also host the SCION beaconing mechanism or help in building and implementing the SCION design.

Joeri de Ruiter is a research engineer at SIDN Labs, the research team of the Dutch CC TLD. Two years ago, they initiated a national research program, called 2STIC, to develop and evaluate mechanisms for increasing security, stability and transparency of Internet communication. In this context, they experiment with and contribute to future or emerging Internet architectures, including SCION. “We’re interested to hear the thoughts from the community around SCION and how to administer it,” Joeri explained.

After those short statements, Victor opened up the discussion. He asked: “Is this something the RIRs and LIRs could take up or do you think it has nothing to do with IP?”

Marco Hogewoning (RIPE NCC) first wanted to know where SCION stands in terms of standardisation, because often the registry function follows from that. Adrian responded that we have been talking to IETF and ITU, but SCION touches so many topics including PKIs, routing, data plane, and so on, that essentially almost every working group at IETF would need to get involved. Therefore, we believe that the fastest way forward is through an independent foundation to create an initial standard, and that’s why we are creating the SCION foundation which will be open for anyone.

The first draft standard will be based on the second book about the SCION architecture, which will be published at the end of 2021. With this, the traditional standardisation organisations will then be approached to see how that standard can be transitioned. Marco replied: “This reminds me about how the Digital Object architecture was and still is handled which is a worrying development for many people.”

Adrian agreed that the aim is not to create a separate community: “We want to be part of the global community”. For instance, SCION was a part of the PANRG BoF at the IETF. But comparing the standardisation process of TLS and IPsec, TLS 1.0 was much quicker as opposed to IPsec which was a decade long standardisation. Therefore, we consider the SCION foundation a good way forward because there is active use already, and so the first standard needs to be described quickly. However, we will also keep working with IETF, ITU and others to see good ways in moving forward.

Daniel Karrenberg (Chief Scientist at RIPE NCC) commented that to do something the traditional way would be to generate interest within RIPE: “There are 84 people listening to this. So you’re well on your way.” He suggested creating a Working Group (WG) and chartering that WG with the question of what activities the RIPE NCC should be doing to help SCION along. The next step is that the WG then recommends the RIPE NCC to do a pilot service. Based on this, the RIPE NCC members will need to be convinced that this is a worthwhile activity to spend money. “And where you standardised the thing, I don’t think the RIPE NCC or the RIPE community cares very much,” Daniel concluded.

Marco then asked what was the main driver for using our own 48-bit address space, instead of simply piggy backing on existing AS numbers which are 32-bits. Sam Hitz (Anapaya, commercial provider of SCION technology) explained that SCION has a 64-bit identifier for the combination of ISD and AS number, where the first 16-bits are used for the ISD number. He expects that there will be many more SCION ASes than in the current Internet, since organisations would potentially apply for a different AS number for every site they operate. The current numbering practice is that the lower 32-bit range is basically reserved for the existing BGP ASNs. For those, the SCION ASN will be the same. But new organisations that don’t have a BGP ASN yet will get a SCION ASN from the range above the 32-bit boundary. And additional SCION ASNs will be allocated from that range as well.

Victor then turned attention to AS certificates: should they be managed and operated exactly the same as the AS numbers themselves? After Adrian confirmed that these certificates prove ownership or the right to use one of those identifiers, Daniel stated: “After we’ve seen the disaster of some other certification trees, I think if the purpose of the certificates is to prove that you’re the registered party for this identifier, then it makes absolute sense to tightly couple the registry with the certificates. Because anything else leads to suboptimal performance.”

Adrian argued that it would be a bit more complicated because, in SCION, there’s also this notion of ISD with respect to sovereignty. Assuming that a nation or a group of ASes wanted to create a sovereign part of the network, they would create their own ISD. Then the AS certificates would need to be signed by roots that are approved by this ISD. While they could of course give RIPE the right to issue these certificates, in some cases they may want to keep that part sovereign. Additionally, an AS can be part of multiple ISDs requiring different signatures. Also, in SCION we assume that the certificates are very short-lived to avoid revocation. The resigning of certificates every few days raises of course additional operational challenges. “We’d love to work with RIPE to pick these details apart, some of which exist because of the architectural properties that we’d like to achieve,” Adrian said.

Finally, Andrei Robachevsky (ISOC) asked about the most common ISD to understand the need for coordination in formation of an ISD. Adrian stated that an ISD is essentially the entity of ASes that can agree on the roots of trust, typically a nation, a group of nations or a group of large-scale providers. In other words, ISDs are formed based on the desire to have a sovereign operation within the group of entities. The creation can then happen in two ways: either a central entity like RIPE creates new ISDs and signs them, or an entity finds neighbouring ISDs that are willing to connect them, and so they’re essentially bootstrapped in a decentralised manner. These two approaches may also co-exist simultaneously and entities can then configure how they trust emerging ISDs.

More details about the first SCION BoF, including a full video recording can be found on the RIPE 81 website.

What Happened Since the First BoF

Based on the momentum created by the first SCION BoF at RIPE 81, we followed-up on the idea of establishing a SCION Working Group at RIPE. This idea was initially discussed with Daniel Karrenberg who kindly introduced us to the specifics of RIPE and encouraged us to propose a SCION working group to discuss the registration of SCION resources and specify a pilot for registry and certification. He also sketched the typical steps of defining a WG charter with associated goals and deliverables, including work items for the planned pilot.

At RIPE 82, we will be holding a second SCION BoF with the aim of discussing the SCION WG proposal with the RIPE community. We already had a brief informal chat about this proposal with the RIPE Chair Team. We expect that members of the community might question whether forming a SCION working group is really the right way forward, or if SCION would better be discussed as a part of the Routing or the Address Policy WG. While it might be intriguing to be part of a larger WG than having our own WG to push the SCION idea forward, we’d like to stress the pilot nature of our initiative, which we believe can be better pushed in our own WG. Also, SCION addresses aspects from both the Address Policy as well as the Routing WG, so it might not fit well in neither one nor the other solely. While those WGs are more focussed on operational issues, the SCION WG also aims to inform the community, which is hardly possible in a 20min slot. Also, SCION AS numbers slightly differ from the Internet AS numbering, since there will be more numbers. This needs to be designed properly, which cannot happen in the scope of operational issues. A dedicated SCION WG will provide the required space to brainstorm on these aspects. In any case, we are committed to coordinate with the Routing and the Address Policy WG as necessary.

We understand that the usual steps are to come along with an implemented draft RFC of the IETF. To this end, our plan is to have a first draft standard ready within a few months (derived from the second SCION book) which we will use to drive the next steps with IETF. Additionally, we have also established the SCION foundation which will play a pivotal role in the SCION standardisation.

To give an insight in today’s practices about registration, issuing numbers, and certificates within SCION, the upcoming BoF will start with two presentations. First, Samuel Hitz (Anapaya) will give a quick introduction to SCION ISD registration, AS numbers, and certificates. Then, Fritz Steinmann (SIX) will outline SCION challenges and risks for certificate issuing and AS number assignment on the example of the Secure Swiss Finance Network (SSFN). The remaining time of the BoF will be reserved to discuss the proposed WG charter and goals for such a working group with the aim to collect feedback and suggestions on the planned deliverables and timeline, as well as the proposed WG organisation (type of meetings, chair selection scheme, etc.). Besides Anapaya and SIX, the discussion will be supported by representatives from SIDN Labs, AMS-IX, NDIX, SWITCH, SURF, ETHZ, and DE-CIX.

More details about the SCION BoF at RIPE 82 can be found on the RIPE 82 website.

What Potential RIPE Involvement Could Look Like

Our aim of establishing a SCION Working Group is to raise awareness in the RIPE community about the role of RIRs and LIRs in the SCION architecture. Therefore, the SCION WG will facilitate discussion about questions related to the registration of SCION resources, such as the establishment of SCION ISDs (Isolation Domains), the assignment of SCION AS numbers, as well as the provision of SCION AS certificates, similar to the role of RIRs in RPKI.

To this end, the goal of the SCION WG is to define, discuss and set up a pilot for the registration and certification of SCION resources. The SCION WG aims to deliver a first version of the pilot specification within one year, which will be implemented in the second year. Based on the collected feedback and experience with the first pilot, a revised version of the specification will be defined and implemented during the third year. Considering the experience with the revised pilot, a permanent implementation of the SCION registry and certification will then be considered during the fourth year.

The SCION WG will hold (plenary) sessions during RIPE meetings twice a year. Additionally, the WG will be holding remote sessions outside of RIPE meetings organised via teleconference and open to everyone. The SCION WG will also monitor and take part in discussions in related working groups, such as the Address Policy Working Group, the Routing Working Group, and the RIPE NCC services Working Group.

More details can be found in our SCION RIPE WG Charter proposal.