An AMS-IX Story
February 21, 2018
How long does it take for an organisation to expand the capacity of the backbone links in order to respond to fast bandwidth growth? How much work is involved to complete such a task? In this blog, we will answer these questions and explain why our latest backbone upgrade in the Amsterdam metro area was such an achievement and will remain an important point of reference for the future.
Let’s assume that you are an organisation with a presence in two colocations in the same city (Data Centre A and Data Centre B). In each colocation, there is a switch/router installed serving as a local gateway. Now let’s assume that you are willing to interconnect the two routers by utilising a single dark fibre. In order to achieve that, the following steps need to be taken:
1/ Find a supplier who can provide a fibre between the two colocations and place an order
2/ Request a cross connect at Data Centre A between your equipment and the MMR
3/ Request a cross connect at Data Centre B between your equipment and the MMR
4/ Test the new fibre connection to ensure that it is free of errors and the attenuation is within acceptable limits. Otherwise contact the fibre provider or the colocation for troubleshooting
As we can see, multiple parties and people need to be involved to complete this “simple” task. Not to mention the delivery time of the new fibre connection: 20 to 30 working days for the dark fibre to be placed between the two colocations and about 5 working days for every cross connect to be delivered. Assuming that the new connection is free of errors and that the two cross connects are ordered at the same time, the best-case scenario would be around five weeks. But realistically, it takes approximately 12 weeks for the new connections to be established, even longer sometimes.
Now if you want to expand the capacity between the two colocations to respond to high demand or unexpected growth, you will once again have to pay the full price for an extra connection and wait at least five weeks before this new connection is delivered.
At AMS-IX, we managed to reduce the expansion time from 5-12 weeks to only 4 hours, establishing a remarkable record and setting up new standards for the development of the IXP industry. This milestone was achieved on February 2nd, 2018, by utilising the capabilities of our DCI solution which we introduced in the majority of our PoPs in Amsterdam. The DWDM technology that is implemented in our DCIs allows engineers to transmit multiple wavelengths over the same physical dark fibre. More details on that solution can be found here.
Our objective was to double the backbone capacity of our newly installed Extreme SLX 9850 (formerly known as Brocade), from 800Gbps to 1600Gbps, in order to provide resilience and stability in case of a failure. This new chassis is located at NIKHEF Data Centre in Amsterdam Science park and had one LAG of 2x100GBps connection to each of our four core sites. In order to have 4x100GBps to every core site, we had to perform the following steps:
1/ Install a new Huawei OSN902 in the rack with the appropriate optics.
2/ Make the correct cabling between the SLX, the OSN902 and the MUX. That part needs to be completed in the core sites as well.
3/ Turn on the ports and confirm that the signal is transmitted/received within acceptable limits.
4/ Test for possible errors/flaps or any other issue.5/ Add the new ports into the existing LAGs
And here we are! Including the documentation work, we managed to complete all the steps in 4 hours. The “plug and play” configuration of the OSN makes it very easy for the field engineers to start transmitting a new wavelength over the existing fibre. At the same time, the port-channel concept of the new SLX makes the aggregation of the links a very simple task without any interruptions and without the need to remove traffic from the existing LAG beforehand.
In sum, the new hardware solutions that we deploy across our 14 Amsterdam PoPs, using the latest technologies, bring significant advantages on saving time and lowering recurring OPEX. Our past experiences led us to enrich our network with a “plug-and-play” feature, a feature that allows us to “grow-on-demand” and respond to the future capacity needs within hours.
In this joint effort of engineering and NOC, we broke some records! Special thanks to Tim who did all the preparations, to Winston and Mohammed who installed the equipment and handled the cabling at the Data Centres and finally to Kostas who helped me configure the network equipment.
Photo (from left to right): Stavros, Tim, Winston, Mohammed and Kostas.