It is now the fourth time in a row that I had the chance to be part of the Cisco NOC team for Cisco Live EMEA.
If we go even further back in time, I had the chance to go to Cisco Live for the Technical Design Clinics back in London and Berlin. The pressure was on the shoulders of the NOC team who had to deliver a working Wi-Fi network with so many random client devices connected. I did not envy their position (although I admired it). I particularly remember a bug from smartphone vendors in Cisco Live London that was repeating the event SSID as a personal hotspot, causing a lot of trouble to other client connectivity. This was the year the CiscoLive SSID went from fully open to a pre-shared key SSID to prevent that type of problem.
End of 2017, the NOC team invited me to be part of the Wireless Controller team for Cisco Live Barcelona 2018. I accepted quickly mostly for the sake of being part of the Cisco Live event, which I consider a privilege. I discovered since then how setting up a large events network is such a unique endeavor and will try to give some insights into certain choices and decisions.
The Planning
Around summer the year before the event, the first meetings start. We set up a team and make sure we have the best people for the job at every position. This is the responsibility of Remco Kamerman, the Cisco Live NOC team lead and pretty much the only fixed team member since he recruits the rest of us. Some people from the software engineering teams, some salespeople, and some CX people (TAC, Customer Success, and Professional Services): team members are not picked for their job role but for their expertise. If you are one of the top people in your technology, chances are that you already know a good part of the NOC team for having worked with them throughout the year since they are the top people too.
Mapping Madness
We receive the venue plans and event blueprints early on but they keep changing until the very last day (less and less as time goes by of course). This is the challenge of the design folks in the team (Professional Services and System Engineers mostly) who have to do a wireless design mostly by looking at regularly changing plans. A few site visits were organized to get a feeling of the venue. I was there on the first day the building team started building for the event and can testify that the number of physical changes the venue goes through in just a couple of days is unthinkable if you are not used to such events.
Maps are an important part of managing a wireless network. We could leverage the interoperability between the venue maps on the RAI Prime Infrastructure appliance, the Cisco DNA Center we used for the event, and the Ekahau design software we used for the design. Maps were cross-imported between those 3 places so that we could have the proper maps for design and day-to-day management.
Keynote Design
A specific challenge was the keynote area which consisted of 4500 chairs around a central stage in an empty hall. 50 9104 stadium antennas were used to provide coverage from the trusses. Mounting those APs/antennas required very close collaboration with the keynote area build team as there are specific moments where the truss is down and accessible and then brought up (after which you need a scissor lift to access it and you want to avoid that as much as possible for efficiency)
The Build Up
The majority of the NOC team consists of people actually physically building up the network. That requires deploying hundreds of switches throughout the venue and the cabling that goes with that without anything visible to the naked eye. It also requires deploying hundreds of wireless access points in various places. They can be on poles, walls, or ceilings, and mounting elegantly and efficiently becomes an art.
Figure 1: Mounting APs and antennas on the structure
Similar to the Fira Barcelona, we inherited around 400 Wi-Fi access points from the RAI Amsterdam venue. They were nice enough to let us control their access points for the duration of the event. This way, we don’t have to deal with two separate wireless networks. A good part of the venue APs were Cisco 9120s with directional antennas mounted on the very high ceiling (as well as some 9104s in one Hall) which are perfect for providing general coverage.
Indeed the RAI hosts a lot of different shows that have nothing in common (Cisco Live was between a horse show and a pregnancy-related show) and their Wi-Fi network needs to stay stable between events. However, since we are Cisco and we are willing to deploy a network just for our own event, we could add access points at the ground level and be better oriented for specific applications (in general, the close the AP is to the clients, the better, if you can afford it). We knew the high-density areas and more complicated ground areas where additional coverage would be welcome and that’s what our design consisted of.
Figure 2: 9104 stadium antennas mounted on a truss that will go up in the Keynote area
Event Wi-Fi Choices
Historically, the main SSID is WPA2 PSK SSID and the organization prints the key on the event badge everyone wears. We added EduRoam support for our education customers to have an SSID their device already knows and can connect to, using their education credentials. We also added OpenRoaming, where your device automatically connects to the Wi-Fi as soon as you enter the venue if you already had an OpenRoaming profile installed on your device. If you didn’t you can install one from the CiscoLive event app. Personally, I installed an OpenRoaming profile on my iPhone after my local supermarket created a profile for me from their app. My phone automatically connected, in a secure and transparent manner, to the venue as soon as I arrived with my profile from my local supermarket thanks to the RAI also having an OpenRoaming SSID even before Cisco arrived onsite.
We definitely wanted to keep the number of SSIDs offered as low as possible to avoid confusion and to keep the wifi network efficiency to the maximum possible, but the convenience (and the security!) of OpenRoaming and Eduroam convinced us to offer those as extra services.
Wi-Fi 6E
This year, we wanted to offer 6ghz Wi-Fi as 6E is the newest coolest thing. The difficulty is that providing this across the whole event would have meant purchasing hundreds of 9166 access points. This is not possible as we prioritize customer deliveries for the first time on a new device. It would also have meant replacing all the venue APs which is impractical for us. We then covered the entire Meeting Village hall with the 40 9166 we had. The challenge with this hybrid approach is that Wi-Fi 6E requires WPA3 and we did not want to make the main SSID WPA3 yet.
Even if the CiscoLive population is typically nerdy (it’s a compliment nowadays I think) and well equipped, you wouldn’t believe some of the older devices that connect to the network and WPA3 support is just not at 100% yet we believe. We had to create a separate WPA3 SSID which was broadcasted both in 5Ghz and 6Ghz (but 6ghz being only available in the Meeting Village) for compatibility reasons.
Legacy and “Bells and Whistles” SSIDs
As a general rule, is good practice to have some kind of legacy SSID and some kind of more performing SSIDs with more bells and whistles. Some years ago, it meant we provided a Cisco Live Legacy SSID which existed on 2.4ghz, while the 5Ghz was the main and “cool” SSID.
In Cisco Live 2023, we completely gave up on 2.4ghz and the CiscoLive SSID was only available on 5Ghz. This meant the main CiscoLive SSID needed to have the most compatible settings to ensure all the clients could connect and that meant giving up on some great Cisco features (like Device Analytics) for the sake of maximum compatibility. I predict that very soon, the WPA3/6Ghz SSID will become the main SSID and the 5Ghz-only/WPA2 SSID will be the legacy one. Maybe too early for that to happen next year but why not 2025?
How the Event Went
Keynote and 6ghz
The event went very well overall. During the keynote or the party, throughput tests returned surprisingly good results. The 9104 antennas were really surprised by their well-defined coverage area with very small leakage outside of the coverage direction. This really helps with channel reuse in a large venue hall.
It was a good surprise to see more than 60% of the Wireless clients using Wi-Fi 6. However, only a few dozen supported 6E. We expect a sharp increase by next year, but it will stay a minority of clients. There were a couple of 802.11n clients but really not many.
The top simultaneous client count was around 13 500. It is slightly lower than the last event in Barcelona. We expect the event to grow by next year since this was the first one post-Covid.
Figure 3: Our custom telemetry graph
Hardware and Software Considerations
It was the first Cisco Live we ran 100% on the Catalyst 9800 in EMEA and 100% on Cisco DNA Center. Indeed in 2020, they were there but we still had 8540 WLCs in the network. We ran the 17.9.2 CCO software and only had minor issues to report. As is becoming more and more commonplace, most of the time we spent troubleshooting was on interoperability issues with specific device types and features. Completely disabling 2.4Ghz was a great idea because we noticed an increased usage of Bluetooth among the attendees and the Wi-Fi network would have disturbed all those Bluetooth devices.
Not everything was perfect though, it can never be in such a large event with so many new technologies. But I’m glad we keep improving year after year. There are always areas of complaint when the client density is higher than what we anticipated: there were some very successful sessions in Devnet theater or World of Solutions and connectivity was subpar during those events. We’ll make sure to come up with an improvement plan for next year to make that better.
Source: cisco.com
Posts
0 comments:
Post a Comment