As AI continues to grow in both application and complexity, it places a heavy demand on digital infrastructure. However, while much attention has been placed on the high-performance compute required to power AI models, one area that has yet to receive the focus it deserves is network efficiency: specifically, the role of switching technology. This often-overlooked element is quietly becoming one of the biggest bottlenecks in scaling AI infrastructure.
Data centres, the backbone of AI applications, are rapidly transforming to meet AI’s unique demands. While traditional data centre infrastructure was designed with enterprise IT in mind, the unique, dynamic, and resource-intensive nature of AI workloads requires a completely different approach. At the forefront of this shift is the need for a more energy-efficient, low-latency, and scalable network, with a particular focus on overcoming the limitations of conventional switching technologies.
Moving beyond compute
It is easy to assume that AI simply means more of the same: more compute, more storage, and more power. But AI is fundamentally different from traditional IT workloads. Its demands are more concentrated, and its traffic patterns more dynamic. A single AI training process can require more energy than an entire rack of traditional servers. This means that AI is not just about expanding existing infrastructures; it is about completely rethinking them to account for the intensive compute, network, and cooling requirements that come with these workloads.
AI workloads compress data transfer that would normally take months into mere hours. This results in surges of traffic that push networks to their limits. For AI to be viable at scale, data centres need to handle this burstiness with high responsiveness, moving data between graphics processing units (GPUs) and servers efficiently while reducing power consumption. This is where the network, particularly the switch, plays a critical role in ensuring the infrastructure can handle AI’s specific demands.
The hidden cost of switching
When we think of energy consumption in data centres, we often focus on the power delivered to servers or the cooling systems that prevent overheating. However, one of the most significant contributors to inefficiency is the network switch – an element that is crucial to how data is routed across servers.
Network switches, particularly traditional electronic switches, rely on optoelectronic conversion to route data. This process involves converting data from optical signals to electrical signals and back again, millions of times per second. While essential, this conversion generates considerable power consumption and heat, exacerbating the already significant energy requirements of AI-centric data centres. As AI workloads grow, this energy demand rises, leading to inefficiencies that impede performance and increase operational costs.
As more data centres begin to handle large-scale AI workloads, traditional switching technology is struggling to keep up. The energy consumption and latency associated with optoelectronic conversion are no longer sustainable. Therefore, rethinking switching technology is vital for scaling AI while mitigating its environmental and operational impact.
Exploring the emerging role of photonic switching
One of the most promising solutions to this challenge is the development of photonic switching technology. Unlike traditional electronic switches, which rely on optoelectronic conversion, photonic switches route data entirely within the optical domain. By removing the need to convert light to electricity and back again, photonic switches significantly reduce energy consumption and latency. Finchetto, a pioneering UK-based startup, is leading the charge in photonic switching with a fully passive, optical switch that is revolutionising how networks handle AI workloads. Its innovative optical switch reduces energy consumption by up to 53 times compared to traditional electronic switches. This breakthrough not only lowers energy costs but also delivers port-to-port latency as low as 40 nanoseconds, making it ideal for high-performance AI applications.
Traditional electronic switches were never designed for the scale and complexity of AI workloads. By using light to control light, a more efficient and scalable network can be created to meet the demands of modern AI applications.
The power-cooling-switching nexus
A successful AI infrastructure cannot rely on one innovation alone. Power, cooling, and switching must evolve in tandem to meet the ever-growing demands of AI workloads. A more efficient switch reduces energy consumption, which in turn reduces the strain on cooling systems. This cascading effect allows data centres to operate more efficiently and accommodate denser, more flexible compute environments. A holistic approach is necessary; one that enables every layer of the infrastructure to be optimised for efficiency across the entire system.
Cooling technology also plays a critical role in AI infrastructure. Techniques such as direct-to-chip cooling and immersion cooling are becoming increasingly popular, allowing for more precise and efficient thermal management. Moreover, integrating heat reuse systems that channel excess energy into nearby districts, industrial sites, and greenhouses can further improve overall site efficiency and reduce emissions.
The integration of these technologies, including advancements in network switching, is essential for creating AI-ready data centres that are both sustainable and scalable. However, this approach requires a collaborative effort among power, cooling, and networking teams to ensure that every design choice supports efficiency across the entire system.
Hybrid and distributed solutions – innovating for the future
As AI workloads continue to evolve, so too must the infrastructure that supports them. While large, centralised compute clusters are still a critical part of the landscape, there is a growing trend towards hybrid and edge computing solutions. These distributed architectures offer the flexibility to process data closer to the source, reducing latency and improving performance. To support this shift, data centre designs must be modular and adaptable, capable of scaling to meet the unique demands of AI workloads.
Modular designs, such as those featuring flexible rack densities and scalable power and cooling systems, are gaining traction. These data centres can quickly adapt to changes in AI workload requirements, ensuring that resources are always allocated efficiently. Moreover, these facilities are built with future-proofing in mind, ensuring that they can evolve as AI and related technologies continue to develop.
Preparing for the AI boom
Ultimately, the next generation of data centres will not only be defined by their size or location but by their ability to adapt to the evolving demands of AI. To succeed, operators must take a holistic approach to infrastructure design, integrating power, cooling, and network innovations to create systems that are both efficient and scalable.
Innovative photonic switching is just one example of how the industry is addressing the challenges of AI infrastructure. By embracing new technologies and adopting integrated, modular designs, data centres can create the foundation needed to scale AI responsibly. The future of AI infrastructure lies in adaptability – treating energy efficiency as a priority, breaking down silos between teams, and adopting technologies that will allow data centres to meet the demands of tomorrow.
To stay competitive and sustainable, data centre operators must look beyond traditional technologies and embrace innovations like photonic switching.
