In a recent paper published in the energy journal Joule, it was revealed that if Google were to implement generative AI in every search, its annual electricity consumption would soar by around 29 billion kilowatt-hours. This is roughly equivalent to the total yearly electricity usage of the entire population of Iceland. This statistic underscores a significant challenge facing AI in the future - the substantial energy demand for training and operating AI models, a demand that is expected to grow.
Companies are beginning to recognize the opportunity to address AI's energy problem. One such company, University College London (UCL) spinout Oriole Networks, has recently secured a £10 million seed funding round. Investors include XTX Ventures, the venture arm of algorithmic trading firm XTX Clean Growth Fund; Dorilton Ventures; and the UCL Technology Fund.
The training of AI models heavily relies on graphic processing units (GPUs) such as those manufactured by Nvidia. For instance, large language models (LLMs), which power ChatGPT, require the clustering of thousands of GPUs. XTX, for example, possesses a supercomputer consisting of 20,000 GPUs.
At present, GPUs are typically linked via ethernet cables, which have become a bottleneck in these clusters. These cables can slow down the computational power of the cluster and consume a substantial amount of, around 20% of the total consumption of the overall AI cluster.
Oriole's solution, stemming from the work of Professor George Zervas at UCL, involves connecting GPUs using light beams transmitted through optical fibers. According to Oriole, this technology can boost the speed of information packets traveling between GPUs by up to 100 times. Furthermore, optical networks use significantly less energy than traditional ethernet networks, reducing the network's energy consumption to 2-3% of a traditional system on a lab scale.
The company has obtained the necessary technology from UCL Business, covering the physical of the optical system and machine learning that enables its functionality. Oriole plans to outsource the manufacturing of the hardware to companies already producing network infrastructure, with the intention of selling the networking system to customers replace ethernet cables within their computing power setup.
Despite the estimated couple of years required for product development, Oriole is poised to commercialize its tech and capitalize on the growing demand for energy-efficient AI infrastructure.
