Creating your own AI factory: an enterprise guide

Earlier this year, NVIDIA CEO Jensen Huang brought new terminology to the forefront of the ever-evolving AI conversation by referring to the growth of AI factories and foundries in his keynote address at the GTC Conference in March. Typically associated with the development of products and transformation of raw materials, Huang’s addition of ‘AI’ to these words presented a new concept: a modern means of product development and innovation, representing the potential to revolutionise software development, resource management, and overall business operations. Businesses already implementing and looking to implement AI into their operations should take note of this new approach to growing business value.

Setting your business up to enable GenAI enhancement

GenAI is fast becoming a key productivity tool for many organisations. In fact, when focused on customer service, software and development, and creative and knowledge-based work, GenAI could enable more than $1 trillion in annual US productivity gains by 2026, according to a 2023 report from IDC. Many of us already see this technology making huge strides in the customer service space, thanks to its ability to mimic human-like interactions, providing speedy and personalised support and allowing businesses to interact with their customers in real time.

GenAI can hugely enhance the productivity of software engineers too, enabling them to complete coding tasks up to twice as fast, write new code in nearly half the time, and optimize, refactor, or find defects in existing code in nearly two-thirds the time, according to McKinsey. Furthermore, enterprises are also beginning to incorporate AI and machine learning (ML) into their software, carrying through GenAI’s ability to enable better decision-making by leveraging a dynamic comprehension of customers and use cases, rather than using problem-solving shortcuts.

For businesses looking to implement more AI and ML-enhanced software, having the right IT infrastructure that is properly set up to withstand this technology is crucial. This infrastructure must be extremely powerful and enable a high level of flexibility in order to keep up with enterprises’ expanding use of GenAI and the enhancements it brings. In this highly digitised world, it's never been more important for organisations to upgrade and modernise their infrastructure to meet these demands – something that can be achieved with the support of trusted partners.

Collaborating for AI success

It's no easy task to power AI and GenAI within your business. To harness the power of these technologies, companies require essential underlying hardware and software components, leading to a need for tightly integrated processes across each step of product lifecycle, as well as within overall business operations.

Another consideration for enterprises is that they and their partners must ensure governance and compliance. This goes for all relevant requirements and requires the enforcement of best practices which must align with the company’s AI deployment model. Compliance will be needed in areas such as the selection of materials used, the manufacturing processes, software design, and solution delivery and transportation. This is particularly important when it comes to GenAI, which demands substantial compute and storage resources and, if not properly managed, can result in sky-rocketing compute expenses, increased power consumption, and higher carbon emissions.

An inescapable truth of implementing GenAI applications is the amount of power required to do so. AI foundries and factories that enable these applications will demand a significant amount of compute, storage, and interconnected networks to link hefty datasets to the compute infrastructure needed to both develop and maintain these models. Organisations must also choose optimised avenues for the timely delivery of services, all while keeping the need to lower carbon emissions and gas consumption in mind.

Understanding how to tackle GenAI

When considering AI as a workload or suite of workloads, it’s essential to recognise that GenAI introduces different workloads compared to traditional IT scenarios. To successfully navigate the GenAI landscape, businesses must be able to adapt their infrastructure strategies to accommodate these new workloads, which can be a challenging task.

Significant portions of this infrastructure will need to reside in the cloud, but on-premises systems also play a crucial part. To maximise efficiency, it’s essential for businesses to carefully choose and construct the right systems for both cloud and on-premises environments, which can be enabled by collaborating with partners who have the right expertise in deploying and managing this mission-critical infrastructure. These partners are highly important to help organisations achieve the best possible results from their GenAI initiatives, both now and in the future.

Organisations should also keep in mind that optimising GenAI is a gradual, iterative journey, and not one that comes with instant solutions. In order to get there, they should look to streamlined infrastructure and automation solutions, and collaborate with partners who can assist them across the entire spectrum of the process. This can range from data preparation (including pruning, to remove any unnecessary data, and safeguarding sensitive information such as personally identifiable information); to consolidating data using adaptable, scalable, and cost-efficient storage, to AI model training and inference, which is when the requirements on infrastructure are very different are require low-latency processing; to model training which is extremely compute intensive. Businesses should prioritise building relationships with trusted partners that have extensive experience in their specific business domain, as well as the data-centric workflows they will encounter.

What's key to remember is that all AI and GenAI applications start with data. That's why it’s so crucial that organisations use the right dataset – meaning the most relevant and complete – as well as being set up with the right data infrastructure, enabling them to keep their data safe and accessible for whenever they need it. With these considerations in mind, enterprises can start the journey of building their own AI factory or foundry, alongside the support of the right software to optimise operations, make business offerings more unique, and ultimately drive business value and growth.