A100: the $10.000 Nvidia chip

The technology industry is experiencing a gold rush as software capable of creating text passages and images that resemble human creations emerges.

Heavyweights such as Microsoft and Google are competing to incorporate cutting-edge AI into their search engines, while billion-dollar competitors like OpenAI and Stable Diffusion race to release their software to the public.

To power these applications, one of the most critical tools in the artificial intelligence industry has become the Nvidia A100, a chip valued at around $10,000.
According to investor and AI industry commentator Nathan Benaich, the A100 has become the “workhorse” for AI professionals. Nvidia holds 95% of the market share for graphics processors used for machine learning, according to New Street Research.

The A100 chip is particularly well-suited for machine learning models, such as those used by ChatGPT, Bing AI, and Stable Diffusion. It has the ability to perform multiple simple calculations simultaneously, a critical feature for training and using neural network models.

Initially designed for rendering 3D graphics in games, the technology behind the A100 has evolved to meet the demands of machine learning tasks. It now runs in data centers rather than in gaming PCs.

Companies working on software such as chatbots and image generators require hundreds or even thousands of A100 chips, either purchasing them directly or gaining access to them through cloud providers. Large language models, for instance, require hundreds of GPUs for training, with the chips needing to process terabytes of data quickly to recognize patterns.

Access to large numbers of A100s is seen as a sign of progress by some entrepreneurs in the field. Stability AI CEO Emad Mostaque recently tweeted, “A year ago we had 32 A100s…now, dream big and stack moar GPUs kids. Brrr.”