Nvidia, Intel and ARM Publish New FP8 AI Interchange Format

Nvidia, Intel and ARM have published a draft specification for a common AI interchange format aimed at faster and more efficient system development. The proposed “8-bit floating point” standard, known as FP8, will potentially accelerate both training and operating the systems by reducing memory usage and optimizing interconnect bandwidth. The lower precision number format is a key factor in driving efficiency. Transformer networks, in particular, benefit from an 8-bit floating point precision, and having a common interchange format should facilitate interoperability advances for both hardware and software platforms. Continue reading Nvidia, Intel and ARM Publish New FP8 AI Interchange Format

Humanloop Raises $2.6 Million as Interest in NLP Tech Grows

Interest in natural language processing (NLP) as an AI training tool is exploding, with analysts predicting a bumper crop of new startups. One such startup, Humanloop, is already gaining attention, having just pulled in $2.6 million in seed funding led by Index Ventures with participation by Y Combinator, LocalGlobe and AlbionVC. Founded in 2020 by computer scientists from the University of Cambridge with alumni from Google and Amazon, the company says its technology makes it “significantly” easier for companies to leverage NLP that helps humans “teach” AI algorithms. Continue reading Humanloop Raises $2.6 Million as Interest in NLP Tech Grows

Nvidia Turbo Charges NeMo Megatron Large Training Model

Nvidia has issued a software update for its formidable NeMo Megatron giant language training model, increasing efficiency and speed. Barely a year since Nvidia unveiled Megatron, this latest improvement further leverages the transformer engine architecture that has become synonymous with deep learning since Google introduced the concept in 2017. New features result in what Nvidia says is a 5x reduction in memory requirements and up to a 30 percent gain in speed for models as large as 1 trillion parameters, making NeMo Megatron better at handling transformer tasks across the entire stack. Continue reading Nvidia Turbo Charges NeMo Megatron Large Training Model

Nvidia Touts New H100 GPU and Grace CPU Superchip for AI

Nvidia has begun previewing its latest H100 Tensor Core GPU, promising “an order-of-magnitude performance leap for large-scale AI and HPC” over previous iterations, according to the company. Nvidia founder and CEO Jensen Huang announced the Hopper earlier this year, and IT professionals’ website ServeTheHome recently had a chance to see a H100 SXM5 module demonstrated. Consuming up to 700W in an effort to deliver 60 FP64 Tensor teraflops, the module — which features 80 billion transistors and has 8448/16896 FP64/FP32 cores in addition to 538 Tensor cores — is described as “monstrous” in the best way. Continue reading Nvidia Touts New H100 GPU and Grace CPU Superchip for AI

Advances by OpenAI and DeepMind Boost AI Language Skills

Advances in language comprehension for artificial intelligence are issuing from San Francisco’s OpenAI and London-based DeepMind. OpenAI, which has been working on large language models, says it now lets customers fine-tune its GPT-3 models using their own custom data, while the Alphabet-owned DeepMind is talking-up Gopher, a 280-billion parameter deep-learning language model that has scored impressively on tests. Sophisticated language models have the ability to comprehend natural language, as well as predict and generate text, requirements for creating advanced AI systems that can dispense information and advice or that are required to follow instructions. Continue reading Advances by OpenAI and DeepMind Boost AI Language Skills

AWS re:Invent Showcases Sizzling Chips, New Tools for Cars

The 10th Amazon Web Services re:Invent cloud computing conference showcased faster chips, better developer tools, smarter AI and two new automotive initiatives. AWS CEO Adam Selipsky’s keynote led with the company’s next-generation processor, the Arm-based Graviton3, and culminated with a peek under the hood at AWS Automotive and AWS IoT FleetWise. Collecting data and spotting trends are enterprise priorities, and AWS is doing its part to advance artificial intelligence and machine learning across that matrix in the cloud. “We know your data is on a journey — and all the stops on this journey matter,” Selipsky said. Continue reading AWS re:Invent Showcases Sizzling Chips, New Tools for Cars

Microsoft and Nvidia Debut World’s Largest Language Model

Microsoft and Nvidia have trained what they describe as the most powerful AI-driven language model to date, the Megatron-Turing Natural Language Generation model (MT-NLG), which has “set the new standard for large-scale language models in both model scale and quality,” the firms say. As the successor to the companies’ Turing NLG 17B and Megatron-LM, the new MT-NLG has 530 billion parameters, or “3x the number of parameters compared to the existing largest model of this type” and demonstrates unmatched accuracy in a broad set of natural language tasks. Continue reading Microsoft and Nvidia Debut World’s Largest Language Model

Cerebras Chip Tech to Advance Neural Networks, AI Models

Deep learning requires a complicated neural network composed of computers wired together into clusters at data centers, with cross-chip communication using a lot of energy and slowing down the process. Cerebras has a different approach. Instead of making chips by printing dozens of them onto a large silicon wafer and then cutting them out and wiring them to each other, it is making the largest computer chip in the world, the size of a dinner plate. Texas Instruments tried this approach in the 1960s but ran into problems. Continue reading Cerebras Chip Tech to Advance Neural Networks, AI Models

Google AI-Enabled MUM Aims to Reinvent, Empower Search

Google revealed its work on a new AI-enabled Internet search tool dubbed MUM (Multitask Unified Model), which can “read” the nuances globally of human language. The company says that users will be able to find information more readily and be able to ask abstract questions. MUM is not yet publicly available but Google independently used it for a COVID-19 related project. Vice president of search Pandu Nayak and a colleague designed an “experience” that gave in-depth information on vaccines when users searched for them. Continue reading Google AI-Enabled MUM Aims to Reinvent, Empower Search

Facebook F8 Event Highlights Tools for Developer Community

At Facebook’s annual F8 developer conference, chief executive Mark Zuckerberg stated that the company would “refocus” on the developer community by spotlighting technologies that “enable developers and businesses to build and grow” on its platforms. The company announced, for example, that the Messenger API for Instagram is now available to all developers. It’s also adding third-party tools to its Facebook Business Suite, which was launched last year. Going forward, PyTorch will be Facebook’s default AI platform.

Continue reading Facebook F8 Event Highlights Tools for Developer Community

IBM Project CodeNet Employs AI Tools to Program Software

IBM’s AI research unit debuted Project CodeNet, a dataset to develop machine learning models for software programming. The name is a take-off on ImageNet, the influential dataset of photos that pushed the development of computer vision and deep learning. Creating “AI for code” systems has been challenging since software developers are constantly discovering new problems and exploring different solutions. IBM researchers have taken that into consideration in developing a multi-purpose dataset for Project CodeNet. Continue reading IBM Project CodeNet Employs AI Tools to Program Software

Microsoft to Buy AI and Speech Recognition Provider Nuance

Microsoft is on track to acquire Nuance Communications, an AI and speech recognition software company, for about $16 billion. The company intends to expand its offerings in medical computing; Nuance already has speech and text data related to healthcare, an established customer base and the transcription tool Dragon. According to Microsoft, the purchase will “double the size of the healthcare market where it competed to almost $500 billion.” With the purchase, Microsoft could also develop advanced AI solutions for the workplace across numerous industries. Microsoft’s last big purchase was LinkedIn, for $26.2 billion in 2015.  Continue reading Microsoft to Buy AI and Speech Recognition Provider Nuance

Virtual Event: GPT-3 and Its Implications for the M&E Industry

To fully examine the inner workings and potential impact of deep learning language model GPT-3 on media, ETC’s project on AI & Neuroscience in Media is hosting a virtual event on November 10 from 11:00 am to 12:15 pm. RSVP here to join moderator Yves Bergquist of ETC@USC and presenter Dr. Mark Riedl of Georgia Tech as they present, “Machines That Can Write: A Deep Look at GPT-3 and its Implications for the Industry.” The launch last June of OpenAI’s GPT-3, a language model that uses deep learning to generate human-like text, has raised many questions in the creative community and the world at large.  Continue reading Virtual Event: GPT-3 and Its Implications for the M&E Industry

Nvidia and University of Florida Partner on AI Supercomputer

The University of Florida (UF) and Nvidia joined forces to enhance the former’s HiPerGator supercomputer with DGX SuperPOD architecture. Set to go online by early 2021, HiPerGator will deliver 700 petaflops (one quadrillion floating-point operations per second), making it the fastest academic AI supercomputer. UF and Nvidia said the HiPerGator will enable the application of AI to a range of studies, including “rising seas, aging populations, data security, personalized medicine, urban transportation and food insecurity.” Continue reading Nvidia and University of Florida Partner on AI Supercomputer

National Research Cloud Gains Big Tech, Legislator Support

The National Research Cloud, which has bipartisan support in Congress, gained approval of several universities, including Stanford, Carnegie Mellon and Ohio State, and participation of Big Tech companies Amazon, Google and IBM. The project would give academics access to a tech companies’ cloud data centers and public data sets, encouraging growth in AI research. Although the Trump administration has cut funding to other kinds of research, it has proposed doubling its spending on AI by 2022. Continue reading National Research Cloud Gains Big Tech, Legislator Support