Grok-1 Architecture Open-Sourced for General Release by xAI

Elon Musk’s xAI has released its Grok chatbot and open-sourced part of the underlying Grok-1 model architecture for any developer or entrepreneur to use for purposes including commercial applications. Musk unveiled Grok in November and announced that it would be publicly released this month. The chatbot itself is available to X social premium members, who can ask the cheeky AI questions and get answers with a snarky attitude inspired by “The Hitchhiker’s Guide to the Galaxy” sci-fi novel. The training for Grok’s foundation LLM is said to include X social posts.

“Grok-1 is a 314 billion parameter ‘Mixture-of-Experts’ model trained from scratch by xAI,” SiliconANGLE reports, explaining that “a Mixture-of-Experts model is a machine learning approach that combines the outputs of multiple specialized sub-models, also known as experts, to make a final prediction, optimizing for diverse tasks or data subsets by leveraging the expertise of each individual model.”

Grok’s open-source release “does not include the full corpus of its training data,” VentureBeat notes, writing that “this doesn’t really matter for using the model, since it has already been trained, but it does not allow for users to see what it learned from.”

An xAI blog post states the “base model trained on a large amount of text data, not fine-tuned for any particular task.”

“Grok was open sourced under an Apache License 2.0, which enables commercial use, modifications, and distribution, though it cannot be trademarked and there is no liability or warranty that users receive with it,” according to VentureBeat, which notes that the license requires licensees to “reproduce the original license and copyright notice, and state the changes they’ve made.”

The New York Times writes that open-sourcing the Grok -1 code “is the latest volley between Mr. Musk and ChatGPT’s creator, OpenAI, which the mercurial billionaire sued” on March 1, over what Reuters says he alleged was “abandoning its original mission for a profit.”

Musk was a co-founder of OpenAI who departed when the company decided to sell a large stake to Microsoft. Musk at the time professed the view that “such an important technology should not be controlled solely by tech giants like Google and Microsoft, which is a close partner of OpenAI,” NYT says.

The beta version of Grok released in November was proprietary. Those interested in using Grok’s open-source code can download it from xAI on GitHub, or via Academic Torrents. “Hugging Face also added a fast download instance,” per VentureBeat.

No Comments Yet

You can be the first to comment!

Leave a comment

You must be logged in to post a comment.