September 19, 2024: Microsoft Unveils GRIN-MoE AI Model - Microsoft introduces the GRIN-MoE (Gradient-Informed Mixture-of-Experts) AI model, excelling in complex tasks like coding and mathematics while optimizing computational efficiency. Scoring high on benchmarks such as MMLU and HumanEval, it outperforms models like GPT-3.5-turbo. GRIN-MoEs innovative SparseMixer-v2 approach improves task performance by activating only 6.6 billion of its 163.8 billion parameters. Ideal for enterprises, it avoids common large model limitations. However, it underperforms in multilingual and conversational AI tasks.
