Apple has revealed in a recent research paper how it trained its latest generative AI models using Google’s neural-network accelerators, rather than the more popular Nvidia hardware.
The paper, titled “Apple Intelligence Foundation Language Models,”[PDF] dives into the details of how Apple developed its language models, from training to inference. These models, known as neural networks, are responsible for converting user queries into text and images, powering the Apple Intelligence features integrated into Apple’s operating systems. These features include text summarization and suggested wording for messages.
While many AI organizations prefer Nvidia GPUs, such as the H100, Apple chose to use Google’s Tensor Processing Unit (TPU) silicon. This decision is not entirely surprising, given Apple’s long-standing strained relationship with Nvidia. Apple seems to have no interest in mending fences with Nvidia, even for the sake of training its Apple Foundation Models (AFMs).
Apple Ignored AMD’s Radeon GPU Chips
What’s surprising is that Apple didn’t opt for Radeon GPUs from AMD, which has supplied chips for Mac devices in the past. Instead, Apple chose Google’s TPU v4 and TPU v5 processors for training its AFMs. This choice aligns with Google’s recent growth in TPU usage, making it one of the top three in the market alongside Nvidia and Intel.
Apple’s server-side AI model, AFM-server, was trained on 8,192 TPU v4 chips, while AFM-on-device used 2,048 newer TPU v5 processors. For comparison, Nvidia claims that training a GPT-4-class AI model requires around 8,000 H100 GPUs. This suggests that, in Apple’s experience, the TPU v4 is roughly equivalent to Nvidia’s H100 in terms of the number of accelerators needed.
Apple claims its models outperform some from Meta, OpenAI, Anthropic, and even Google itself. However, the research paper doesn’t provide detailed specifications for AFM-server. It does highlight that AFM-on-device has just under three billion parameters and is optimized to have a quantization of less than four bits on average for efficiency.
Apple’s Human Evaluation
Apple prefers human evaluation over standardized benchmarks, believing it aligns better with user experience. In tests, Apple presented real people with responses from different models and asked them to choose the better one. According to Apple, users often preferred its models over competitors’. However, the paper lacks specifics about the prompts and responses, so readers must take Apple’s claims at face value.
Apple’s AFM-on-device model generally scored second or third overall, outperforming models like Gemma 7B, Phi 3 Mini, and Mistral 7B but not LLaMa 3 8B. The paper did not provide comparisons with GPT-4o Mini. AFM-server couldn’t match GPT-4 and LLaMa 3 70B and likely doesn’t fare well against GPT-4o and LLaMa 3.1 405B either.
Apple justifies its performance by highlighting that AFM-on-device outperformed all small models for the summarization tool in Apple Intelligence, despite being the smallest model tested. However, the paper doesn’t show similar data for other tools, leaving gaps in the evaluation.
Apple claims a significant win in generating safe content. According to the paper, AFM-on-device and -server output harmful responses 7.5 percent and 6.3 percent of the time, respectively, while other models did so at least ten percent of the time. Mistral 7N and Mixtral 8x22B were the worst offenders, with harmful response rates of 51.3 percent and 47.5 percent, respectively.
Apple’s Decision to Use Google TPU Chips
The decision to use Google’s TPUs over Nvidia’s GPUs might stem from Apple’s need for efficiency and scalability. TPUs are designed for high-performance machine learning tasks, and Google’s expertise in AI and data centers likely provides Apple with a robust infrastructure for training its models. This partnership also underscores the competitive nature of the AI hardware market, where companies are exploring various options to find the best fit for their needs.
Apple’s choice to use Google’s hardware despite past criticisms highlights a pragmatic approach to technology partnerships. The focus is on leveraging the best available tools to achieve the desired outcomes, even if it means working with a competitor. This decision aligns with Apple’s broader strategy of integrating advanced AI features into its products, enhancing user experience, and maintaining its competitive edge in the market.
The paper’s emphasis on human evaluation over benchmarks suggests that Apple values real-world performance and user satisfaction. By involving real users in the evaluation process, Apple aims to ensure that its AI models deliver practical benefits and intuitive interactions. This approach reflects a user-centric philosophy that prioritizes tangible improvements over theoretical performance metrics.
While the lack of detailed comparisons and the reliance on Apple’s internal testing might raise some skepticism, the overall transparency in revealing the training process and hardware choices provides valuable insights into Apple’s AI development strategy. The focus on safety and minimizing harmful outputs is particularly noteworthy, as it addresses a critical concern in the deployment of AI technologies.
The Future of Apple and Google Chips
Looking ahead, Apple’s collaboration with Google and its use of TPU hardware could pave the way for further advancements in AI capabilities. As AI models become increasingly integrated into everyday devices, ensuring their reliability, safety, and efficiency will be paramount. Apple’s ongoing research and development efforts in this area demonstrate a commitment to pushing the boundaries of what’s possible with AI, ultimately benefiting users with more intelligent and responsive technologies.
In conclusion, Apple’s detailed exploration of its AI training process using Google’s TPUs highlights a strategic and user-focused approach to developing advanced language models. The paper provides a glimpse into the complex decisions and considerations involved in creating AI systems that enhance user experiences while maintaining high standards of safety and efficiency. As the AI landscape continues to evolve, Apple’s insights and innovations will likely play a significant role in shaping the future of intelligent technologies.
