Skip to content

iOS as Acceleration

🕒 Published (v1): 2025-12-19 13:30 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This proof-of-concept explores using consumer iOS phones as distributed pipeline-parallel compute workers to accelerate ML on modest desktop hardware. Beyond training/inference speedups, the paper demonstrates that an iOS device can serve as an asynchronous tool-execution worker for agentic LRMs, offloading expensive tool calls so the host model continues reasoning without idle waiting.

Problem

Small research labs and individuals often lack sufficient compute for practical ML training and inference, yet cannot rely on cloud computing due to data privacy, physical environment constraints, or cost. Existing mobile compute potential is underutilized because memory constraints and OS sandboxing prevent mobile devices from running large models independently.

Method

The system connects an iOS device to a host computer via USB (TCP socket using iproxy + Swift Network framework) and implements a two-stage pipeline-parallel training loop. On the mobile side, ML operations are manually translated from PyTorch into Apple's Metal Performance Shaders Graph (MPSGraph), a static-graph framework. Because MPSGraph cannot separate forward and backward passes, a hybrid GPipe/1F1B pipeline is used. For the agentic use case, the iOS device exposes a vector-database search tool (dot-product search over 100k AG News embeddings via all-MiniLM-L6-v2) through a split-tool interface: begin_search (non-blocking, queues work on iOS) and retrieve_search (FIFO result retrieval). Qwen3-8B running on a MacBook Pro interleaves summarization work with queued tool calls, eliminating idle time while the iOS worker processes searches.

Key Contributions

  • Proof-of-concept pipeline-parallel training/inference framework spanning a host computer and an iOS device over USB.
  • Manual PyTorch→MPSGraph layer translation with numerical verification (max error < 1×10⁻⁷).
  • Hybrid GPipe/1F1B pipeline adapted to MPSGraph's static-graph constraint.
  • Asynchronous split-tool abstraction enabling Qwen3-8B to parallelize tool execution on an iOS worker with model generation on the host, without any LRM fine-tuning.
  • Thermal characterization of sustained compute on iPhone 11 Pro (throttling onset ~batch 13–17).

Results

  • ResNet-34 training, Intel Xeon (2013) + iPhone 11 Pro: 22% reduction (13,105 ms/batch → 10,163 ms/batch).
  • ResNet-34 training, Intel Xeon + iPhone 16: 44% reduction (13,105 ms/batch → 7,308 ms/batch).
  • ResNet-34 training, MacBook Pro M2 Max (CPU-only) + iPhone 16: 25% reduction (9,009 ms/batch → 6,719 ms/batch).
  • ResNet-34 batch inference, Intel Xeon + iPhone 11 Pro: 36% reduction (4,400 ms/batch → 2,811 ms/batch).
  • Agentic tool parallelism: Theoretical elimination of tool-call idle time; Qwen3-8B successfully interleaved three sequential searches with summarization using the split-tool API zero-shot.
  • Thermal test: Thermal state escalated Minimal→Fair→Serious after ~13–17 batches of heavy load on iPhone 11 Pro, causing per-batch slowdowns of ~200+ ms.

Limitations

  • ResNet-34 (modest model) only; LLM-scale training was found impractical on iOS due to memory.
  • Training run of only 20 batches—thermal degradation over a full training run would reduce aggregate benefit.
  • Tool experiment uses a simulated 5-second delay (real vector search took ~10 ms), not a genuine heavy tool workload.
  • Qwen3-8B is a small model; complex multi-tool reasoning behavior may differ at scale.
  • Only iOS tested; no Android, tablets, or multi-device (>2-stage) pipelines due to MPSGraph static-graph constraint.
  • PyTorch→MPSGraph translation is manual and arduous, blocking rapid experimentation with larger architectures.
  • iOS sandbox caps usable RAM well below physical RAM (e.g., iPhone 11 Pro's 4 GB physical → ~2 GB usable).

Relevance to Agentic AI / LLM Agents

The split-tool / asynchronous tool-call pattern is directly relevant to agentic LLM pipelines: it operationalizes the idea that tool execution and model reasoning can be decoupled and pipelined, turning tool latency into "free" computation time. This builds on async LLM function-calling literature and demonstrates it end-to-end with a real (if small) LRM on commodity hardware. For researchers tracking agentic AI, the proposal that ubiquitous mobile sensors (LiDAR, GPS, audio) could serve as cheap, always-available tool endpoints for embodied agents is a forward-looking use case worth watching as mobile hardware continues to improve.