Federated Prompt Learning for Weather Foundation Models on Devices

Shengchao Chen, Guodong Long, Tao Shen, Jing Jiang, Chengqi Zhang

arXiv preprint · 23 May 2023

In one sentence. Federated prompt tuning of a frozen weather foundation model for on-device forecasting, using dynamic graphs to handle heterogeneity and cut communication cost.

Abstract

On-device intelligence for weather forecasting uses local deep learning models to analyze weather patterns without centralized cloud computing, holds significance for supporting human activates. Federated Learning is a promising solution for such forecasting by enabling collaborative model training without sharing raw data. However, it faces three main challenges that hinder its reliability: (1) data heterogeneity among devices due to geographic differences; (2) data homogeneity within individual devices and (3) communication overload from sending large model parameters for collaboration. To address these challenges, this paper propose Federated Prompt Learning for Weather Foundation Models on Devices (FedPoD), which enables devices to obtain highly customized models while maintaining communication efficiency. Concretely, our Adaptive Prompt Tuning leverages lightweight prompts guide frozen foundation model to generate more precise predictions, also conducts prompt-based multi-level communication to encourage multi-source knowledge fusion and regulate optimization. Additionally, Dynamic Graph Modeling constructs graphs from prompts, prioritizing collaborative training among devices with similar data distributions to against heterogeneity. Extensive experiments demonstrates FedPoD leads the performance among state-of-the-art baselines across various setting in real-world on-device weather forecasting datasets.

Labels

Labels and summary are generated by Claude and reviewed weekly. Wrong? Edit this entry.