Haoming Wang

We show that current VLMs hold a latent topological map of 3D scenes, but it is overshadowed by non-geometric semantics like color and shape. Isolating this subspace via cross-scene linear feature extraction reveals its correspondence to the Laplacian eigenmaps of the scene’s 3D Gaussian-kernel graph, motivating a Dirichlet-energy-based latent regularizer that beats standard SFT by up to 12.1% on real-world spatial benchmarks.

MosaicThinker is an inference-time method for on-device embodied AI that fuses spatial cues from multiple video frames into a unified semantic map, then prompts a small VLM to reason over it. The approach upgrades cross-frame spatial reasoning accuracy on resource-constrained devices across diverse manipulation and planning tasks.

This work examines how multi-view visual spatial reasoning can be explained through reasoning-path tracking and latent-state analysis, bridging cognitive science perspectives with modern multimodal models.

Modern VLMs require robust spatial-reasoning evaluation, but existing benchmarks lack diversity, scalability, and fine-grained control over scene complexity. To address this, we introduce InfiniBench, a fully automated and customizable benchmark generator capable of producing an unlimited variety of photo-realistic 3D scenes and videos from natural-language descriptions. Through its agentic constraint-refinement framework, cluster-based layout optimizer, and task-aware camera trajectory design, InfiniBench enables precise analysis of VLM failure modes and outperforms prior 3D generation methods, while supporting diverse spatial-reasoning tasks such as measurement, perspective-taking, and spatiotemporal tracking.

Personalizing Large Language Models (LLMs) is crucial for meeting individual user needs on mobile devices. However, on-device personalization faces challenges from limited computational resources and scarce personal data. We propose XPerT, a technique that fine-tunes an already personalized LLM using user data and selects models based on explainability of prior fine-tuning.

Federated AIoT leverages distributed IoT data to train AI models, but heterogeneous devices introduce data heterogeneity and varying staleness, degrading model performance and slowing training. Existing FL frameworks treat device delays as independent of data heterogeneity, which is unrealistic. We propose FedDC, a technique that mitigates delay impacts when these factors are correlated. FedDC uses gradient inversion to infer local data distributions and compensate for delay-induced update bias.

Federated Learning (FL) is challenged by intertwined data and device heterogeneities—differences in clients’ local data distributions and model update staleness. Traditional FL methods treat these separately, which is unrealistic and often ineffective. We propose a novel FL framework that converts stale model updates into unstale ones, addressing these intertwined heterogeneities efficiently. Our method estimates clients’ local data distributions from their stale updates to compute unstale updates, without requiring auxiliary datasets, fully trained local models, or extra client-side computation or communication.

Personalized image generation models better serve diverse user needs but often lack explainability regarding how personalization occurs. While visual cues exist, they are hard for users to interpret, and current natural language explanations are too coarse-grained to capture multiple aspects or degrees of personalization. We propose FineXL, a technique for Fine-grained eXplainability in natural Language, which generates detailed textual descriptions and quantitative scores for each personalization aspect.

Text-to-image diffusion models can be fine-tuned for personalized domains, but this adaptability also enables misuse, such as forging public figures, replicating copyrighted artworks, or producing explicit content. Existing detection or unlearning methods fail to prevent illegal adaptations. We introduce FreezeAsGuard, a technique that irreversibly mitigates such misuse by selectively freezing tensors in pre-trained diffusion models that are critical to illegal adaptations, while preserving legal fine-tuning capabilities.