Fine-Tuning and Deploying LLMs for Autonomous Homes: From Supercomputing Validation to Edge AI Agents
About this session
This session delivers a practical, engineering-focused look into the fine-tuning, validation, and localization of open-source LLMs for 'The Safe Haven' project. We will explore the pipeline of training multiple models and utilizing rigorous validation tests to determine the optimal model deployment per country based on language and performance metrics. Moving away from traditional RAG architectures, we will demonstrate how the deployed Edge AI Agent operates using internal domain-adapted weights to understand the home environment and predictively control home systems. Finally, we will break down the multimodal interface where the resident speaks to the home, and the agent processes the query to respond in real-time using a custom-trained cloned voice and a photorealistic digital avatar.
Speaker
Key takeaways
- LLM Fine-Tuning & Localization: How to design validation benchmarks to select and deploy the best performing local models for specific geographic regions.
- Predictive Edge Agents vs. RAG: Utilizing fine-tuned model weights for autonomous home system control and behavior prediction without relying on external databases.
- Multimodal Voice & Video Integration: The architectural pipeline for syncing voice-to-text processing with custom voice cloning and photorealistic avatar rendering in real time.