This research aims to enable early detection of health deterioration by analysing behavioural and physiological signals collected passively in everyday living settings. The proposed approach integrates two main sensing modalities: ambient audio monitoring and video-based pose estimation. Audio data will be analysed to detect respiratory symptoms such as coughing, wheezing, or wet cough associated with COPD exacerbations. To protect privacy, signal processing techniques such as pitch shifting and band-pass filtering will be applied to remove intelligible speech from recorded audio. Video monitoring will employ pose estimation algorithms that capture only the key joint locations of the human body rather than identifiable images. This allows the system to track physical activity, sedentary behaviour, and deviations from normal routines while preserving user privacy. Reduced movement and increased inactivity are important behavioural indicators of COPD exacerbation.
The research will combine these modalities through artificial intelligence models that analyse cough sounds and activity patterns simultaneously. Multimodal analysis helps reduce false positives, for example by verifying that a detected cough is accompanied by corresponding body movement. Additional safeguards will include training models to recognise cough sounds specific to an individual’s vocal tract and identifying sound direction to distinguish patient coughs from background sources such as television audio.
The system will be implemented as a low-cost embedded platform, potentially using hardware such as a Raspberry Pi with integrated microphone and camera sensors. Additional optional sensing may periodically capture clinical indicators and support monitoring for related conditions such as Alzheimer’s disease or depression, which may also benefit from unobtrusive home monitoring.
To support use in multi-occupant households, the system will employ non-intrusive identification methods such as gait patterns, body dimensions, or personal accessories (e.g., glasses or watches). Face recognition may be used only if the data remains locally processed within the home device. A proof-of-concept prototype will be developed, with initial testing conducted using simulated coughing data from research team members before progressing to real-world studies involving participants.
The project introduces novelty in several areas. First, it explores privacy-preserving in-home video monitoring, which remains under-researched due to privacy concerns. Second, it combines audio and activity visual data to improve the reliability of respiratory symptom detection. Third, the system will integrate agentic AI models, including large language models such as BioMedLM or MedAlpaca, to interpret aggregated patient data and provide explainable health assessments. These AI agents will analyse patterns such as cough frequency and activity levels, generate evidence-based insights, and support early intervention or preventative care.
Overall, the research aims to demonstrate that multimodal, privacy-aware sensing and advanced AI reasoning can enable accurate, unobtrusive health monitoring in domestic environments, improving early detection and management of chronic respiratory conditions.
