Not started
Work Package
WP6 — Development of Move Through Sensors, Tracking and Localization for all modalities (Lead: IDEMIA)
Goals & Objectives
- Develop human verification (anonymous person count) and tracking using heartbeat as well as ultrasound localization to identify, localize and track smartphones’ digital wallets.
- Incorporate privacy-preserving, non-invasive methods to ensure secure verification using smartphone sensors.
- Integrate robust multispectral and camera-based person face verification and localization solutions for accurate real-time tracking for best image extraction.
Tasks
- Task 6.1: Heartbeat localization for people/pet detection and distinction (persons/pets) and Ultrasound-based Smartphone localization (CAFA (L), FHG; M4-30) CAFA develops a system that uses heartbeat signals for the precise localization and tracking of individuals (person count). The technology relies on specialized sensors capable of detecting and analysing the unique bio-signatures emitted by a person’s heartbeat, even through walls or in crowded environments. These sensors, positioned strategically within the environment, capture and trilateration the heartbeat signals to determine an individual’s position and track movement in real time. The development includes creating algorithms that filter out background noise and other bio-signatures, ensuring accurate and reliable tracking, even in complex and dynamic environments. The system must also accommodate varying heart signatures to filter out pets. Privacy and ethical considerations are paramount, so the tracking data is anonymized, encrypted, and stored securely in compliance with data protection regulations before being deleted again (e.g., GDPR). This technology will be transferred from security monitoring, emergency response, and healthcare applications, allowing for non-invasive, continuous tracking of individuals while ensuring data integrity and confidentiality.
- Task 6.2: Camera-based localization and tracking of faces for best image extraction and localization (Walk/move through) (IDEMIA (L), UL, FHG, VSENS; M4-30) The task consists of development of a system that uses image processing techniques to localize and track faces in real time using camera feed. Using deep learning models, the system can detect and track faces across frames, ensuring accuracy under various conditions such as different lighting, angles, and partial occlusions. The development focuses on optimizing detection algorithms for speed and accuracy, ensuring minimal latency for real-time tracking. The system also includes robust measures for data privacy and security, encrypting facial data and ensuring compliance with privacy regulations like GDPR. Ethical considerations are addressed by implementing transparency measures and allowing for opt-in consent mechanisms where applicable. The architecture is designed for scalability, supporting multiple cameras and integrating with existing surveillance systems or mobile devices to enhance coverage and tracking capabilities.
- Task 6.3: Smartphone camera image processing for move-through, and border inspection (AIT (L), UL, THALCZ; M4-30) Smartphone camera images optimized for on-the-move scenarios enable real-time data capture during walk-throughs or inspections, enhancing mobile verification and inspection processes. These sensors are designed to stabilize images and video while the user is in motion, using advanced features like optical image stabilization, motion compensation, and AI-based image enhancement to ensure clear, reliable data capture. For walk-through applications, the system can detect and track faces, objects, or specific items of interest, seamlessly capturing relevant information without requiring the user to pause. This capability is particularly valuable in inspection and monitoring contexts, allowing users to document conditions or verify identities efficiently and with high accuracy. Data security is ensured through encryption during capture and transfer, maintaining privacy and compliance with regulatory standards. This on-the-go capability provides a robust solution for dynamic, mobile environments, making it ideal for security, inspection, and compliance tasks.
- Task 6.4: Robust facial detection, tracking and counting of passengers in slowly moving cars (VSENS (L), UL, IDEMIA; M4-30) The objective of this task is to provide a multi-camera solution which allows to detect faces and count passengers in slowly moving cars under 24/7 outdoor-conditions (drive-through) utilizing multi-spectral sensors, such as visible, infrared, and near-infrared. Hereby, one sub-component will detect approaching / passing cars and count passengers inside a vehicle, exploring hereby the usefulness of different spectral ranges. A second sub-system will detect and track faces from passengers with the objective to provide best suited facial images for matching within WP6. Key challenges hereby include real-time tracking of travellers using visible / NIR video streams under out-door variations, mapping extracted tracks from different camera perspectives to individuals and selection of best shots. The implementation will utilize edge devices to reduce latency, enable efficient deployment, and allow scalability, while the management software can be deployed locally or in the cloud, based on operational needs.
- Task 6.5: ICAO-compliant travel document reading (THALES (L), XLAB; M4-30) Document reader is a tool designed to streamline the process of capturing, analysing, and verifying identity documents such as ICAO compliant passports, IDs, and driver licenses. Equipped with OCR (Optical Character Recognition) and AI-driven data extraction capabilities, the document reader quickly pulls relevant information from documents, reducing manual entry and minimizing errors. This solution is particularly beneficial for border control, and onboarding, where fast and accurate document processing is essential. The reader can flag discrepancies, authenticate document features (e.g., watermarks), and cross-reference information with databases for verification. Integrated with secure data transmission protocols, the document reader ensures that sensitive information is handled confidentially and in compliance with data privacy regulations. This automated document reader enhances helpdesk efficiency, accuracy, and security, allowing staff to focus on higher-level tasks and providing customers with a seamless, responsive service experience.