Research Project
Ongoing · Human Memory Lab @ UPenn
This project aims to develop a generalized automated annotation framework for continuous data, focusing on applications across EEG, speech data, and other time-series signals. The framework leverages advanced machine learning models to enhance annotation efficiency and precision.
Automated annotation of continuous data is essential for fields that handle large-scale datasets, such as neuroscience and speech analysis. This project is designed to create a flexible, domain-agnostic framework that can be applied to various types of continuous data, automating the detection and annotation of key events with high accuracy.
Under the mentorship of my PI Dr. Michael J. Kahana and postdocs in the lab, I am leveraging state-of-the-art AI methods to tackle the specific challenges of event detection in continuous data streams:
WhisperX for Speech Transcription and Onset Detection: Utilizing OpenAI’s Whisper model, WhisperX provides robust transcription and onset detection. While accurate in many settings, the model faces challenges with millisecond-level timing precision, which I am working to enhance in this framework.
EEG Event Detection Using Adaptive Thresholding: Building on methods like EEG-Annotate, I am detecting and labeling events in EEG data, especially for non-time-locked neural responses. Adaptive thresholding techniques are applied to improve accuracy in complex, real-world environments.
Domain Adaptation Techniques: By employing domain adaptation, I am working to ensure that the framework generalizes effectively across different data types, allowing reliable performance in applications ranging from speech to neural data. This adaptability is crucial for scaling automated annotation to multiple research areas.
Improving Onset Precision: A key focus of my work is enhancing onset detection precision for applications requiring fine-grained temporal accuracy, such as continuous EEG data and noisy audio environments. I am building on WhisperX’s capabilities, aiming to achieve sub-millisecond accuracy.
The primary goals of this project are:
Under the mentorship of my PI and postdoctoral collaborators, I am working on a primary authorship publication that will detail the AI methodologies and innovations behind this framework. The paper will describe my approach to enhancing WhisperX for greater temporal precision, the application of adaptive thresholding for EEG signals, and the successes achieved in generalizing the framework across diverse data types. This research is expected to make significant contributions to fields reliant on automated annotation, such as neuroscience and computational linguistics.