Summary: Researchers report improvement of up to 90% in seizure prediction, thanks to crowdsourcing over 10,000 algorithms world wide.
Source: University of Melbourne.
A study by University of Melbourne researchers reveals clinically relevant epileptic seizure prediction is possible in a wider range of patients than previously thought, thanks to the crowdsourcing of more than 10 000 algorithms worldwide.
In 2016 researchers ran the Melbourne University AES-MathWorks-NIH Seizure Prediction Challenge on the online data science competition platform Kaggle.com.
The contest focused on seizure prediction using long-term electrical brain activity recordings from humans obtained in 2013 from the world-first clinical trial of the implantable NeuroVista Seizure Advisory System. Researchers rigorously evaluated the top algorithms and these findings are detailed in research published today in Brain: A Journal of Neurology.
University of Melbourne Dr Levin Kuhlmann, from the Graeme Clarke Institute and St Vincent’s Hospital Melbourne, said the contest was a huge success, with more than 646 participants, 478 teams and more than 10 000 algorithms submitted from around the world.
“Epilepsy affects 65 million people worldwide,” Dr Kuhlmann said. “We wanted to draw on the intelligence from the best international data scientists to achieve advances in epileptic seizure prediction performance for patients whose seizures were the hardest to predict.”
Contestants developed algorithms to distinguish between 10-minute inter-seizure verses pre-seizure data clips and the top algorithms were tested on the patients with the lowest seizure prediction performance based on previous studies.
“Our evaluation revealed on average a 90 per cent improvement in seizure prediction performance, compared to previous results,” Dr Kuhlmann said.
“Epilepsy is highly different among individuals. Results showed different algorithms performed best for different patients, supporting the use of patient-specific algorithms and long-term monitoring.”
Building on this success, researchers have developed Epilepsyecosystem.org, an online ecosystem for algorithm and data sharing to further develop and improve seizure prediction.
“Accurate seizure prediction will transform epilepsy management by offering early warnings to patients or triggering interventions,” Dr Kuhlmann said.
“Our results highlight the benefit of crowdsourcing an army of algorithms that can be trained for each patient and the best algorithm chosen for prospective, real-time seizure prediction.
“It’s about bringing together the world’s best data scientists and pooling the greatest algorithms to advance epilepsy research. The hope is to make seizures less like earthquakes, which can strike without warning, and more like hurricanes, where you have enough advance warning to seek safety.”
The research was led by the Graeme Clark Institute of Biomedical Engineering, in collaboration with St. Vincent’s Hospital Melbourne, Swinburne University of Technology, Mayo Clinic, Perelman School of Medicine at the University of Pennsylvania and Seer Medical.
Source: Holly Bennett – University of Melbourne
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com image is credited to HKUST.
Original Research: Abstract for “Epilepsyecosystem.org: crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG” by Levin Kuhlmann, Philippa Karoly, Dean R Freestone, Benjamin H Brinkmann, Andriy Temko, Alexandre Barachant, Feng Li, Gilberto Titericz, Jr., Brian W Lang, Daniel Lavery, Kelly Roman, Derek Broadhead, Scott Dobson, Gareth Jones, Qingnan Tang, Irina Ivanenko, Oleg Panichev, Timothée Proix, Michal Náhlík, Daniel B Grunberg, Chip Reuben, Gregory Worrell, Brian Litt, David T J Liley, David B Grayden, and Mark J Cook in Brain. Published August 8 2018.
Epilepsyecosystem.org: crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG
Accurate seizure prediction will transform epilepsy management by offering warnings to patients or triggering interventions. However, state-of-the-art algorithm design relies on accessing adequate long-term data. Crowd-sourcing ecosystems leverage quality data to enable cost-effective, rapid development of predictive algorithms. A crowd-sourcing ecosystem for seizure prediction is presented involving an international competition, a follow-up held-out data evaluation, and an online platform, Epilepsyecosystem.org, for yielding further improvements in prediction performance. Crowd-sourced algorithms were obtained via the ‘Melbourne-University AES-MathWorks-NIH Seizure Prediction Challenge’ conducted at kaggle.com. Long-term continuous intracranial electroencephalography (iEEG) data (442 days of recordings and 211 lead seizures per patient) from prediction-resistant patients who had the lowest seizure prediction performances from the NeuroVista Seizure Advisory System clinical trial were analysed. Contestants (646 individuals in 478 teams) from around the world developed algorithms to distinguish between 10-min inter-seizure versus pre-seizure data clips. Over 10 000 algorithms were submitted. The top algorithms as determined by using the contest data were evaluated on a much larger held-out dataset. The data and top algorithms are available online for further investigation and development. The top performing contest entry scored 0.81 area under the classification curve. The performance reduced by only 6.7% on held-out data. Many other teams also showed high prediction reproducibility. Pseudo-prospective evaluation demonstrated that many algorithms, when used alone or weighted by circadian information, performed better than the benchmarks, including an average increase in sensitivity of 1.9 times the original clinical trial sensitivity for matched time in warning. These results indicate that clinically-relevant seizure prediction is possible in a wider range of patients than previously thought possible. Moreover, different algorithms performed best for different patients, supporting the use of patient-specific algorithms and long-term monitoring. The crowd-sourcing ecosystem for seizure prediction will enable further worldwide community study of the data to yield greater improvements in prediction performance by way of competition, collaboration and synergism.