Welcome to the Spike Sorting Evaluation Project
Extracellular recordings are a standard procedure to analyzing the activity of neurons. A problem with this kind of recording is the simultaneous recording of not a single neuron but a small local population of neurons. It is not straight-forward to reconstruct the single neuronal activities. To estimate the single neuron activity (the so called spike train) from this multi neuron activity, spike sorting is applied. Many different algorithms for spike sorting were proposed. However, despite many efforts to tackle this problem, it is still difficult to tell under which circumstances which spike sorting algorithm is the best. If you are interested in some basic reading about spike sorting please consult the following references.
"A review of methods for spike sorting: the detection and classification of neural action potentials" M. S. Lewicki (1998), Network: Computation in Neural Systems, Vol. 9, No. 4. (1998), pp. 53-78
"Towards reliable spike-train recordings from thousands of neurons with multielectrodes." Einevoll GT, Franke F, Hagen E, Pouzat C, Harris KD (2011), Curr Opin Neurobiol. 2011 Oct 22.In May 2011, a workshop on Validation of Automatic Spike Sorting Methods took place in Ski, Norway.
The Problem - No Benchmark
In our opinion the lack of systematic evaluations of different spike
sorting algorithms derives from the fact that there is no common benchmark
dataset used in the spike sorting literature, rendering comparisons of the
results of different publications virtually impossible.
However, a good
benchmark dataset is difficult to establish - since it is currently
impossible to record the true activity of a larger number of neurons
simultaneously with extracellular recordings.
Aim of this project
We want to establish a platform for spike sorting with two different goals: First, we want to provide a sophisticated benchmark data set which could be used for future publications on this topic. Second, we want to give researchers using spike sorting software the possibility to evaluate their software under different conditions and compare their results to other spike sorters.
How this website works
This website provides extracellular data of various types for download. Common to this data is, that we know the ground truth i.e. we know how many neurons are present in the data and when which neuron fired an action potential. The datasets vary in their underlying questions.
If you have decided which dataset you are interested in, go to the download page and download the datafiles in the format which is most convinient to you.
You can now apply the spike sorter of your choice to the dataset. Keep in mind, that we do not publish the ground-truth for the datafiles.
The result of your sorting can then be uploaded via the website. This data will be treated confidential, we will not publish it, unless you say so. You will retrieve a unique key for your upload.
The website now automatically calculates an evaluation of your sorting using the hidden ground truth information. However, this may take a while. Once finished you can access the results under the MyResults page using the key provided under 3.
If you are comfortable with the results you can help us by allowing us to publish your results on the Summary Page for this dataset. You will have to provide some additional information about the algorithm you used. Furthermore, only complete results (for all files of the benchmark dataset) can be published. Published results can be seen by everybody and are compared to the published results of other users.
What this website is not
This website is not providing spike sorting algorithms or the functionality to actually sort your data. It is just about validation of spike sorting methods.