-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TRCA variation #39
TRCA variation #39
Conversation
Regularization in covariance matrices estimations + riemannian mean instead of euclid mean for S computation
Codecov Report
@@ Coverage Diff @@
## master #39 +/- ##
==========================================
+ Coverage 77.94% 78.27% +0.33%
==========================================
Files 20 20
Lines 2158 2200 +42
==========================================
+ Hits 1682 1722 +40
- Misses 476 478 +2
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @ludovicdmt, that's great thanks a lot! I've left a few comments if you could please address them?
If you don't have time I'll do it myself later this week. I'll also try to improve the example_trca.py a bit
Thanks for the feedback, I'll have a look and do that tomorrow. |
Should be good now |
I'm a bit concerned by the very poor performance compared to the original method. A performance drop can be expected, but in this cas we sometimes get ~30% accuracy on the example dataset. The tests fail because I am testing all combinations of edit: Actually the accuracy drop is also visible even when always the euclidean mean ( edit2: actually |
Regarding performance, as explained in 1 geodesic mean is more robust to outliers than the euclidean mean. But it is also really sensible to ill-conditioned covariance matrices. On data with smaller SNR and with more training samples, this TRCA variation gave us ~10% of performance improvement. I think the two methods are complementary in terms of use cases. But maybe that's not the point of Meegkit toolbox which is more about collecting standard methods and I can't definitely understand that. |
The fact that the riemann method doesn't work as well on this dataset is a problem in itself, it's just that it makes it difficult to unit-test. For the original implementation, I can be reasonably sure that my code is correct because it yields good classification results. Here it's a bit more complex, and it will make catching potential future bugs (due to dependency changes, etc.) more difficult to track |
OK I restricted the test scope, and added an illustration to the example. Thanks @ludovicdmt |
Ok thank you for the help and work @nbara ! |
Hello,
In addition to the original method of TRCA, I propose in this PR a variation that use regularization and riemannian mean for the computation of the matrix S (instead of euclidean mean as in original implementation). It provides improvements when data are noisy and have access to large number of calibration data.
This variation is inspired by a similar work from A. Barachant on CSP: https://hal.archives-ouvertes.fr/hal-00602686/document
Ideas are :
However the quality of the estimation of the riemannian mean depends on the number of covariance matrices used.
It does not provide an improvement of performance for the specific dataset used in ./testd/data/trcadata.mat but we notice at least 10% of improvement on our data that are more noisy and use more calibration data (about 10 trial per class).
This variation rely on Pyriemann toolbox which is already in requirements.