Merge branch 'master' of gitlab.svi.edu.au:biocellgen-private/mage_2020_marker-gene-benchmarking

notes/benchmarks.md

+# Benchmarks
+
+* Overlap of methods
+
+* Recovery of true markers (ROC, TPR/FPR, etc.)
+
+* Recovery of known markers (from database + expert human annotation) 

notes/methods.md

@@ -4,19 +4,84 @@ This file contains information about the different methods we will benchmark.
 
 ### scanpy
 
-This new default is t-test not t-test over_estim_var
+By default compares one vs rest
+
+Tests:
+
+* t-test: Default 
+* t-test_overestim_var (overestimates variance of each group)
+* Logistic regression
+
+Uses p-value correction 
 
 ### Seurat
 
+Testing one vs rest 
+
+Uses log fold change threshold
+
+Tests:
+
+* Wilcoxon Rank Sum test (default)
+* Likelihood ratio test (McDavid et. al. 2013)
+* ROC - Evalulates a classifier built on each gene alone
+* t-test
+* Negative Binomial GLM (not with shrinkage etc)
+* Poisson GLM
+* Logistic regression (also prediction)
+* MAST
+* DESeq2
+
 ### scran 
 
-### Others 
+Pairwise testing b/w cluster and all other clusters
+
+Tests
+
+* t-test
+* Binomial test
+* Wilcox rank sum test 
+
+Implements TREAT
+
+### rankcorr
+
+Multiclass marker selection
+
+Non-parametric ranking transformation 
+
+"one-vs-all" method
+
+### Venice 
+
+Odd algorithm based on the predictive accuravy of genes
+
+One vs rest
+
+In R package Signac
+
+### Semitones
+
+Can be used for other purposes
+
+Uses enrichement scoring
+
+### Other
+
+Elastic net (scikit-learn) benchmarked in rankcorr
+
+Random markers - sanity check on simulations 
+
+SCDE - found to be too slow in rankcorr
+
+scGeneFit - not individual clusters, clustering as a whole
 
-#### scTIM 
+COMET - only select 4 markers - designed for further FACS sorting
 
-diffxpy?
+scVI - not sure how it is used
 
-Natranos et al. (2018)
+Other single-cell DE methods
 
-Come up with method categorisation
+scTIM - doesn't find per cluster marker genes
 
+singleCellHaystack - find regions where DEGs exists not for existing clusters

notes/nomenclature.md

@@ -6,3 +6,5 @@ Biomarkers - Seruat tutorials
 Signature genes -  bryan2018
 
 Cell type specific vs disease markers
+
+Marker genes as feature selection HVGs etc (SCMarker, Wang 2019)