Automated quality control for genomic machine learning datasets: scores the biases, duplicates and data leakage a classifier could exploit before you train on it.
Searching test set transcription_factor_prediction_mouse_1_test.csv.gz against train set transcription_factor_prediction_mouse_1_train.csv.gz
Every test sequence searched against the train set with
MMseqs2,
and scored on its best hit. Similarity is
min(query coverage, target coverage) × percent identity: how much of
the shorter sequence the alignment spans, times how much of that matches.
The histogram is the distribution of those best hits.
The flag is the percentage of test sequences at or above the
threshold: Pass at 0%, Warning below 2%, Fail at 2% or more. A test
sequence that near-duplicates a training one is one the model has
already seen, so the accuracy it earns there is not evidence.
What to do about it →
The panel under the histogram lists those alignments, up to the first
100; every one of them, listed or not, is exported to
mmseqs/mmseqs2_search_result.tsv beside this report. Its columns are:
Query (Q)
Identifier of the test sequence: seq_<i>_test, where i is its 0-based position in the input test file.
Target (T)
Identifier of the train sequence: seq_<i>_train, where i is its 0-based position in the input train file.
Q Cov.
Fraction of the query sequence covered by the alignment (0–1).
T Cov.
Fraction of the target sequence covered by the alignment (0–1).
% Identity
Percent identical aligned positions in the aligned region.
% Similarity
Similarity score used for leakage detection, calculated as min(Q Cov., T Cov.) × % Identity.
E-value
MMseqs2 E-value (lower is more "significant").
Alignment
Click “Show” to expand the alignment visualisation for a row.
T CCACATGGTGGCTCACAACCATCCGTAACGAGATCTGGCGCCCTCTTCTGGAGTGTCTGAAGACAGCTACAGTGTACTTA
||||||||||||||||||||||||||||.|||||||||||||||||||||||||||||||||||||||||||||||||||
Q CCACATGGTGGCTCACAACCATCCGTAATGAGATCTGGCGCCCTCTTCTGGAGTGTCTGAAGACAGCTACAGTGTACTTA
T CATATAATAAATAAATAAATC
||||||||||||||.||||||
Q CATATAATAAATAAGTAAATC
seq_683_test
seq_36302_train
0.99
1.00
98.0
97.02
4.01e-47
T ATAGCTTTCAATTTCCCTGGCTTTGGAAACATCTTAACTTCCTGTAAACACTGTGGTCACTATGAAAAAAAAAATCACAG
||||||||||||||||||||||||||||||||||||||||||||||||.||||||||||||||| |||||||||||||||
Q ATAGCTTTCAATTTCCCTGGCTTTGGAAACATCTTAACTTCCTGTAAATACTGTGGTCACTATG-AAAAAAAAATCACAG
T CTTAGTCAGGCTCACATTCTT
|||||||||||||||||||||
Q CTTAGTCAGGCTCACATTCTTT
seq_5539_test
seq_21467_train
0.99
0.98
97.0
95.06
8.87e-45
T GATTTCTGAGTTCAAGGCCAGCCTGGTCTACAGAGTGAG-TTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCG
|||||||||||.||||||||||||||||||||||||| ||||||||||||||||||||||||||.|||||||||||||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTCTCG
T AAAAACCAAAAAAAAAAAAAAA
||||||||||||||||||||||
Q AAAAACCAAAAAAAAAAAAAAAA
seq_1395_test
seq_8280_train
0.98
0.98
96.9
94.96
1.91e-45
T TTCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAAAAA
.||||.||||||||||||||.|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Q TTGAGGTCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAAAAA
T AAAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAAAAAAAAAAAAAAAAAAAAA
seq_691_test
seq_32037_train
0.99
0.99
95.0
94.05
8.65e-43
T TAAAACAACTTTTAAAGTACTCAAAGTTGGGTAAAAGTTTGGACAACCATCTTTGGCGTTTTTTCAGACAGGAAGGGCGG
||||.||||||||||.||||||||||||||||||||||||||||..||||.||||||||||||||||||||||||||||
Q AAAATAACTTTTAAATTACTCAAAGTTGGGTAAAAGTTTGGACATGCATCCTTGGCGTTTTTTCAGACAGGAAGGGCGG
T GGGAGAGGCACTCACAAGCTC
|||||||||||||||||||||
Q GGGAGAGGCACTCACAAGCTCT
seq_1395_test
seq_49500_train
0.97
0.99
96.0
93.12
1.89e-43
T AGGCCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCGAGGGCTACACAGAGAAACCCTGTCTCAAAAAACCAAAAA
|||.|||||||||||||||||||||||||||||||||.|||||||||||||||||||||||||||||||| |||||
Q TTGAGGTCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAA--AAAAA
T AAAAAAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||||
Q AAAAAAAAAAAAAAAAAAAAAAA
seq_5539_test
seq_6573_train
0.99
0.98
95.0
93.10
1.78e-41
T TTCTGAGTTTGAGGCCAGCCTGGTCTACAGAGTGAG-TTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTTTCAAA
|||||||||.|||||||||||||||||||||||||| ||||||||||||||||||||||||||.|||||||||.||.||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTCTCGAA
T AAACCAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAACCAAAAAAAAAAAAAAAA
seq_5597_test
seq_43317_train
0.99
0.99
94.0
93.06
3.79e-41
T TGTCTTCAGACACTCCAGAAGAGGGCATCAGATTTCGTTACGGATGGTTGTGAGCCACCATGTGGTTGCTGGGATTTGA
|||||||||||||.|||||||||||..||||||.||.||||||||||||||||||||||||||||||||||||||||||
Q CTGTCTTCAGACACACCAGAAGAGGGAGTCAGATCTCATTACGGATGGTTGTGAGCCACCATGTGGTTGCTGGGATTTGA
T ACTCGGGACCTTTGGAAGAGCA
||||.||||||||||||||||
Q ACTCAGGACCTTTGGAAGAGC
seq_5913_test
seq_32488_train
0.99
0.99
94.0
93.06
3.79e-41
T ACCACATGGTGGCTCACAACCATCCGTAACAAGATCTGACTCCCTCTTCTGGTGTGTCTGAAGACAGCTACAGTGTACTT
||||||||||||||||||||||||||||..|||||||.|.|||||||||||.|||||||||||||||||||||||||||
Q CCACATGGTGGCTCACAACCATCCGTAATGAGATCTGGCGCCCTCTTCTGGAGTGTCTGAAGACAGCTACAGTGTACTT
T ACATATAATAAATAAATAAAT
|||||||||||||||.|||||
Q ACATATAATAAATAAGTAAATC
seq_2628_test
seq_34494_train
0.96
0.96
96.9
93.02
4.10e-44
T TTTTTTTTTTTTTCTTTTTTTTTTTTTTTTGGTTTTTCGAGACAGGGTTTCTCTGTGTAGCCCTGGCTGTCCTGGAACTC
|||||||||.|||||||||||||||.||||||||||||||||||||.|||||||||||||||||||||||||||||
Q TTTTTTTTTTTTTTTTTTTTTTTTTGGGTTTTTCGAGACAGGGTTTTTCTGTGTAGCCCTGGCTGTCCTGGAACTC
T ACTTTGTAGACCAGGCTGGCC
|||||||||||||||||||||
Q ACTTTGTAGACCAGGCTGGCCTTGT
seq_1846_test
seq_37636_train
0.96
0.96
96.9
93.02
4.10e-44
T GCAGAGGCAGGCGGATTTCTGAGTTCAAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTATACAG
||||||||||||||||||||||||.|.||||||||||||||||||||||||||||||||||||||||||||||||||
Q GAGGCAGAGGCAGGCGGATTTCTGAGTCCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTATACAG
T AGAAACCCTGTCTGGAAAAACCAA
|||||||||||||.||||||
Q AGAAACCCTGTCTCGAAAAAA
seq_1846_test
seq_27773_train
0.98
0.98
94.9
93.00
3.94e-42
T TCGAGGCAGAGGCAGGCGGATTTCTGAGTTCGAGGCCAGCCTGGTCTACAAAATGAGTTCGAGGACAGCCAGGGCTATAC
|||||||||||||||||||||||||||.||||||||||||||||||||||.|||||||.|||||||||||||||||||
Q GAGGCAGAGGCAGGCGGATTTCTGAGTCCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTATAC
T TGAGAAACCCTGTCTTGAAAA
.||||||||||||||.|||||
Q AGAGAAACCCTGTCTCGAAAAAA
seq_3771_test
seq_49531_train
0.98
0.98
94.9
93.00
3.94e-42
T TGGAATTGGGTCTTCTTTGAAAATCTGATAGAACTCTACACTAAACCCATCTGGTCCAGGGCTTTTTTTTTTTTTTTTTT
||||||.|||||||||||||..||||||||||||||||.|||||||||||||||||.||||||||||||||||||||||
Q GGAATTAGGTCTTCTTTGAAGGTCTGATAGAACTCTACTCTAAACCCATCTGGTCCTGGGCTTTTTTTTTTTTTTTTTT
T TTTTTTTTTTTTTTTTTTTTG
||||||||||||||||||||
Q TTTTTTTTTTTTTTTTTTTTAG
seq_1103_test
seq_21467_train
0.98
1.00
94.0
92.12
8.03e-41
T GATTTCTGAGTTCAAGGCCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCG
|||||||||||||.||||||||||||||||||.||||||||||||||||||||||||| ||||||||||.||||||.|
Q GGATTTCTGAGTTCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCT--ACAGAGAAACTCTGTCTTG
T AAAAACCAAAAAAAAAAAAAAA
||||||||||||||||||||||
Q AAAAACCAAAAAAAAAAAAAAAA
seq_5913_test
seq_8250_train
1.00
0.99
93.0
92.07
7.08e-39
T CCACATGGTGGCTCACAACCATCTGTAATGAGATCTGGCGCCCTCTTGTGGTGTGTGTGAAGACAGCTACAGTGTACTTA
|||||||||||||||||||||||.|||||||||||||||||||||||.|||.||||.|||||||||||||||||||||||
Q CCACATGGTGGCTCACAACCATCCGTAATGAGATCTGGCGCCCTCTTCTGGAGTGTCTGAAGACAGCTACAGTGTACTTA
T CATATAA-CAATAAATAAATCT
||||||| .|||||.||||||
Q CATATAATAAATAAGTAAATC
seq_5539_test
seq_20139_train
0.96
0.95
96.9
92.06
8.65e-43
T GCGGATTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAG-TTCCAGGACAGCCAGGGCTATACAGAGAAACCCTGTC
||||||||||||||||||||||||||||||||||||| ||||||||||||||||||||.|||||.||||||||||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTC
T TCGAAAAACCAAAAAAAAAAAA
||||||||||||||||||||||
Q TCGAAAAACCAAAAAAAAAAAAAAAA
seq_1395_test
seq_33086_train
0.98
0.98
93.9
92.02
1.70e-40
T TTCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTATACAGAGAAACCCTGTCTCAAAAAAAAAAAA
.||||.||||||||||||||.||||||||||||||||||||||||||.||||||||||||||||||||||||||||||
Q TTGAGGTCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAAAAA
T AAAAAAAAAAACCAAAAAAAA
|||||||||||..||||||||
Q AAAAAAAAAAAAAAAAAAAAAA
seq_2264_test
seq_12871_train
1.00
1.00
92.0
92.00
1.49e-38
T TTTTTTTTTTTTTTTTTTTTTTTTTTTGAGACAGGGTTTCTCTGTGTAGCCCTGGCTGTCCTGGAACTCACTTTGTAGAC
||.|||.|||.|||.|||.|||.|||||||||||||||||||||||||||||||||||||||||||||||||.|||||||
Q TTATTTATTTATTTATTTATTTATTTTGAGACAGGGTTTCTCTGTGTAGCCCTGGCTGTCCTGGAACTCACTCTGTAGAC
T CAGGCTGGCCTCGAACTCAGA
||||||||||||||||.||||
Q CAGGCTGGCCTCGAACACAGA
seq_1103_test
seq_20139_train
0.96
0.98
94.9
91.10
3.79e-41
T GCGGATTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTATACAGAGAAACCCTGTCT
|||||||||||||||||||||||||||||||||.|||||||||||||||||||||||| |||||||||||.||||||
Q GGATTTCTGAGTTCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGC--TACAGAGAAACTCTGTCT
T CGAAAAACCAAAAAAAAAAAA
.||||||||||||||||||||
Q TGAAAAACCAAAAAAAAAAAAAAAA
seq_2628_test
seq_24830_train
0.94
0.94
96.8
91.09
8.65e-43
T TTTTTTTTTTTTTTTTTTTTTTTGGTTTTTCGAGACAGGGTTTCTCTGTGTAGCCCTGGCTGTCCTGGCACTCACTT
||||||||||||||||||||||.||||||||||||||||||||.||||||||||||||||||||||||.||||||||
Q TTTTTTTTTTTTTTTTTTTTTTTTTGGGTTTTTCGAGACAGGGTTTTTCTGTGTAGCCCTGGCTGTCCTGGAACTCACTT
T TGTAGACCAGGCTGGCCTCGAACT
||||||||||||||||||
Q TGTAGACCAGGCTGGCCTTGT
seq_5539_test
seq_36430_train
0.98
0.97
93.9
91.08
3.37e-39
T TCTGAGTTCGAGGCCAGCCTGGTCTACAAAGTGAG-CTCCAGGACAGCCAGGGCTATACAGAGAAACCCTGTCTCAAA
||||||||||||||||||||||||||||.|||||| .|||||||||||||||||||.|||||.||||||||||||.||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTCTCGAA
T AAACCAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAACCAAAAAAAAAAAAAAAA
seq_1395_test
seq_44490_train
0.98
0.98
92.9
91.04
7.08e-39
T TTCGAGGCCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTATGCAGAGAAACCCTGTCTCGAAAAACCAAAA
.||||.|||||||||||||||||||||||||||||||||||||||||..|||||||||||||||||.|||||..||||
Q TTGAGGTCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAAAAA
T AAAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAAAAAAAAAAAAAAAAAAAAA
seq_1395_test
seq_49905_train
0.98
0.98
92.9
91.04
7.08e-39
T TTCGAGGCCAGCCTGGTCTACAACGTGAGTTCCAGGACAGCCAGGGCTATGCAGAGAAACCCTGTATCAAAAAAAAAAAA
.||||.||||||||||||||..|||||||||||||||||||||||||..||||||||||||||.||||||||||||||
Q TTGAGGTCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAAAAA
T AAAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAAAAAAAAAAAAAAAAAAAAA
seq_1395_test
seq_13558_train
0.96
0.95
95.8
91.01
3.79e-41
T GAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTCCAGGA-AGACAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAA
.||||.||||||||||||||||||||||||||||| ||.||||||||||||||||||||||||||||||||||||
Q TTGAGGTCAGCCTGGTCTACAGAGTGAGTTCCAGGACAGCCAGGGCTACACAGAGAAACCCTGTCTCAAAAAAAAA
T AAAAAAAAAAAAAAAAAAAAAA
||||||||||||||||||||||
Q AAAAAAAAAAAAAAAAAAAAAAAAA
seq_2074_test
seq_22045_train
0.96
0.96
94.8
91.01
8.03e-41
T GCCATCCGTAACAAGATCTGGCGCCCTCTTCTGGAGTGTCTGAAGACAGCTACAGTGTACTTAGATATAATAAATA
.|||||.|||||||||||||||||||||||||||||||.||||||||||||||||||||||.|.||||||||||||
Q CACAACCATCTGTAACAAGATCTGGCGCCCTCTTCTGGAGTGGCTGAAGACAGCTACAGTGTACTAACATATAATAAATA
T AATAAATCTTTAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AATAAATCTTTAAAAAAAAAA
seq_1846_test
seq_2666_train
0.97
0.97
93.8
90.99
7.59e-40
T CGGGAGGCAGAGGCAGGCGGATTTCTGAGTTCAAGGCCAGCCTGGTCTACAAAGTGAGCTCAAGGACAGCCAGGGCTACA
|||||||||||||||||||||||||||.|.|||||||||||||||||||||||||.||.||||||||||||||||.|
Q GAGGCAGAGGCAGGCGGATTTCTGAGTCCGAGGCCAGCCTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTATA
T CAGAGAAACCCTGTCTCAAAA
|||||||||||||||||.|||
Q CAGAGAAACCCTGTCTCGAAAAAA
seq_5539_test
seq_31577_train
0.98
0.97
92.9
90.11
1.36e-37
T TCTGAGTTCGAGGCCAACCTGGTCTACAAAGTGAG-TTCCAGGACTGCCAGGGCTATACAGAGAAACCCTGTCTCAAA
||||||||||||||||.|||||||||||.|||||| |||||||||.||||||||||.|||||.||||||||||||.||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTCTCGAA
T AAACCAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAACCAAAAAAAAAAAAAAAA
seq_5539_test
seq_40505_train
1.00
0.99
91.0
90.09
1.09e-35
T TTTCTGAGTTCGAGGCTAGCCTGGTCTACAGAGTGAG-TTCCAGGACAGCTGAAGCTATATAGAGAAACCCTGTCTCGAA
||||||||||||||||.|||||||||||||||||||| ||||||||||||....||||.|.|||.|||||||||||||||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTCTCGAA
T AAACCAAAAAAAAAAAAAAAAG
|||||||||||||||||||||
Q AAACCAAAAAAAAAAAAAAAA
seq_5539_test
seq_47572_train
0.97
0.96
93.8
90.05
1.49e-38
T CTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAG-TTCCAGGATAGCCAAGGCTACACAGAGAAACCCTGTTTCAAA
|||||||||||||||||||||||||||||||||| ||||||||.|||||.|||||||||||.|||||||||.||.||
Q TTTCTGAGTTCGAGGCCAGCCTGGTCTACAGAGTGAGTTTCCAGGACAGCCAGGGCTACACAGAAAAACCCTGTCTCGAA
T AAACCAAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAACCAAAAAAAAAAAAAAAA
seq_2404_test
seq_46989_train
0.96
0.96
93.8
90.05
3.37e-39
T CTGGTCTACAAAGTGAGTTCCAGGACAGCCAGGGCTATACAGAGAAACCCTGTCTCGAAAAACCAAAAAAAAAAAAAAAA
||||||.|||||||||||||||||.||.|.|||.|||||||||||||||||.||||||||||||||||||||||||
Q TCTACAGAGTGAGTTCCAGGACAGGCAAGACTACACAGAGAAACCCTGTCTAGAAAAACCAAAAAAAAAAAAAAAA
T AAAAAAAAAAAAAAAAAAAAA
|||||||||||||||||||||
Q AAAAAAAAAAAAAAAAAAAAAGGTA