Dataset

BEACON:Isoform

BEACON:Isoform

Not in the Core Set 10 of the 93 cleared all five gates

Task: Regression, RNA Processing Prediction
Level: RNA
Species: Homo sapiens
Sequences: 228,388 across 3 splits
Sequence length: 186 to 186
Licence: Apache-2.0

Read the 5 full reports for this dataset Every distribution, probe fit and overlapping sequence behind the numbers below

Fail worst flag across the ten checks on train, this dataset's reference split
4 of the ten checks returned Fail on that split 1 returned Warning, 0 could not be scored
0.749 AUROC from the strongest single sequence feature, which is Per sequence nucleotide content A lower bound on what a model could recover from the sequences alone
37.24% of one split's sequences were also present in another, at worst The largest of 2 split comparisons: train vs test
0.960 AUROC when all those features are read together rather than one at a time Measured on a held-out split. This is what Core Set membership turns on

Where this sits in the corpus

Stronger than 70 of the 93 probed datasets
Higher than 80 of the 92 jointly scored datasets
More overlap than 84 of the 92 scanned datasets

A rank among the datasets this run measured, not among benchmarks in general.

What the checks returned

Worst flag across the ten checks on the train split: Fail

Of the nine checks that measure a single split, 9 ran on train and 9 were scored: 4 returned Fail, 1 returned Warning, 0 returned Unknown. The leakage scan is counted in the flag above but not in these figures, because it compares pairs of splits rather than measuring one.

Every rollup figure on this page was measured on train, which is this dataset's reference split — not a maximum over its splits.

The strongest single feature is Per sequence nucleotide content, which reaches AUROC 0.749 on the train split from that feature alone. That is a lower bound: a stronger feature, or a model, may recover more.

The leakage scan found that 37.24% of the sequences in test were also present in train; the same comparison the other way puts 29.89% of train in test. That is the largest of the two split comparisons the scan made here.

What each of the ten checks measures · What a flag is, and what it does not say · Why one split is quoted

The reports behind those flags

The full report genomic-benchmarks-qc rendered for each split and each split pair, with every distribution, every probe fit, and every individually-listed overlapping sequence behind the numbers on this page. Each opens inside this site, and the file the tool wrote is one click further on.

train · seq class distributions
val · seq class distributions
test · seq class distributions
train vs test leakage scan
train vs val leakage scan

Worked examples of a report

The same features, read together

Every flag above is one feature on its own. This is all of them at once: length, GC, mononucleotide and dinucleotide content and the two per-position profiles, handed to a single model and scored on a held-out split. Nothing is added that the checks do not already look at — the only change is that they are read jointly.

0.960 AUROC from all the checks’ features at once, measured on the test split 95% interval 0.959 to 0.962, over 49,755 held-out sequences

The strongest single feature reaches 0.749 on its own.

How it was measured, why it exists, and the six datasets it changes the reading of

The nine per-split checks

One row per split, one column per check. The leakage scan is not here because it compares split pairs rather than measuring one split; it has its own section below.

The nine class-distribution checks on each of BEACON:Isoform’s 3 splits. The rollup reports the train row.

Table view

Flag and, where measured, the strongest probe AUROC per check per split.
Split	GC content	Nucleotide content	Dinucleotide content	Sequence lengths	Per position nucleotide	Per position from 3′	Duplicates within label	Duplicates across labels	Unique bases
train	Warning 0.685	Fail 0.749	Fail 0.700	Pass 0.500	Fail 0.719	Fail 0.719	Pass	Pass	Pass
val	Fail 0.705	Fail 0.757	Fail 0.703	Pass 0.500	Fail 0.708	Fail 0.708	Pass	Pass	Pass
test	Warning 0.694	Fail 0.738	Fail 0.713	Pass 0.500	Warning 0.699	Warning 0.699	Pass	Pass	Fail

Each split, the flag it rolled up to, its strongest probe, and the report every flag on that row was read out of.
Split	Sequence columns	Worst flag	Checks scored	Strongest probe	AUROC	Report
`train` reference	`seq`	Fail	9	Per sequence nucleotide content	0.749	seq
`val`	`seq`	Fail	9	Per sequence nucleotide content	0.757	seq
`test`	`seq`	Fail	9	Per sequence nucleotide content	0.738	seq

How far each probe gets

The AUROC each of the six probe checks reaches on each split, from that one feature alone. The axis starts at 0.50 because that is what a feature carrying no class information returns.

What each single-feature probe recovered, on each split. Where a dataset has several sequence columns the strongest is shown; the table view lists every value.

The axis starts at 0.50 because that is what a probe recovering nothing would read. A value is a lower bound: it is what one feature reached, not the most that could be reached.

Table view

Probe AUROC per check per split.
Check	train	val	test
Per sequence GC content	0.685	0.705	0.694
Per sequence nucleotide content	0.749	0.757	0.738
Per sequence dinucleotide content	0.700	0.703	0.713
Sequence lengths	0.500	0.500	0.500
Per position nucleotide content	0.719	0.708	0.699
Per position reversed nucleotide content	0.719	0.708	0.699

Split comparisons

For every pair of splits the scan compared, the share of each split whose sequences also appear in the other. Each bar is coloured by the split it counts.

The two directions of a pair are different statements. A held-out split largely contained in train is the one that bears on whether a score on it is a held-out score.

Table view

Exact-sequence overlap, both directions, per split pair.
Comparison	Share	Flag
train also in val	27.44%	Fail
val also in train	36.48%	Fail
train also in test	29.89%	Fail
test also in train	37.24%	Fail

Each pair, in both directions. The two percentages differ whenever the two splits differ in size, so both are shown rather than the larger one.
Comparison	Flag	Of the first split	Of the second
`val` in `train`	Fail	36.48%	27.44%
`test` in `train`	Fail	37.24%	29.89%

Licence, credit, and the data itself

What this data may be used for is its source's decision and not this project's. It is set down here as the curation read it, with the evidence, so that a reader can check the reading rather than take it.

Licence: Apache-2.0
Credit: 10.1016/j.cell.2019.04.046 — this dataset's

Each split's file, and where it is served from:

train — Isoform_train.csv.gzhuggingface.co
val — Isoform_val.csv.gzhuggingface.co
test — Isoform_test.csv.gzhuggingface.co

This licence permits redistribution, so the split files may be republished under its terms rather than only fetched from the links above. The paper states that the datasets and the source code of the benchmark are available at github.com/terry-r123/RNABenchmark, and that repository is Apache-2.0, so the release the authors point at is the licensed one. Recorded with the qualification that the files themselves are served from the Google Drive folder the repository links (19ddrwI8ycvIxkgSV3gDo_VunLofYd4-6), which carries no terms of its own; neither the paper nor its NeurIPS datasheet states a licence for the data separately. Apache-2.0 permits redistribution provided the licence and the attribution notices are kept.

How to cite the run these numbers come from

Nearby

Nearest by name in BEACON: BEACON:CRISPROnTarget, BEACON:MeanRibosomeLoading