Echoes under Drift

Echoes under Drift

You monitor short soundscapes through embeddings produced by a frozen audio
encoder. Twelve known event classes can appear between stretches of
background. Some events belong to none of those classes and must be marked as
unknown_event. A useful monitoring system must also say when every event
starts and ends.

The training and validation recordings come from three familiar acoustic
domains. The test set mixes familiar domains with microphones and venues whose
response has shifted. It also contains unknown event families that are absent
from the labeled training split. Test-domain and leaderboard membership are
hidden and the rows are interleaved.

This is a file-prediction task. You do not submit code.

Data

Download the dataset ZIP and extract echoes_data/. All arrays use the order
given by the corresponding *_scene_ids.npy file.

Outputs 0..11 are the named known events from metadata.json; output 12
is unknown_event. A background frame has thirteen zeros. Events never overlap
in this dataset. Boundary channel 0 marks the first active frame (onset), and
channel 1 marks the last active frame (offset).

The values are original synthetic embeddings, not redistributed recordings.

Submission

Upload one ZIP containing exactly these two files at its root:

1submission.zip2├── boundary_probs.npy3└── frame_probs.npy

• frame_probs.npy: C-contiguous float32, shape (300, 32, 13);
• boundary_probs.npy: C-contiguous float32, shape (300, 32, 2);
• every value must be finite and in [0, 1];
• no folder, metadata, notebook, model, or source file may be added.

The two output channels are scored as probabilities and need not sum to one.
The dataset includes baseline.py, validate_submission.py, and a ready-made
sample submission.

Evaluation

Each leaderboard split receives a metric in [0, 1]:

1metric = 0.50 × known-class macro AP2       + 0.25 × unknown-event AP3       + 0.25 × onset/offset F14score  = 100 × metric

Known-class AP is calculated frame-wise and averaged equally across the twelve
known classes. Unknown-event AP uses output 12. For boundary F1, predictions
are thresholded at 0.5; a predicted onset or offset is correct if it can be
matched one-to-one to the same boundary type in the same scene within one
frame.

The partial leaderboard uses familiar acoustic domains. The complete
leaderboard emphasizes shifted domains and held-out unknown families. Exact
membership, domain IDs, and labels are private. The platform has no configured
subtasks: partial and complete are leaderboard partitions, not separately
submitted files.

Practical direction

A functional solution can start from the provided multi-output ridge baseline.
To improve, validate by leaving one familiar domain out, calibrate unknownness
instead of treating it as just another closed-set class, and model temporal
continuity explicitly. Robust normalization, self-supervised adaptation on the
unlabeled test inputs, out-of-distribution scores, and sequence models offer
different paths to partial credit.

This is an original RO-OAI training task. It is not an official IOAI problem,
leaked material, or a guaranteed prediction of the Astana contest.