Nightwatch Soundscapes

Nightwatch Soundscapes

Story

A city safety team monitors short nighttime soundscapes. A frozen audio
transformer has already converted every recording into one embedding per time
frame. Your task is to detect which sound events are active in every frame.

Several events may overlap: speech can occur while a car passes, rain can mask
footsteps, and an alarm can continue across many frames. The first 16 event
classes are familiar from the earlier study; 13 newly introduced classes have
fewer labeled examples. Some test scenes also come from unseen microphones and
venues.

You do not receive raw audio and should not reconstruct it. Work directly
with the supplied frozen embeddings.

Data

Every scene contains exactly 24 ordered frames. Every frame has a 48-dimensional
float32 embedding. Labels are 29-dimensional multi-hot vectors:

• classes 0–15: old, comparatively frequent classes;
• classes 16–28: new, comparatively rare classes;
• 1 means the event is active in that frame;
• several entries in one frame may be 1.

The dataset ZIP contains a nightwatch_data/ directory:

Load all NumPy files with allow_pickle=False.

The test rows are deliberately interleaved. The public/partial leaderboard uses
144 familiar-domain scenes. The final/complete leaderboard uses 216 scenes from
shifted, previously unseen domains. Split membership is hidden.

Submission

Upload a ZIP archive containing exactly one root-level file:

1submission.zip2└── predictions.npy

predictions.npy must be a C-contiguous NumPy array with:

• dtype exactly float32;
• shape exactly (360, 24, 29);
• finite probabilities in [0, 1];
• axis 0 in the exact order of test_embeddings.npy / test_scene_ids.npy;
• axis 1 in chronological frame order;
• axis 2 in class-ID order 0 through 28.

Example:

1import io2import zipfile3import numpy as np4​5predictions = model_predictions.astype(np.float32)6assert predictions.shape == (360, 24, 29)7assert np.isfinite(predictions).all()8assert ((0 <= predictions) & (predictions <= 1)).all()9​10buffer = io.BytesIO()11np.save(buffer, np.ascontiguousarray(predictions), allow_pickle=False)12with zipfile.ZipFile("submission.zip", "w", zipfile.ZIP_DEFLATED) as archive:13    archive.writestr("predictions.npy", buffer.getvalue())

Additional files, folders, symbolic links, other dtypes, malformed arrays,
NaN/Infinity, and values outside [0, 1] are rejected and receive score 0.

Evaluation

Each leaderboard split is scored independently using:

1. Old-class macro average precision over classes 0–15;
2. New-class macro average precision over classes 16–28;
3. Temporal micro F1 over every scene/frame/class decision after applying
   the fixed probability threshold 0.5.

For one split:

1composite = 0.35 × old_macro_AP2          + 0.35 × new_macro_AP3          + 0.30 × temporal_micro_F1@0.54score     = 100 × composite

The score is between 0 and 100. There are no separately configured platform
subtasks. During the competition, the visible score is computed only on the
partial split; the complete split determines the final result.

Average precision rewards good probability ranking, while temporal F1 also
requires calibrated probabilities. A model that ranks events correctly but
never crosses 0.5 will lose the F1 component.

Runtime note

Only the submitted array is evaluated: participant code is not executed. The
evaluator is designed for the platform limit of 30 seconds and 3 GiB RAM.

Reproducibility

The task is an original, deterministic synthetic simulation. See SOURCE.md
and the dataset metadata for provenance and generator version.