GridForget

1. Context

A warehouse robot was trained with linear Q-learning in several grid environments. Some of the training environments contained a portal shortcut that was useful at the time. The corresponding client has now requested that the influence of those environments be removed.

You no longer have the original training trajectories. You receive:

• the deployed Q-function;
• retain-only proxy transitions from related safe warehouses;
• a compact parameter-importance memory;
• unlabeled probes from the environments that must be forgotten.

Your goal is to output a new Q-function that behaves as closely as possible to an oracle model retrained without the forgotten environments.

The problem contains a second difficulty: after your unlearning procedure, the robot will continue Q-learning on new retain-only experience. A superficial mask may initially hide the old portal behavior and then allow it to reappear. Your model must therefore remain forgotten after additional updates.

2. Grid environment

Each environment is a $9 \times 9$ grid.

The action set is:

Ordinary movement has reward $-0.03$. Hitting a wall leaves the robot in the same state and gives reward $-0.08$. Reaching G gives $+1$, while entering X gives $-1$.

In the retain environments, entering P returns the robot to S and gives reward $-0.35$. In the environments covered by the deletion request, P was a beneficial shortcut: it teleported the robot near G and gave $+0.35$.

The deletion request is not equivalent to globally banning every portal feature. Portal observations interact with walls, traps, goal direction, and the chosen action. A successful solution should approximate the policy that would have been learned from retain environments only.

3. Q-function

The model is linear:

The 19-dimensional feature vector is:

1. bias;
2. normalized row and column;
3. relative goal row and column;
4. normalized Manhattan distance to the goal;
5. four indicators for adjacent walls;
6. four indicators for adjacent traps;
7. four indicators for adjacent portals;
8. one indicator showing that the robot currently stands on a portal.

The exact order is available in data/pretrained_model.csv and data/compact_memory.csv.

4. Files

data/pretrained_model.csv

The deployed model before unlearning.

data/proxy_retain.csv

Retain-only transitions:

Columns f0 ... f18 contain $\phi(s)$, and nf0 ... nf18 contain $\phi(s')$.

data/compact_memory.csv

A diagonal importance estimate for each parameter. Large values indicate that changing the parameter may damage retained behavior. The values may be used as an EWC-like penalty:

The compact memory is intentionally imperfect. Treat it as a hint, not as a certificate.

data/forget_probes.csv

Feature vectors sampled from environments covered by the deletion request. The file contains the old teacher's preferred action and confidence margin, but does not reveal the oracle action after retraining.

data/public_ground_truth.csv

A small public validation subset used by local_eval.py. The official score also uses hidden maps, hidden probes, and hidden update streams.

data/maps/

Human-readable examples of retain and forget grids.

5. Submission format

Submit one CSV file named submission.csv with exactly 76 rows:

1parameter_id,subtaskID,value2a0_f0,1,0.01233a0_f1,1,-0.18404...5a3_f18,1,0.0912

Every expected parameter_id must appear exactly once. Values must be finite and have absolute value at most 25.

The action indices are:

6. Evaluation

The evaluator compares your model with an oracle model retrained without the forgotten environments. It does not require your parameters to match the oracle parameters exactly; it compares their behavior on hidden states.

Task 1 — Immediate unlearning: 50 points

The evaluator measures, separately on retain and forget probes:

• greedy-action agreement with the oracle;
• similarity between softmax action distributions;
• similarity of the confidence margin between the best two actions.

The retain and forget metrics are combined using a geometric mean. Therefore:

• keeping the old forbidden behavior is insufficient;
• destroying the whole policy is insufficient;
• constant or uniformly random Q-values receive a low score.

Task 2 — Stable unlearning: 50 points

The evaluator continues Q-learning on hidden retain-only transition streams:

with $\gamma=0.95$ and $\alpha=0.004$.

The model is reevaluated after 25, 50, 100, and 200 updates. The weakest checkpoint matters, so a forgotten behavior that resurfaces later is penalized.

For each task, a raw metric at or below 0.55 gives 0 points, while a metric at or above 0.92 gives the full 50 points. Intermediate values are mapped linearly.