A Program-Synthesis Challenge for ARC-Like Tasks

Abstract

We propose a program synthesis challenge inspired by the Abstraction and Reasoning Corpus (ARC) [3]. The ARC is intended as a touchstone for human intelligence. It consists of 400 tasks, each with very small numbers (3–5) of ‘input-output’ image pairs. It is known that the tasks are ‘human-solvable’ in the sense that, for any of the tasks, there exists a human-authored description that transforms input images in the task to the corresponding output images. Besides the ‘small data problem’, other features of ARC make it hard to use as a yardstick for machine learning. The solutions are not provided, nor is it known if they are unique. The use of some basic prior knowledge is acknowledged, but no definitions are available. The solutions are known also to apply to images that may be significantly different to those provided, but those images are not described. Inspired by ARC, but motivated to address some of these issues, in this paper we propose the Inductive Program Synthesis Challenge for ARC-like tasks (IPARC). The IPARC challenge is much more controlled, focusing on the inductive synthesis of structured programs. We specify for the challenge a set of ‘ARC-like’ tasks characterised by: training and test example sets drawn from a clearly-defined set of ‘ARC-like’ input-output image pairs; a set of image transformation functions from the image-processing field of Mathematical Morphology (MM); and target programs known to solve the tasks by transforming input to output images. The IPARC tasks rely on a result known as the ‘Structured Program Theorem’ that identifies a small set of rules as sufficient for construction of a wide class of programs. Tasks in the IPARC challenge are intended for machine learning methods of program synthesis able to address instances of these rules. In principle, Inductive Logic Programming (ILP) has the techniques needed to identify the constructs implied by the Structured Program Theorem. But, in practice, is there an ILP implementation that can achieve this? The purpose of the IPARC challenge is to determine if this is the case.

A. Srinivasan—AS is currently visiting the Centre for Health Informatics, Macquarie University and the School of CSE, UNSW.

A Proofs

We prove Proposition 1 by using the following two lemmas.

Lemma 1

Given any $m_1 \ne m_2$ and $n_1 \ne n_2$, there exists a sequence of $\psi$ transitions which take an image from $\mathcal {L}_{m_1,n_1}$ and returns an image from $\mathcal {L}_{m_2,n_2}$.

Lemma 2

Given any two images $I_{m,n}^{k}, \tilde{I}_{m,n}^{k} \in \mathcal {L}_{m,n}^{k}$, one can find a sequence of transitions within $\phi$ operators which takes in as input $I_{m,n}^{k}$ and returns $\tilde{I}_{m,n}^{k}$.

From Lemmas 1 and 2, it is clear that there exists a sequence of transitions which takes in as input $I_{m_1,n_1}^{k}$ and returns $\tilde{I}_{m_2,n_2}^{k}$. We first prove Lemma 1

Proof

(Proof of Lemma 1). The proof follows from the observation that one can find an integer p such that $pm_1 \ge m_2$ and $pn_1 \ge n_2$. Thus, consider the resizing operator $\psi ^{1,p}$ which copies the image $I_{m_1,n_1}^{k}$ on a grid of size $p\times p$, followed by $\psi _{(0,0),(m_2,n_2)}(.)$ which crops the image with corners (0, 0) and $(m_2, n_2)$. This achieves the desired result.

Proof

(Proof of Lemma 2). Observe the following - Given any two binary images (of same shape) $I_1, I_2$, with $I_1$ as non-zero, we have that,

$$\begin{aligned} \delta _{I_2}\epsilon _{I_1}(I_1) = I_2 \end{aligned}$$

(5)

Case A: If all of $I_{m,n}^{(i)}$ (refer Fig. 3.2) are non-empty, we have that there exists a transition $\phi ^{(i)}$ which takes $I_{m,n}^{(i)}$ to output $\tilde{I}_{m,n}^{(i)}$.

Case B: Let $I_{m,n}^{(i_0)}$ be empty for some colour $i_0$. Since $I_{m,n}^{k}$ is non-empty, there exists an $i'$ such that $I_{m,n}^{(i')}$ is non-empty. We define the operator $\phi$ as follows - $\phi ^{(i)} = id$, where id denotes the identity map, and the colour change c maps colour $i'$ to $i_0$. Then the proof is as in Case A.

Proof

(Proof of Proposition 2). The proof of Proposition 2 follows from the following observation - Any given transition in $\varSigma _1$ either

• Takes an element from $\mathcal {L}_{m,n}^{k}$ to itself, or

• Takes an element from $\mathcal {L}_{m,n}^{k}$ to $\tilde{\mathcal {L}}_{m,n}^{k}$, or

• Takes an element from $\mathcal {L}_{m,n}^{k}$ to $\mathcal {L}_{m',n'}^{k}$ where $m' \ne m$ and $n' \ne n$.

Thus, given any $I_1, I_2$ and f we have that the shape of $(\tilde{I}_1) =_{\varSigma } f(I_1)$ should be the same as $(\tilde{I}_2) =_{\varSigma } f(I_2)$.

However, it is possible for an ARC-like task to have different shapes for $(\tilde{I}_1)$ and $(\tilde{I}_2)$. An example of this is shown below (Fig. 7).

Fig. 7.

Example of ARC-like task with different output shapes