# Generalisation and specialisation

# 9.1. Generalisation and specialisation#

An *example* is a ground fact for the predicate of which a definition is to be induced. A *positive* example is true in the intended interpretation, while a *negative* example is false. Consequently, the inductive \(Hypothesis\) should be such that for every positive example \(p\)

while for every negative example \(n\)

We say that \(p\) is *covered* by \(Hypothesis\), given \(Theory\). For instance, if \(Hypothesis\) is the standard recursive definition of `element/2`

:

```
element(X,[X|Z]).
element(X,[Y|Z]):-element(X,Z).
```

then the example `element(b,[a,b])`

is covered (with empty \(Theory\)). This can be demonstrated by a simple meta-interpreter for definite clauses. Note that this proof requires **both** of the above clauses. Alternatively, if `element(b,[b])`

is also known to be a positive example, we can say that `element(b,[a,b])`

is covered by the second, recursive clause alone. The first definition of coverage, which refers to the complete hypothesis, is called *intensional* coverage, while the second, referring to single clauses plus the rest of the examples, is called *extensional* coverage. In the induction programs to be developed, we will employ both notions of coverage; for the moment, however, the distinction is immaterial.

Write a predicate `covers_ex/3`

which, given a clause, an example, and a list of positive examples, tests whether the clause extensionally covers the example.

If \(Hypothesis1\) covers at least all the examples covered by \(Hypothesis2\), we say that \(Hypothesis1\) is at least as *general* as \(Hypothesis2\), or that \(Hypothesis2\) is at least as *specific* as \(Hypothesis1\). From the definition of coverage, one can see that \(Hypothesis2\) must be a logical consequence of \(Hypothesis1\), given \(Theory\):

Suppose \(p\) is a positive example covered by \(Hypothesis1\) but not by \(Hypothesis2\). This means that \(Hypothesis2\) is too specific; if it is our current hypothesis, it needs to be *generalised*, for instance to \(Hypothesis1\). Similarly, if a hypothesis covers a negative example, it needs to be *specialised*. Generalisation and specialisation are the basic operations of induction.

Although we defined generality between hypotheses being **sets** of clauses, practical approaches to induction usually generalise or specialise single clauses. For instance, the following are clauses of increasing generality:

```
element(X,[Y|Z]):-element(X,Z).
element(X,V):-element(X,Z).
element(X,V).
```

This shows that a more specific clause can be constructed by adding a literal, by applying a substitution, or both. This relation of generality between clauses is called \(\theta\)-subsumption. Formally, `Clause1`

*\(\theta\)-subsumes* `Clause2`

if there is a substitution \(\theta\) that can be applied to `Clause1`

, such that every literal in the resulting clause occurs in `Clause2`

.

Notice that \(\theta\) only replaces variables in `Clause1`

, not in `Clause2`

. One way to test if such a \(\theta\) exists is to ground all variables in `Clause2`

, and then unify the ground version of `Clause2`

with `Clause1`

. Grounding the variables in a term can be done by means of the built-in predicate `numbervars/3`

, which unifies different variables with terms of the form `'$VAR(N)'`

.

```
theta_subsumes1((H:-B1),(H:-B2)):-
ground(B2),
subset(B1,B2).
ground(Term):-
numbervars(Term,0,N).
%%% subset/2: see Section 10.2 (appendix)
```

This approach has the disadvantage that one or both clauses are changed after a call to `theta_subsumes1/2`

. To avoid this, we apply the following little programming trick:

```
theta_subsumes((H1:-B1),(H2:-B2)):-
not((H1=H2,ground(B2),
not subset(B1,B2))).
```

`theta_subsumes/2`

succeeds exactly when `theta_subsumes1/2`

does, but by means of the double negation unifications are ‘undone’ after the call succeeds.

Next, we turn to the issue of how to construct generalisations of clauses. First we consider the simpler case of generalising two atoms. Consider the following two ground facts:

```
element(1,[1]).
element(z,[z,y,x]).
```

The following atom \(\theta\)-subsumes both of them:

```
element(X,[X|Y])
```

Note that this atom is \(\theta\)-subsumed by every other possible generalisation (such as `element(X,[Y|Z])`

or `element(X,Y)`

). For this reason, it is called a *least general generalisation under \(\theta\)-subsumption* or \(\theta\)-LGG. \(\theta\)-LGG’s of atoms can be computed by means of *anti-unification*. This operation is the dual of unification. It operates by comparing the terms occurring at the same position in the two atoms, and replacing them by a new variable if they are different. The terms which have already been replaced by a variable are collected in two lists, because if the same pair of terms is encountered again, it should be replaced by the same variable (see `1`

and `z`

in the example above). For obvious reasons, such lists are called *inverse substitutions*.

```
:-op(600,xfx,'->'). % operator for inverse substitution
% anti_unify(T1,T2,T) <- T is the anti-unification
% of T1 and T2
anti_unify(Term1,Term2,Term):-
anti_unify(Term1,Term2,Term,[],_S1,[],_S2).
% anti-unification with inverse subst.s and accumulators
anti_unify(Term1,Term2,Term1,S1,S1,S2,S2):-
Term1 == Term2,!. % same terms
anti_unify(Term1,Term2,V,S1,S1,S2,S2):-
subs_lookup(S1,S2,Term1,Term2,V),!. % already substituted
anti_unify(Term1,Term2,Term,S10,S1,S20,S2):-
nonvar(Term1),nonvar(Term2),
functor(Term1,F,N),functor(Term2,F,N),!, % same
functor(Term,F,N), % functor
anti_unify_args(N,Term1,Term2,Term,S10,S1,S20,S2).
anti_unify(T1,T2,V,S10,[T1->V|S10],S20,[T2->V|S20]).
anti_unify_args(0,_Term1,_Term2,_Term,S1,S1,S2,S2).
anti_unify_args(N,Term1,Term2,Term,S10,S1,S20,S2):-
N>0,N1 is N-1,
arg(N,Term1,Arg1),
arg(N,Term2,Arg2),
arg(N,Term,Arg),
anti_unify(Arg1,Arg2,Arg,S10,S11,S20,S21),
anti_unify_args(N1,Term1,Term2,Term,S11,S1,S21,S2).
subs_lookup([T1->V|_Subs1],[T2->V|_Subs2],Term1,Term2,V):-
T1 == Term1,
T2 == Term2,!. % no alternative solutions needed
subs_lookup([_S1|Subs1],[_S2|Subs2],Term1,Term2,V):-
subs_lookup(Subs1,Subs2,Term1,Term2,V).
```

The following query illustrates the operation of the program, including the use of inverse substitutions:

```
?-anti_unify(2*2=2+2,2*3=3+3,T,[],S1,[],S2).
T = 2*X=X+X
S1 = [2->X]
S2 = [3->X]
```

Note that the inverse substitution `[2->X]`

does not indicate which occurrences of `2`

should be replaced by `X`

. This means that `S1`

applied to the first term does not yield `T`

(the inverse of `S1`

applied to `T`

yields the first term, however). Therefore, a proper definition of inverse substitution should include the positions of terms which are to be replaced by variables. We will not elaborate this any further here.

The construction of the \(\theta\)-LGG of two clauses makes use of, but is more complicated than anti-unification. The basic difference with anti-unification is that the body of a clause is logically speaking unordered, whereas subterms within a term have fixed positions. Therefore, we cannot just compare the literals occurring at the same position in the respective bodies, but should consider all pairs of literals, one from each body. For instance, the \(\theta\)-LGG of the following two clauses

```
element(c,[b,c]):-element(c,[c]).
element(d,[b,c,d]):-element(d,[c,d]),element(d,[d]).
```

is the clause

```
element(X,[b,c|Y]):-element(X,[c|Y]),element(X,[X]).
```

The head of this clause is simply obtained by anti-unifying the heads of the original clauses, and the body is obtained by anti-unification of `element(c,[c])`

and `element(d,[c,d])`

, giving `element(X,[c|Y])`

, and anti-unification of `element(c,[c])`

and `element(d,[d])`

, giving `element(X,[X])`

.

The program for constructing \(\theta\)-LGG’s is given below. Note that the inverse substitutions found in each step are passed on to the next, so that the literals share variables.

```
:-op(900,fy,not).
% theta_lgg(C1,C2,C) <- C is the θ-LGG of clause C1 and C2
theta_lgg((H1:-B1),(H2:-B2),(H:-B)):-
anti_unify(H1,H2,H,[],S10,[],S20), % heads
theta_lgg_bodies(B1,B2,[],B,S10,_S1,S20,_S2). % bodies
% select literal from first body...
theta_lgg_bodies([],_B2,B,B,S1,S1,S2,S2).
theta_lgg_bodies([L|B1],B2,B0,B,S10,S1,S20,S2):-
theta_lgg_literal(L,B2,B0,B00,S10,S11,S20,S21),
theta_lgg_bodies(B1,B2,B00,B,S11,S1,S21,S2).
% and one from second body
theta_lgg_literal(_L1,[],B,B,S1,S1,S2,S2).
theta_lgg_literal(L1,[L2|B2],B0,B,S10,S1,S20,S2):-
same_predicate(L1,L2),anti_unify(L1,L2,L,S10,S11,S20,S21),
theta_lgg_literal(L1,B2,[L|B0],B,S11,S1,S21,S2).
theta_lgg_literal(L1,[L2|B2],B0,B,S10,S1,S20,S2):-
not same_predicate(L1,L2),
theta_lgg_literal(L1,B2,B0,B,S10,S1,S20,S2).
% same_predicate(L1,L2) <- literals L1 and L2 have
% the same predicate and arity
same_predicate(L1,L2):-functor(L1,P,N),functor(L2,P,N).
```

To check the above example, we pose the following query:

```
?-theta_lgg((element(c,[b,c]):-[element(c,[c])]),
(element(d,[b,c,d]):-
[element(d,[c,d]),element(d,[d])]), C).
C = element(X,[b,c|Y]):-[element(X,[X]),element(X,[c|Y])]
```

Tip

Use `portray_clause/1`

to ‘pretty-print’ a clause with readable variable names:

```
?-theta_lgg((element(c,[b,c]):-[element(c,[c])]),
(element(d,[b,c,d]):-
[element(d,[c,d]),element(d,[d])]), C),
portray_clause(C).
```

Technically, this grounds the variables with `numbervars/3`

which was mentioned earlier.

Determine the \(\theta\)-LGG of the following two clauses:

```
reverse([2,1],[3],[1,2,3]):-reverse([1],[2,3],[1,2,3]).
reverse([a],[],[a]):-reverse([],[a],[a]).
```

The relation between \(\theta\)-subsumption and logical consequence#

If `Clause1`

\(\theta\)-subsumes `Clause2`

, then also `Clause1`

\(\models\) `Clause2`

. The reverse, however, is not always true. Consider the following two clauses:

```
list([V|W]):-list(W).
list([X,Y|Z]):-list(Z).
```

Given `list([])`

, the first clause covers lists of arbitrary length, while the second covers only lists of even length. All lists covered by the second clause are also covered by the first, which is therefore more general. However, there is no substitution that can be applied to the first clause to yield the second (such a substitution should map `W`

both to `[Y|Z]`

and to `Z`

, which is impossible).

It may seem that \(\models\) provides a better notion of generality than \(\theta\)-subsumption. However, such a semantic definition of generality introduces two problems. One is that it does not suggest a simple procedure to generalise clauses, as \(\theta\)-subsumption does. The second problem is that LGG’s under logical consequence are not always unique. Consider the two clauses

```
list([A,B|C]):-list(C).
list([P,Q,R|S]):-list(S).
```

Under logical consequence, these clauses have two LGG’s: one is `list([X|Y]):-list(Y)`

, and the other is `list([X,Y|Z]):-list(V)`

. Under \(\theta\)-subsumption, only the latter is an LGG. Note that the first LGG looks in fact more plausible!

In the following section we develop a program which generalises the examples by constructing \(\theta\)-LGG’s. This corresponds to a *specific-to-general* search of the space of possible predicate definitions; it is also called *bottom-up* induction. Alternatively, one could start with the most general definition, which is specialised as long as it covers some negative example. A program for *top-down* induction is given in Section 9.3.