Shattering-Extremal Set Systems of Small VC-Dimension
We say that a set system shatters a given set if . The Sauer inequality states that in general, a set system shatters at least sets. Here, we concentrate on the case of equality. A set system is called shattering-extremal if it shatters exactly sets. We characterize shattering extremal set systems of Vapnik-Chervonenkis dimension 1 in terms of their inclusion graphs. Also, from the perspective of extremality, we relate set systems of bounded Vapnik-Chervonenkis dimension to their projections.
Throughout this paper, will be a positive integer, and the set will be referred to shortly as , the power set of it as , and the family of subsets of size as ().
The central notion of our study is shattering. We say that a set system shatters a given set if . The family of subsets of shattered by is denoted by . The following inequality gives a bound on the size of .
Proposition 1. .
The statement was proved by several authors, such as Aharoni and Holzman , Pajor , Sauer , and Shelah . It is often referred to as Sauer inequality. One of the most interesting cases is the case of equality, that is, when the set system shatters exactly sets. We call such set system shattering extremal or s-extremal for short. Many interesting results have been obtained in connection with these combinatorial objects, among others in [5–9].
The Vapnik-Chervonenkis dimension of a set system , denoted by , is the maximum cardinality of a set shattered by . The general task of giving a good description of s-extremal systems seems to be too complex at this point. We restrict therefore our attention to the simplest cases, where the -dimension of is bounded by some fixed natural number .
After the Introduction, in Section 2, we first investigate s-extremal set systems of -dimension 1 from a graph theoretical point of view. We give a bijection between the family of such set systems on the ground set and trees on vertices. As a consequence, one can exactly determine the number of such s-extremal set systems. In combinatorics, when considering set systems with a given property, it is a common step to first consider families of some special structure. According to , uniform set systems cannot be s-extremal. As a next possibility, set systems from two consecutive layers turn up. In Section 3, we prove that they are just special cases of the previous ones. After this in Section 4, we switch to an algebraic point of view and investigate bases of the polynomial ideals attached to extremal set systems. The main result of Section 5 is a connection between s-extremal set systems of -dimension and their projections. At the end, we propose an open problem and make some concluding remarks.
In the paper, we will use the terminology of  for graph theoretical notions.
2. s-Extremal Set Systems of VC-Dimension 1
Let be an s-extremal family. Let be the labelled Hasse diagram of considered as a graph, that is, a graph whose vertices are the elements of , and there is a directed edge going from to , labelled with exactly when . will be called the inclusion graph of . When representing the elements of by their characteristic vectors, can also be considered as the subgraph in the Hamming graph spanned by the elements corresponding to the sets in with edges directed and labelled in a natural way. Actually, for the next proposition, we can forget about the directions of the edges, and consider as an undirected edge-labelled graph. We further assume that . Our aim is to characterize these kinds of s-extremal set systems in terms of their inclusion graph.
Proposition 2. A set system is s-extremal and of at most 1 if and only if is a tree and all labels on the edges are different.
Proof. For the “only if” direction, suppose that is s-extremal and . According to [10, Theorem ], we know that must be isometrically embedded into (i.e., for any two elements , the distance between and is the same in and in ). This means, in particular, that is connected. Next, we prove that all labels on the edges of are different. Suppose for contradiction that there are two edges with the same label. Without loss of generality, we may assume that this label is . Since there are no two edges going out from a set with the same label, there are sets , all different, such that , , and . Since , is nonempty; so, there is an element . Without loss of generality, we may assume that . Now,
So, is shattered by ; consequently, , contradicting the assumption that .
To finish with this direction, note that the fact that all labels are different implies that is acyclic. Suppose for contradiction that it is not the case, and contains a cycle. Pick one edge from this cycle, and let be its label. On the remaining part of the cycle, there must be another edge labelled with , since it connects a set containing with one not containing . However, this is impossible, since all labels are different. Adding the connectedness of , we obtain that it is actually a tree as wanted.
For the reverse direction, suppose that is a tree and all labels on the edges are different. It is easily seen that this implies that is isometrically embedded into . (Otherwise a path from a set to in which is not shortest in would contain edges with the same label, corresponding to the addition and deletion of the same element of .)
Now, we prove that . Suppose the contrary, namely, that shatters a set of size , for example, . This means that there are sets such that
Consider a shortest path in from to . Since , this shortest path has to contain an edge labelled with . Repeating this argument for and , one gets another different (since on a shortest path between and every set contains the element , on the other hand, on a shortest path between and none of the sets does) edge with label , which contradicts the assumption that all labels are different.
Now, we calculate . If is not an edge label, then either all sets contain , or none of them does. In particular, is not shattered by . Thus, consists of and the sets , where is an edge label. However, all edge labels are different. So, we get that (since is a tree); that is, is s-extremal.
Let be an s-extremal family such that and . By Proposition 2, to every s-extremal family of -dimension 1, one can associate a directed edge-labelled tree , all edges having distinct labels. We have seen that consists of and the sets , where is an edge label. On the other hand, since , we also have that . As a consequence the tree must have edges and thus vertices; that is, such an s-extremal family has elements.
Now, conversely suppose that we are given a directed edge-labelled tree on vertices with edges, all having different labels. This tree at the same time also defines a set system . Take the edges one by one. When considering an edge with label going from to , then for all vertices closer to than to in the undirected tree, put into . Clearly, , and by the previous proposition, must be s-extremal.
To illustrate this, consider the following example with : (3)
We have , .
This gives a bijection between the set of all s-extremal families of VC-dimension 1 and directed edge-labelled trees.
Theorem 3. Let be an integer. There is a one-to-one correspondence between s-extremal families of Vapnik-Chervonenkis dimension with , , and directed edge-labelled trees on vertices, all edges having a different label from .
As a corollary, one can prove the following statement.
Corollary 4. There are different s-extremal families of Vapnik-Chervonenkis dimension 1 with and .
Proof. There are different edge-labelled undirected trees on vertices (see ), all edges having a different label from , and each of these trees can be directed in ways.
Simple examples of s-extremal set systems are downsets, that is, set systems such that for all implies that . For downsets, ; so, they are obviously s-extremal. One can obtain other examples from downsets using different set system operations, for example, bit flips. For , let be the th bit flip; that is, for ,
For , set . It is easily seen that s-extremality is invariant with respect to this operation since it keeps the family of shattered sets. However, not all s-extremal set systems can be obtained in this way. For this, note that in terms of the inclusion graph, a bit flip in the th coordinate corresponds just to reversing the direction of the edge with label in ; that is, bit flips preserve the undirected structure of the inclusion graph. Using this we can obtain many s-extremal examples not coming from downsets using bit flips. It is enough to pick a tree that is not a star and consider a set system corresponding to any possible orientation.
3. s-Extremal Set Systems from Two Consecutive Layers
For a uniform family , the graph is not connected; hence, cannot be s-extremal. As a relaxation of uniformity, we consider families which belong to two consecutive layers of . The next proposition shows that extremal families among them are actually special cases of the previously studied ones.
Proposition 5. Let , , be an s-extremal family of subsets of . Then, one has .
Proof. For , the statement can be verified by an easy case analysis. For , we can do induction on . For , the statement is just trivial. Now, suppose that and the result holds for all values smaller than . We prove that such an s-extremal family cannot shatter a subset of size . Suppose the contrary, namely, that shatters, for example, . Let
Since is s-extremal, both and must be s-extremal (it follows easily from the proof of the Sauer inequality; see, e.g., ), and for the shattered sets, we have that
Since , by the induction hypothesis, cannot hold; thus, we have . In this way, we constructed an s-extremal family with the same properties but on a smaller ground set. Continuing this we get to an s-extremal family that shatters . However, this is easily seen to be impossible, because for any , we have . This finishes the proof.
Using essentially the same argument, one can prove the following.
Proposition 6. Let , , be an s-extremal family of subsets of . Then, one has .
We return now to the situation when for some and , . Proposition 2 says in this case that is s-extremal iff (the undirected version) is a tree and all labels on the edges are different. As before, we also have that this tree has n + 1 vertices and edges. Permuting the labels on the edges corresponds just to a permutation of the ground set; so, if we want to characterize s-extremal set systems up to isomorphism, we can freely omit the labels from the edges.
Now, suppose that we are given a tree on vertices having edges. can also be viewed as a bipartite graph (since it is acyclic and so contains no odd cycles) with partition classes . Direct all edges from to , and let be as before the set system this directed tree just defines. It is easily seen that we have , where , and using the characterization of s-extremal families, we also get that is s-extremal. If we swap the role of and , we get the “dual” set system which is clearly also s-extremal using the same reasoning.
Summarizing the preceding discussion, we have the following.
Theorem 7. Up to isomorphism and the operation of taking the “dual” of a set system, there is a one-to-one correspondence between s-extremal set systems from two consecutive layers on the ground set and and trees on vertices. The bijection is realized via the map .
4. Ideal Bases of s-Extremal Set Systems of VC-Dimension 1
Take a family , and let be a field. For a set , let be its characteristic vector; that is, the th coordinate of is exactly when . One can associate to a polynomial ideal , the vanishing ideal of the set of characteristic vectors of the elements of :
A total order on the monomials composed from variables is a term order, if is the minimal element of , and is compatible with multiplication with monomials. One important term order is the lexicographic (lex for short) order. We have if and only if holds for the smallest index such that . Reordering the variables gives another lex term order.
The leading monomial of a nonzero polynomial is the largest monomial (with respect to ) which appears with nonzero coefficient in , when written as the usual linear combination of monomials. We denote the set of all leading monomials of polynomials of a given ideal by , and we simply call them the leading monomials of . A monomial is called a standard monomial of , if it is not a leading monomial of any . Let denote the set of standard monomials of . Obviously, a divisor of a standard monomial is again in . Standard monomials have some very nice properties; among other things, they form a linear basis of the -vector space .
A finite subset is a Gröbner basis of , if for every , there exists a such that divides . It is not hard to verify that is actually a basis of ; that is, generates as an ideal of . It is a fundamental fact that every nonzero ideal of has a Gröbner basis [19, Corollary ].
Gröbner bases and standard monomials turned out to be very useful when studying s-extremal set systems. Let be a set system and its vanishing ideal. Subsets of can be identified with square-free monomials via the map . With this identification in mind, one can prove that , viewed as a set of monomials, is just the union of the sets of standard monomials of for all lexicographic term orders, that is
On the other hand, for vanishing ideals, we have that for all term orders. These facts altogether result in the fact that a set system is s-extremal iff the set of standard monomials is the same for all lexicographic term orders. This algebraic characterization of s-extremal set systems leads to an efficient algorithm for testing s-extremality of a set system and offers also the possibility to generalize the notion to arbitrary sets of vectors (for more details and proofs, see ).
As an application of Proposition 2, we determine the Gröbner bases of s-extremal set systems of VC-dimension . Suppose that we are given a family together with . According to [13, Section 4.2], one can construct a Gröbner basis of as follows. For a pair of sets , define the following polynomial Now, if , then there exists a set such that there is no set with . For this set , we have . If the set is minimal (i.e., all proper subsets of are in ) and is s-extremal, then we also have uniqueness for the corresponding . Moreover, in the s-extremal case, the collection of all these polynomials corresponding to minimal elements outside together with form a Gröbner basis of for all term orders. Actually, more is true.
Proposition 8 (see ). is s-extremal iff there are polynomials of the form , which together with form a Gröbner basis of for all term orders.
If we restrict ourselves to s-extremal set systems of -dimension , things become very simple. Suppose that is a set system such that , , and . In this case, is the collection of all sets of size at most ; so, the minimal sets outside are exactly the sets of size . Fix one such set , and consider the edges in the inclusion graph labelled by and . From Proposition 2, we know that is a tree. Consider the unique path connecting the edges. There are possibilities. (i)The edges are directed towards each other on this path: (11) In this case, the corresponding set is ; so, . Indeed, then, every contains either or .(ii)The edges are directed away from each other on this path: (12) In this case, the corresponding set is ; so, . No contains .(iii)The edges are directed in the same direction towards the edge with label on this path: (13) In this case, the corresponding set is ; so, . If for some , then as well.(iv)The edges are directed in the same direction towards the edge with label on this path: (14) Similar to the previous case, ; so, .
Now, if we have , then using the previous case analysis, one can easily compute a Gröbner basis for . If we want just a basis of and not necessarily a Gröbner basis, we do not need to consider all pairs. Consider consecutive edges in , that is, a path of length with labels : (15) They define pairs and, hence, polynomials, , , and , where , , and are or depending on the orientations of the edges (in the above example we have and ). However, where is either or ; so, is superfluous, since it can be obtained from and . This means that when constructing a basis of , it is enough to consider only adjacent pairs of edges in .
5. s-Extremal Set Systems of Bounded VC-Dimension
The ideas from the previous sections can also be used to step a bit further and obtain results for s-extremal set systems of bounded -dimension in general.
Let the projection of a set system to a set be
Note that iff .
The main result of this section considers the projections of a set family from the perspective of extremality.
Theorem 9. Let be family of -dimension such that is s-extremal for all element subsets of . Let
Then, contains , and is s-extremal and of -dimension . Moreover, if is s-extremal, then one has .
Before proving Theorem 9, we first present some observations about extremal set systems in general.
For a set system and , let and be as defined previously in Section 3. The downshift operation of a set family by the element is defined as follows:
It is not hard to see that preserves s-extremality (e.g., [7, Lemma ]), and as already noted earlier, if is s-extremal, then so is for and .
For a set system and , we put
Proposition 10. Suppose that one is given a set system and an arbitrary element . Then, if is s-extremal, then so are , , and for all . Actually, for , one has that ; more precisely, a set is in iff it is in .
Proof. The following equalities follow easily from the definitions:
From these it follows that if is s-extremal, then so are and , since we can obtain them from using operations preserving s-extremality.
Next, note that if , then is just ; thus, if the original set system is s-extremal, then so is its projected version.
For the second part, we only have to note that for some , we have that for all , and the result follows.
The collection of all sets strongly traced by is denoted by . It can be shown that is bounded from above by (reverse Sauer inequality, [6, 7]), and a set system is called extremal with respect to the reverse Sauer inequality if . The authors in  proved that a set system is extremal with respect to the original Sauer inequality exactly when it is extremal with respect to the reverse one, and thus in this case, .
Similar to the case of , can also be obtained from the standard monomials of the vanishing ideal , namely, if viewed as a set of monomials, then it is the collection of those monomials which are standard monomials for all lexicographic term orders.
For a set system and a set , denote the set family by . We remark that if , then is just and, hence, is s-extremal if is such. This together with the fact that s-extremality of a set system implies the connectedness of its inclusion graph proves in a simple way the “if” direction of the following remarkable result of Bollobás and Radcliffe [7, Theorem ].
Theorem 11 (see ). is s-extremal iff is connected for every .
Now, we prepare the ground for the proof of Theorem 9 and return to the algebraic point of view. Let be an arbitrary family, and fix one term order on the monomials in . Suppose that we have at our disposal a Gröbner basis of with respect to a term order (e.g., in the s-extremal case, we can compute one as described in Section 4). From , we can compute a Gröbner basis for ; we only have to take the polynomials in depending only on the variables . In general, for a finite set of polynomials , denote by . The leading term of a nonzero polynomial with respect to is together with its coefficient from . The S-polynomial of nonzero polynomials in is where is the least common multiple of and . Buchberger’s theorem [19, Theorem ] states that a finite set of polynomials in is a Gröbner basis for the ideal generated by iff the S-polynomial of any two polynomials from can be reduced to using . (For more details on reduction and proofs, see [19, Chapter 1].)
Proof of Theorem 9. The facts that and just follow immediately from the definition of .
Now, fix one term order and one element subset of . Since by assumption is extremal, according to Proposition 8, the polynomials of the form , where is a minimal element outside and is the unique subset of such that , together with form a Gröbner basis of with respect to . We have hence, for all polynomials of the form in . Write
Note that a polynomial from is a member of for all element subsets for which . First, we prove using Buchberger’s theorem that is a Gröbner basis of with respect to . Take two polynomials , and let be the number of variables occurring in them. If , then there is some element set such that . However, since is a Gröbner basis of , can be reduced to using and so using with respect to as well. On the other hand, is possible only if and for some appropriate sets such that and . The leading terms of and are and , respectively; so, we can write them in the following forms: where and depend on disjoint sets of variables. Consider
Here, if we replace by and by , the resulting polynomial will be identically ; that is, reducing using gives . Moreover, the previous reasoning works for all term orders ; so, is a Gröbner basis of for all term orders. Consider next
Clearly, we have , and so defines also an ideal of . is actually isomorphic to the ring of all functions from to , which in turn is isomorphic to . In this ring, every ideal is the intersection of maximal ideals, and, hence, every ideal is a radical ideal. This implies that every ideal in , in particular as well, is a vanishing ideal of some finite point set from . When considering the 0-1 vectors as characteristic vectors, this finite point set also defines a finite set system. It is not difficult to see that in case of , the only possible candidate for this set system is itself; so, . However, in this case, we get that is a Gröbner basis of for all term orders, and, hence, according to Proposition 8, is s-extremal.
Finally, we note that if itself is already s-extremal, then according to Proposition 8, is a Gröbner basis for as well, and so .
6. Concluding Remarks
Concerning the structure of s-extremal set systems, the question arises whether an extremal family can be built up from the empty system by adding sets to it one-by-one in such a way that at each step we have an s-extremal family.
Open Problem 1. For a nonempty s-extremal family , does there always exist a set such that is still s-extremal?
From [7, Theorem 2], we know that is s-extremal iff is s-extremal; thus, the previous question has an equivalent form.
Open Problem 2. For an s-extremal family , does there always exist a set such that is still s-extremal?
There are several special cases when the answer appears to be true.
(1) If is a nonempty downset (, and ; then, ), then is extremal since . Moreover, in this case if, we omit a maximal element from ; then, it remains still a downset, and so it will be still s-extremal.
(2) If is an extremal family of -dimension , then according to Proposition 2, if we omit a set corresponding to a leaf, that is, to a vertex of degree in , then the resulting set system will still be extremal.
(3) Anstee in  constructed set systems , without triangles, that is, set systems with the property that for all -element subsets , we have that does not contain all -element subsets of . Note that, in particular, is bounded from above by ; hence, we have that , implying that . Comparing the sizes of and , we obtain that such set systems are s-extremal.
Clearly, any such set system contains the extremal subsystem . For the remaining part of these set systems, Anstee’s construction can be interpreted in an inductive way as follows. (i)Let . (ii)For , suppose that we already constructed . Let be the collection of all -element sets in . Define to be a graph, whose vertex set is , and there is an edge between exactly when . Take a spanning tree of . Consider (iii)Let .
It is not hard to prove that when we add , there will be a unique new element that gets into the family of shattered sets, namely, ; hence, the resulting system after each step will be s-extremal. Reversing it, if is such an example, then its elements can be deleted one-by-one in such a way that the remaining set system is s-extremal after each step.
(4) More generally, one can consider set systems with the property that for all -element subsets , we have that does not contain all -element subsets of , for some with . Furedi and Quinn in  constructed for all values a set system with the desired property and of size . The same argument as aforementioned shows that consists of all sets of size at most , and, hence, is s-extremal for all possible values. Their construction is as follows.
For , , let in particular, . Let consist of all , where . Order the sets of as follows: if either or , and with respect to the standard lexicographic ordering. It is not hard to see that if we remove the elements of with respect to this ordering one-by-one, starting from the largest one, then each time when we remove some , is eliminated from the family of shattered sets; hence, after each step, the resulting family will be still s-extremal.
The authors are grateful to Zoltán Füredi for discussions on the topic. L. Rónyai is supported in part by OTKA Grants K77476, K77778, NK 105645 and TÁMOP Grant 4.2.2.B-10/1-2010-0009.
R. Aharoni and R. Holzman, Personal communication.
A. Pajor, Sous-Spaces 1: Des Espaces de Banach Travaux en Cours, Hermann, Paris, France, 1985.
N. Sauer, “On the density of families of sets,” Journal of Combinatorial Theory A, vol. 13, no. 1, pp. 145–147, 1972.View at: Google Scholar
S. Shelah, “A combinatorial problem: stability and order for models and theories in infinitary language,” Pacific Journal of Mathematics, vol. 41, pp. 247–261, 1972.View at: Google Scholar
B. Bollobas, I. Leader, and A. J. Radcliffe, “Reverse kleitman inequalities,” Proceedings of the London Mathematical Society, vol. s3-58, pp. 153–168, 1989.View at: Google Scholar
B. Bollobás and A. J. Radcliffe, “Defect Sauer results,” Journal of Combinatorial Theory A, vol. 72, no. 2, pp. 189–208, 1995.View at: Google Scholar
P. Frankl, S-Extremal Set Systems, Handbook of Combinatorics, vol. 2, MIT Press, Cambridge, Mass, USA, 1996.
Z. Furedi and F. Quinn, “Traces of finite sets,” Ars Combinatoria, vol. 18, pp. 195–200, 1983.View at: Google Scholar
G. Greco, “Embeddings and the trace of finite sets,” Information Processing Letters, vol. 67, no. 4, pp. 199–203, 1998.View at: Google Scholar
R. Diestel, Garph Theory, Springer, New York, NY, USA, 2000, Electronic Edition 2000.
O. Pikhurko, “Generating edge-labeled trees,” American Mathematical Monthly, vol. 112, no. 10, pp. 919–921, 2005.View at: Google Scholar
L. Ronyai and T. Meszaros:, “Some combinatorial applications of grobner bases,” in Proceedings of the 4th International Conference on Algebraic Informatics (CAI '11), vol. 6742 of Lecture Notes in Computer Science, 2011.View at: Google Scholar
B. Buchberger, “An algorithm for finding the basis elements in the residue class ring modulo a zero dimensional polynomial ideal,” Journal of Symbolic Computation, vol. 41, pp. 475–511, 2006, Special Issue on Logic, Mathematics, and Computer Science: Interactions; Ein Algorithmus zum Auffnden der Basiselemente des Restklassenringes nach einem nulldimensionalen Polynomideal. Doctoral thesis, University of Innsbruck, 1965.View at: Google Scholar
B. Buchberger, “An algorithmic criterion for the solvability of algebraic systems of equations,” B. Buchberger and F. Winkler, Eds., vol. 251 of London Mathematical Society Lecture Note Seriesm, pp. 535–545, Gröbner Bases and Applications; Cambridge University Press, 1998.View at: Google Scholar
B. Buchberger, “Grobner-bases: an algorithmic method in polynomial ideal theory,” in Multidimensional Systems Theory—Progress, Directions and Open Problems in Multidimensional Systems Theory, N. K. Bose, Ed., pp. 184–232, Reidel Publishing Company, Boston, Mass, USA, 1985.View at: Google Scholar
D. Cox, J. Little, and D. O'Shea, Ideals, Varieties, and Algorithms, Springer, Berlin, Germany, 1992.
W. W. Adams and P. Loustaunau, An Introduction to Grobner Bases, Graduate Studies in Mathematics, vol. 3, American Mathematical Society, 1994.
R. P. Anstee, “Properties of (0-1) matrices with no triangles,” Journal of Combinatiral Theory A, vol. 29, pp. 186–198, 1980.View at: Google Scholar