The boolean operation argument is one of these:
A polygon is a pcb layer object, that is: if it's fullpoly, it may have multiple islands. The code operates on the clipped shape of the input polygons, not on the 'as drawn shape'. The result is zero or more polygon layer objects, each with a single island, created on the current layer.
The rest of the arguments will build a flat list of polygon objects to work on and need to address at least two polygons. Each argument may be one of:
Note: for the sub operation the first polygon is special, so order of attributes matter. When using selected or found to address polygons, if multiple polygons are selected or found at the time, the order they end up added to the operation object list is undefined.
Input polygons are unchanged, new polygons are created on the current layer.
Demo video: PolyBool(sub) mini-howto.