Masks are pixel based. They represent the intersection of one set of pixels with another. They nest and intersect with the masks above them.  

Booleans are geometry based, more complex, and super high-res. You can have a mix of different operations (not just intersection). 

Did this answer your question?