Reasoning complex logical queries on incomplete and massive knowledge graphs (KGs) remains a significant challenge. The prevailing method for this problem is query embedding, which embeds KG units (i.e., entities and relations) and complex queries into low-dimensional space. Recent developments in the field show that embedding queries as geometric shapes is a viable means for modeling entity set and logical relationships between them. Despite being promising, current geometric-based methods face challenges in capturing hierarchical structures of complex queries, which leaves considerable room for improvement. This paper presents POINE², a geometric-based query embedding framework based on hyperbolic geometry to handle complex queries on knowledge graphs. POINE² maps entities and queries as geometric shapes on a Cartesian product space of Poincaré ball spaces. To capture the hierarchical structures of complex queries, we use the Poincaré radius to represent the different levels of the hierarchy, and we use the aperture of the shape to indicate semantic differences at the same level of the hierarchy. Additionally, POINE² offers a flexible and expressive definition of logical operations. Experimental results show that POINE² outperforms existing salient geometric-based embedding methods and significantly improves these methods on evaluation datasets.