A new method to construct task graphs for $\mathcal{H}$-matrix arithmetic is introduced, which uses the information associated with all tasks of standard recursive algorithms, e.g., block index set matrix blocks involved in computation. Task refinement, i.e., replacement by subcomputations, then used proceed hierarchy until containing actual data are reached. This process a natural extension cl...