Abstract This paper focuses on the Rayleigh–Taylor instability in two-dimensional system of equations nonhomogeneous incompressible viscous fluids with capillarity effects a horizontal periodic domain infinite height. First, we use modified variational method to construct (linear) unstable solutions for linearized capillary problem. Then, motivated by Grenier’s idea (Grenier Commun. Pure Appl. ...