An indirect-forcing immersed boundary method for solving the incompressible Navier-Stokes equations involving the interfaces and irregular domains is developed. The rigid boundaries and interfaces are represented by a number of Lagrangian control points. Stationary rigid boundaries are embedded in the Cartesian grid and singular forces at the rigid boundaries are applied to impose the prescribed velocity conditions. The singular forces at the interfaces and the rigid boundaries are then distributed to the nearby Cartesian grid points using the immersed boundary method. In the present work, the singular forces at the rigid boundaries are computed implicitly by solving a small system of equations at each time step to ensure that the prescribed velocity condition at the rigid boundary is satisfied exactly. For deformable interfaces, the forces that the interface exerts on the fluid are computed from the configuration of the elastic interface and are applied to the fluid. The Navier-Stokes equations are discretized using finite difference method on a staggered uniform Cartesian grid by a second order accurate projection method. The ability of the method to simulate viscous flows with interfaces on irregular domains is demonstrated by applying to the rotational flow problem, the relaxation of an elastic membrane and flow in a constriction with an immersed elastic membrane.