Energetic Variational Approach is used to derive a novel thermodynamically consistent three-phase model of a mixture of Newtonian and visco-elastic fluids. The model which automatically satisfies the energy dissipation law and is Galilean invariant, consists of coupled Navier-Stokes and Cahn-Hilliard equations. Modified General Navier Boundary Condition with fluid elasticity taken into account is also introduced for using the model to study moving contact line problems. Energy stable numerical scheme is developed to solve system of model equations efficiently. Convergence of the numerical scheme is verified by simulating a droplet sliding on an inclined plane under gravity. The model can be applied for studying various biological or biophysical problems. Predictive abilities of the model are demonstrated by simulating deformation of venous blood clots with different visco-elastic properties and experimentally observed internal structures under different biologically relevant shear blood flow conditions.