Aundre Hyatt chipped in 11 for Rutgers, which survived shooting 36.7 percent and a quiet showing from Clifford Omoruyi.
Miller Kopp scored 21 points for Indiana (7-1, 0-1) but the rest of the Hoosiers experienced frustrating showings against Rutgers' varied defenses.
Logan Johnson recorded 17 points, six rebounds and five assists for Saint Mary's (6-3), which lost its third straight game.
Matthew Cleveland added 11 points and a game-high 10 rebounds, with Jalen Warley and Caleb Mills each scoring seven points.
Alex Arians finished with (17 points, six rebounds), followed by William Kyle III (eight points, six rebounds, five assists) and Zeke Mayo (12 points, seven rebounds).