The 29-year-old Pegula had no answer to Mertens' versatile game.
Former doubles world number one Mertens used her complete arsenal of shots, including sliced forehands, drop shots, lobs and deep groundstrokes to move her opponent around and pin her back.
Pegula, who had advanced into the third round after her ailing Italian opponent Camila Giorgi retired after the first set, broke Mertens early in the second but the Belgian responded in the next game.
Pegula earned another rare break point at 3-2 but sunk a three consecutive backhands into the net.
Reporting by Karolos Grohmann, editing by Pritha SarkarOur Standards: The Thomson Reuters Trust Principles.
Persons:
Roland Garros, Belgium's Elise Mertens, Jessica Pegula, Jessica Pegula's, Belgian Elise Mertens, Pegula, Mertens, Camila Giorgi, Karolos Grohmann, Pritha Sarkar
Organizations:
U.S, REUTERS, Belgian, Thomson
Locations:
Paris, France, PARIS, Italian, Belgian