It's an accepted fact," Gandhi, who belongs to the opposition Congress party, said in remarks at The National Press Club in a visit to Washington.
China and India have been uneasy neighbors for decades following a war on their disputed Himalayan frontier in the early 1960s.
In May, Modi said peace on India's border with China is essential for normal relations between the nuclear-armed neighbours.
Separately, Gandhi blamed Modi for India's religious polarization, saying his Hindu nationalist party was not inclusive.
They don't embrace everybody, and they divide society," Gandhi said.
Persons:
Rahul Gandhi, Narendra Modi's, It's, Gandhi, Modi, Simon Lewis, Kanishka Singh, Jamie Freed
Organizations:
National Press Club, Bharatiya Janata Party, BJP, 161st, Thomson
Locations:
China, Beijing, Washington, India, Arunachal Pradesh, Tibet