It was narrated that Abul Tufail said:

I was with Ibn 'Abbas and Mu'awiyah, and Mu'awiyah did not pass any corner (of the Ka’bah) but he touched it. Ibn 'Abbas said: The Messenger of Allah (ﷺ) only touched al-Hajar (the corner where the Black Stone is) and (ar-Rukn) al-Yamani. Mu'awiyah said: No part of the House is to be forsaken.