He already conceded defeat and congratulated Sanchez, 46, as the next prime minister.
Mr Rajoy told MPs on Friday: 'It has been a honour to leave Spain better than I found it. Thank you to all Spaniards and good luck'.
Mr Rajoy became Spain's first sitting Prime Minister to stand witness in a trial over the corruption cases, believed to have taken place between 1999 and 2005, and many called for him to resign. Please refresh the page for the fullest version.More news: Roseanne Barr Blasted for Blaming Michelle Obama for Show Getting Pulled
Rajoy went to shake hands with Sanchez after the result was announced.
Mr Sánchez tabled the motion of no confidence after the PP was implicated in a huge corruption scandal.
In order to push through the no-confidence motion, the Socialists, who hold just 84 of the parliament's 350 seats, had to cosy up to parties they had previously clashed with, like Catalan separatists and the anti-establishment Podemos.
It also ordered the party to pay back 245,000 euros ($290,000) received from the scheme to help finance election campaigns.More news: WWDC 2018 Decorations Begin Popping Up Ahead Of Monday’s Keynote
Aitor Esteban, a Basque party MP, warned Mr Sanchez: "Your government will be very complicated, weak and hard".
But his term in office was also marred by a series of corruption scandals involving former PP members.
In its ruling, the court said the credibility of Rajoy's testimony "should be questioned".
He said Mr Rajoy, 63, had failed to take responsibility for his party's involvement, which came to light last week after one of its former treasurers was given a 33-year jail sentence.More news: Korea's economic growth forecast at 3 pct. in 2018
Sanchez had been Rajoy's most loyal backer in his takeover of Catalonia's regional government following its failed secession attempt a year ago.