Mr Sánchez tabled the motion of no confidence after Mr Rajoy's centre-right People's Party (PP) was implicated in a huge corruption scandal.
There have been three other such votes since Spain returned to democracy following the death of long-time dictator Francisco Franco in 1975, including one against Rajoy previous year.
Pedro Sanchez has arrived in Spain's parliament, set to become the country's new prime minister after his socialist party yesterday secured enough votes to topple Mariano Rajoy in a confidence vote over a corruption case.
"Sanchez has reiterated a commitment with European orthodoxy and budget control in Spain", UBS analysts said.More news: Roseanne Barr Blasted for Blaming Michelle Obama for Show Getting Pulled
Mr Rajoy, who was one of Europe's longest-serving leaders, told the Spanish parliament he was proud of his record.
The 63-year-old is accused of failing to take responsibility for his party's involvement, which came to light last week after a former treasurer Luis Bárcenas was jailed for 33 years for receiving bribes, money laundering and tax crimes. The party was fined €245,000 ($287,000) for benefiting from the kickbacks-for-contracts scheme.
"The PP has had corrupt people, I acknowledge it but the PP is not a corrupt party", he said, before accusing Sanchez of "opportunism at the service of personal ambition".
He says the corruption accusations concern only a tiny number of politicians and no current members of government.More news: 'Slain' Russian Journalist Arkady Babchenko Rises From the Grave
Sanchez, who is expected to be sworn in by Monday and appoint his cabinet next week, has promised to start talks with the Catalans but said he will not give them an independence referendum.
The pro-European politician - nicknamed Mr Handsome in Spain - said he wants to implement reforms in some areas, including salaries, pensions and gender equality.
With the PNV's support, and that of the far-left Podemos and Catalan pro-independence parties, which is all but guaranteed, Rajoy will be forced out and replaced as PM by Sánchez on Friday.
Aitor Esteban, a Basque party MP, warned Mr Sanchez: "Your government will be very complicated, weak and hard".More news: Ukrainian Interior Minister Slams Public Backlash Over 'Murder' of Babchenko