Mateus Uribe shot against the bar to put England back on an even keel but it took a brilliant save from Pickford, denying Carlos Bacca, to pave the way for victory.
"It's a great night to win a penalty shootout". Jordan Henderson's miss didn't seem so devastating the second time around.
"Yesterday was a massive milestone for us as a group because we hadn't won a knockout game for so long", he told the FA's YouTube channel.
The pair could yet meet again in the World Cup final with England to face Sweden on Saturday for the chance to reach the final four.More news: How to Watch Brazil vs. Belgium
England celebrate after winning the FIFA World Cup 2018, round of 16 match at the Spartak Stadium, Moscow. To get knocked down like we did [with Mina's goal], it's hard to come back from.
He said: "I get criticised about the top hand but I don't think I'll be getting criticised about it now!"
Pickford only emerged as a leading Premier League keeper a year ago and made his debut for England in November.
Asked by a TASS correspondent whether the team was feeling under the gun to reach the final of the 2018 FIFA World Cup in Russian Federation, the 27-year-old player of Tottenham Hotspur FC replied that "there is no pressure at all".
Said Kelly: "I was at Preston in 1990 and one of the cover keepers got a bit of an injury".More news: Ruthless Federer marches on with another straight-sets win
"But I understand why Gareth did it, it's the first thought at the time that you've got to defend and hang onto what you've got, but I thought besides that England were very composed".
"To say that about another goalkeeper, you know we are in the same position".
"We managed to break the mould", another said.
"If it had been to keep us in it, it could have been a more nerve-wracking experience".More news: The World Cup causes a problem for Atletico's Super Cup squad