Marcos Rojo missed a clear header from four yards and full debutant Radamel Falcao struck the bar, but Van Gaal's team then went to pieces. Jamie Vardy and Estebian Cambiasso found the net after some terrible defending while David Nugent and Ulloa also scored penalties to seal the Foxes' first home win over the Red Devils in 29 years.
After enjoying such a comprehensive win over QPR last week, Van Gaal found the defeat at the King Power Stadium hard to take.
"We start the last match against QPR very well with a new team, new players and then we played very well here, but we gave the game away," the United manager said. "Leicester had five shots on goal, and that was it. "These five goals were existing because we made errors in ball possession.
"We created a lot of chances and made superb goals but you have to do that over 90 minutes, not 60 minutes. It was not enough today. "You have to kill the game and we didn't do that." At times in the first half, United were simply irresistible.
Argentinian duo Rojo and Di Maria looked dangerous down the left flank while Falcao, Van Persie and Wayne Rooney looked a devastating front three. But in defence, United's concentration and marking was easily exposed by the Premier League newcomers. Eyebrows were raised when Van Gaal decided to not to spend any of his vast budget on a centre-half this summer, and it is easy to see why after this display. Chris Smalling, who came on for the injured Jonny Evans in the first half, was easily bullied by Leicester's front three.
Tyler Blackett, one of only three players to start every game for United this year, was sent off when he hacked down Vardy to hand Leicester their second penalty while Rafael was at fault for the first spot-kick when he barged over the former Fleetwood man.
United lacked the energy to mount a comeback late on as Van Gaal was forced to withdraw Falcao and Di Maria through fatigue.
"I had to change Falcao, not because he was bad but because I have to build his fitness up. That is my problem," Van Gaal said. "Di Maria also had a problem."