The Enquirer understands that ISG told its supply chain last week that it would no longer be part of the consortium led by leisure operator GLL delivering the stadium.
Work on site has been delayed by a judicial review into the planning approval but main construction was expected to start next Easter.
One subcontractor said: “The job has suffered a few delays but we’ve now been told that ISG will no longer be the main contractor.
“The word is they have pulled out because costs have gone up while the job has been in planning but the client won’t move on the budget.”
ISG was named as main contractor back in January 2015 after fighting off competition from Barr and Carillion.
A former leisure centre on the site was demolished in late 2015 but the main construction phase has been held-up by planning wrangles.
The 8,000-seater stadium will be home to York’s football and rugby teams alongside leisure, retail, office and community facilities.
Another subcontractor said: “We’ve heard that the client is already talking to other firms to replace ISG on the job.”
An ISG spokesman confirmed the news on Monday morning.
He said: “We’ve worked for over two years, alongside our consortium partners, to realise the vision for a new community stadium at York, and successfully delivered the enabling package for the scheme in late 2015.
“Our commitment to the stadium has never been in question and we have worked diligently to explore every opportunity to bring the scheme to site.
“However, with further delays caused by the Judicial Review and increasing cost pressures, we’ve been unable to reach a consensus for the next phase of the project.
“We will continue to support our partners as they seek an alternative contractor.”