The Carey family confirmed he died on Sunday morning aged 83.
John Carey and his two brothers Tom and Pat started the firm in 1969 after moving to London from Ireland.
The “three brothers and a van” started a groundworks contracting company called PJ Carey Plant Hire (Oval) Ltd which evolved into the current contracting group.
A family statement said: “It is with great sadness we share that on Sunday morning, John Carey, one of the three founding partners of the Carey Group, sadly passed away.
“John was a dearly loved husband, father, grandfather, brother, uncle and friend to many. He will be greatly missed by his loving family and close friends, whom we ask you to keep in your thoughts and prayers.
“His departing will leave a void for the family as John was a leader, figurehead and inspiration. We know from the many kind words already shared how well respected he was, and we look forward to sharing these with many of you in the coming weeks.
“We, the family, now ask for privacy at this time to allow us to come to terms with this loss.
“Thank you for your support at this sad time.”