The current contract is held by Kier and was extended in 2017 for two years until March 2019.
Riney will take over once this term is completed, delivering civil engineering works, street lighting and electrical works and surfacing works on carriageways and footways.
Its initial contract term will be for five years with the option to extend on one or more occasions for any combination of years up to a maximum of 10 in total.
The deal is a big win for Riney under the new ownership of Tarmac, which bought the firm last May.
Last year JB Riney won a contract worth over £70m to deliver major and minor highway maintenance works for Bromley Council.
John Davis, managing director, JB Riney said: “This contract extends our presence in London and demonstrates the strength and quality of the Riney offering. We’re looking forward to partnering with the team at Harrow to deliver a first class service for the Borough.”