Extra Management and admin on our part

Hi All, Can I have any feedback at all from anyone that has had either success or failure in suing a building company for going well over their estimated build time frame? Our build contract states four weeks as of August 2018 (they were only erecting a shed shell, not hard) and as of today they are still dragging their heels fixing things that the council have ordered them to do and their council deadline runs out today, but we still have plenty of outstanding issues left. Chances are the council will fine them an infringement fee for going over their deadline but I want to try to and recoup some costs for ourselves if we can. They are the biggest bunch of muppets, and bullies to boot so if their is anyway I can keep as much of my retainer as possible I will give it a damn good go! We had our own builder, sparky and plumber who were contracted to fit out the inside (thank god for this!) as a third of the shed is habitable, but when their shell failed the council inspections on epic proportions, we had to take down and will have to re-do some of our own work so they can fix their mistakes and it can’t be helped. This does not worry me as we have a retainer up our sleeve for just such a thing and I will be deducting all our extra costs from this amount BUT can I claim costs for going well over the deadline? Stress? Extra Management and admin on our part? Please can anyone who’s done this let me know how you went and what terminology you used etc. Any help would be much appreciated and when we finally get them out of our lives and a C.O.C for our building I will tell everyone who we used and what they did so you don’t end up in the same boat!