Welcome to SDCIA.
I assume we are talking about Carlsbad. You, unfortunately, have run afoul of an institution. Reality is severely distorted in an institution as unseen internal goals and self-interests are far more important than reason and logic. You have been accused of having a non-conforming shed even though they complain of fence violations. The fence is in violation of the 6 foot limit even though they measured it at less than that. Reality, as it appears from the outside, is completely different as seen from inside the institution.
What probably happened is that a middle manager got forwarded a complaint. He wrote out a numbered violation ticket and told a subordinate to “take care of it”. The subordinate is very aware of career advancement and “conviction” rates. Letting someone slide is a huge career deflator. (The District Attorney doesn’t want to lose a murder trial even if, oopsey, the wrong man was arrested. Anyone can convict a guilty man, but you have to a genius to convict an innocent man.) So you can see that the inside and outside reality of a large institution are completely different. They have entirely different goals and self-interests. You can’t see the inside of an institution from the outside, in any case.
You could do the direct frontal attack and have a lawyer send a letter showing that a licensed contractor has found no shed. If you contributed significant money to the mayor’s or governor’s campaign then just contact them and say you would love to support them in the future but you are too distracted by this (honest) mistaken code violation. Middle managers just love it when their political bosses owe them a favor. They would be delighted to make it go away. I’m assuming you don’t want to spend that kind of money. The City of Carlsbad will probably just pursue the fence violation anyway, even if they were completely mistaken about the shed.
The only solution I can offer is to think of this from the perspective of the middle manager or his subordinate. They may be ticked off that you didn’t get a permit before building the fence. You didn’t get on your knees and say “Please may I?” Much the same as if your teenage daughter took the car without permission, even though you would have granted it. After the fact “permission” requires punishment. Maybe the “Gee fellas, I’m just an honest citizen and really need help from a big smart, sophisticated, powerful person like you.” ploy would work. If you are a car dealer who can give them a great deal on their next car, even better. There are probably dozens of other ploys you could use. You have to give something to the employees of the City of Carlsbad that satisfies them, because they wield actual power. So stop grinding your teeth and losing sleep and just think of what would satisfy you if you were the power wielding city employee.