Lawyer offered my full retainer back if I didn’t file a complaint, but threatened to take legal action against me if I did.

Long story not so short I hired this attorney for a custody case, and after having a custody investigator come to both homes I won the case.
The issue is that neither lawyer ever filed the agreement with the court and the case was closed due to inactivity. We only found out 11 months after agreement was reached when we had a parenting time dispute. I notified my lawyer hoping to get the agreement typed up and the case reopened to finish it off. He was responsive right away but never typed up the agreement and eventually after 2 weeks of no response from him I drove to his office to see what was going on, and we got into an argument in his office about the timeline and the fact that I had requested an invoice several times and been ignored. In our argument I told him the right thing to do would be to get the agreement done and give me my money back. He did get the agreement typed that day, but still refused to show me an invoice after I asked again over email, and sent me the response stating he would give me my retainer back if I didn’t file a complaint about the delays, but would take legal action against me for time spent if i did. What do I do, and outside of a custody attorney to finish the agreement what type of lawyer could best handle this?
Location: Minnesota