I'm at my wits end regarding my 5-year old son. He doesn't follow rules at my house and his mother only makes it worse. : Parenting

I'm at my wits end regarding my 5-year old son. He doesn't follow rules at my house and his mother only makes it worse.

TLDR: 5-year old son doesn't follow rules at my house, frequently disciplined. Mother is no help. He's now declining visits. I'm about to give up.

I have a 5-year old son from a previous relationship. He comes every other weekend and every time he comes, we have issues. He does not follow rules. Frequently, he will be released from time out and break the same rule 2 minutes later. We (my wife and I) remind him of the rules every time he comes, yet he spends a ton of time in time out and frequently loses privileges like screen time.

On Thanksgiving, he was banging toys around and we told him to cut it out. He did it again and ended up breaking the TV. After time out, he resumed his rough play, so we had to ban him from those toys.

His mother has admitted multiple times that she provides no structure and few rules. She has even said that our rules are "dumb" in front of our son. So I get no help from her, if anything, she's making the issue worse. His grandparents spoiling him rotten is adding to the frustration. I tried speaking to them, but I got shut down with a "I raised 4 kids, don't tell me how to handle kids."

Limit digital media for your youngest family members. Avoid digital media for toddlers younger than 18 to 24 months other than video chatting. For children 18 to 24 months, watch digital media with them because they learn from watching and talking with you. Limit screen use for preschool children, ages 2 to 5, to just 1 hour a day of high-quality programing. Co-viewing is best when possible and for young children. They learn best when they are re-taught in the real world what they just learned through a screen. So, if Ernie just taught the letter D, you can reiterate this later when you are having dinner or spending time with your child. See Healthy Digital Media Use Habits for Babies, Toddlers & Preschoolers.

Our rules aren't extreme, either. No rough play inside, no running inside, don't touch things that don't belong to you without asking, etc. We may have to be on top of these rules more than most families, due to a small apartment and having up to 4 kids at a time, but the rules are similar to what he experiences in kindergarten. Speaking of school, I got a report from his teacher in the first month that he often disrupts the class and doesn't keep his hands to himself.

I just don't know what to do with him. His mother is no help, his grandparents are no help, the courts are no help. I just now messaged his mother about this weekend and I got "He said that he doesn't want to come." This makes the third time in a row that he has voiced that he doesn't want to come (I assume because he spends so much time in time out). I tempted to just let it go and wait for him to miss me. What can I do?