Howard Stern Slams Simon Cowell Over Gabrielle Union's Controversial AGT Firing

Howard Stern is calling out Simon Cowell following Gabrielle Union‘s controversial departure from America’s Got Talent.

On Monday’s episode of The Howard Stern Show, Stern, 65, slammed Cowell — America’s Got Talent‘s creator and judge — accusing him of having “orchestrated” Union’s firing.

“How is it that Simon Cowell has orchestrated this?” Stern said. “He sets it up that the men stay, no matter how ugly they are, no matter how old they are, no matter how fat they are, no matter how talentless they are.”

Stern himself was a judge on AGT for four seasons from 2012 through 2015.

“But what he manages to do on all his shows is he constantly replaces the hot chicks with hotter chicks and younger chicks. Which is so obvious,” Stern added.

A rep for Cowell did not immediately respond to PEOPLE’s request for comment regarding Stern’s remarks.

Stern’s comments come just one day after Cowell’s company spoke out about Union’s grievances with AGT during her one-season run on the NBC talent competition series.

Syco Entertainment, which produces AGT, released a joint statement with NBC and production company Fremantle on Sunday. “We remain committed to ensuring a respectful workplace for all employees and take very seriously any questions about workplace culture,” said the statement, obtained by PEOPLE.

“We are working with Ms. Union through her representatives to hear more about her concerns, following which we will take whatever next steps may be appropriate,” the statement concluded.

RELATED: Debra Messing Slams NBC for Alleged ‘Disgusting Behavior’ After Gabrielle Union’s AGT Firing

Following the statement, Union, 47, shared a tweet from freelance writer and host Joelle Monique and commented on the proper way to apologize.

In her tweet, Monique outlined what makes for a “solid apology”: “1. Sincerely admit wrong doing directly to the offended party. 2. Be twice as loud correcting your mistake as you were making the mistake. 3. Lay out steps to correct your behavior in the future. Anything less is covering your own a–.”

Union retweeted Monique’s post, adding, “This! This! AND THIIISSSSSSS!!!!!!”

This! This! AND THIIISSSSSSS!!!!!! https://t.co/RCrtKtkQ4L

— Gabrielle Union (@itsgabrielleu) December 1, 2019

https://platform.twitter.com/widgets.js

News broke on Nov. 22 that Union and Julianne Hough — who joined the season 14 AGT judging panel back in February, replacing spots vacated by Mel B and Heidi Klum — would not be returning for the show’s upcoming 15th season. (It has yet to be announced who will replace Hough and Union for the upcoming season.)

Last Tuesday, a report by Variety claimed that while working on the show, Union had expressed concerns over racially insensitive situations during her time as a judge, including a joke guest judge Jay Leno allegedly made that was later edited out of the episode.

Union and Hough, 31, were also both subject to “excessive notes” on their physical appearance, sources alleged in Variety‘s report.

A spokesperson for NBC and Fremantle previously responded to such claims with a statement to PEOPLE on Nov. 26, saying, “America’s Got Talent has a long history of inclusivity and diversity in both our talent and the acts championed by the show. The judging and host line-up has been regularly refreshed over the years and that is one of the reasons for AGT’s enduring popularity. NBC and the producers take any issues on set seriously.”

Representatives for Hough, Union and Leno have not responded to PEOPLE’s request for comment.

RELATED: Gabrielle Union Says ‘Support Is Everything’ as She Leans on Dwyane Wade After AGT Firing

Blog Love B Scott was first to report about complaints Union had allegedly made prior to her departure, with an NBC source saying that she had spoken out about “problematic” situations on set.

Union broke her silence on her controversial departure last Wednesday, thanking her followers for their support during this difficult time. “So many tears, so much gratitude,” she tweeted. “THANK YOU! Just when you feel lost, adrift, alone… you got me up off the ground. Humbled and thankful, forever.”