When hardmode starts, everyone is scrambling to get the loot so they can be prepared for the first boss. Something about hardmode is that enemies can start to drop treasure maps, which can be used to summon the Pirates.
Issue is, nobody really likes them. But sometimes someone “accidentally” summons one. Everyone starts to hate that person for summoning the Pirates. The reason people really hate the Pirate Invasion event is because it is an extremely disruptive event. In Survival it can easily drain away Hardmode, and by then some of the people who helped beat the Wall of Flesh would have already left, so rebeating WoF will be harder, all because someone “accidentally” summoned a very disruptive invasion. It’s somewhat the same energy as griefing.