Following the events of Heroes Chronicles: The Sword of Frost, Gelu, the leader of AvLee's Forest Guard and wielder of Armageddon's Blade, attacks Kilgor, the Barbarian King of Krewlod, in an attempt to claim and destroy the Sword of Frost. As foretold by a prophecy, when Armageddon's Blade and the Sword of Frost clash in battle, the result is a massive explosion (known as the Reckoning), which destroys the world of Enroth. Many of the inhabitants of the planet, however, escape through mysterious portals that appear during the Reckoning that lead them to another world, called Axeoth – also the setting for Might and Magic IX. Included among the refugees are many of the heroes of previous installments of the series, and many return to their old ways shortly after arriving on their new home.
The story of Heroes of Might and Magic IV unfolds primarily through the game's six campaigns, each of which establishes how the major kingdoms of one of Axeoth's continents were forged in the wake of the Reckoning, and the arrival of the Enrothian refugees on Axeoth. Each of the campaigns centers on a faction leader, and tells the story of how that leader came to prominence. Unlike Heroes of Might and Magic III, all of the six campaigns are accessible from the beginning, and each unfolds as a stand-alone tale.