After the succession was settled and his brother Kublai Khan was established as Great Khan, Hulagu returned to his lands by 1262. When he massed his armies to attack the Mamluks and avenge the defeat at Ayn Jalut, however, he was instead drawn into civil war with Batu Khan 's brother Berke . Berke Khan, a Muslim convert, had promised retribution in his rage after Hulagu's sack of Baghdad and allied himself with the Mamluks. He initiated a series of raids on Hulagu's territories, led by Nogai Khan . Hulagu suffered a severe defeat in an attempted invasion north of the Caucasus in 1263. This was the first open war between Mongols and signaled the end of the unified empire.