Probably all the bickering amongst themselves. I read in Moonwalker (the book), that he always used to get outvoted whe they pitched ideas.
i also remember mj saying that he was also the only one in the creative department, writing the songs, doing the arrangements and all, while all of the others were chasing girls. he was pretty much left alone there as well.
and after all: mj left the group cuz he felt it was too limiting for him as well. he was well aware that any reunion would only be a success because of him, i think. so i figure it might not have been just being reluctant, he simply didn't need to.
how long did the jacksons last after michael? one hot minute...? it was always all about michael, he always had the most fans. not knocking the others' talents but it is very clear who was the genius behind it all.