He did come as fast as he could. I'm sure he is more than a little concerned about the slaver's reaction to Xander's escape, and another slaver might have done Xander a lot more damage in terms of punishment. Ironically, Dareh's punishment, in the long run, would have broken Xander far more effectively.

And yeah, he would kill himself before being used for evil... that's our Xander.