~Tommes Pommes~: If someone challenges you (which you can only be challenged by 1 person at a time), the game will automatically start - so no, there is no way to decline a stair game.