While the potato, with skin on, might be less prone to the food safety issues if it still had any pathogens in the dirt after washing and boiling, leaving out at room temp is not good. Often, peeled potato that has been cooked will turn gray from exposure to air, accelerated when left out at room temp.
