Control Who Speaks For You