Maybe we need 2 boards, one game board as it is now, and one with our own labels. As the opponent moves on the game board, our labeled board gets updated automatically. So if I replace a ? with a 3 on my board, if the player moves the ? piece, on my board the 3 is moved to the same square.