If Game Center has an icon (also known as an avatar) for a player, it'll appear next to his/her name in the lobby and elsewhere. If not, a default icon using the first letter of the player's name will be created.
If you don't like the default icon, you can change it by tapping it. You can pick a photo from your photo collection, and then move and scale it within the provided circle. When you're done, the new icon is saved and should appear in the game. The circle is just for reference and does not mean the icon must appear in a circle.
You can tap a player's icon to change it again, if you made a mistake or just want to change it to something else.
If Game Center provides an icon, it'll appear instead. Saved icons are only used when Game Center has no icon.
If a photo has a transparent background, the background may appear black in the icon.
Icons are stored on the device under your Game Center ID. If another person uses your device and signs in with their Game Center ID, they will not use nor be able to change your icons. Each icon uses about 30K of storage space.