If you don’t want to set the environment variable, another option is to modify the scripts part of package.json from:

"start": "react-scripts start"

to

Linux (tested on Ubuntu 14.04/16.04) and MacOS (tested by @aswin-s on MacOS Sierra 10.12.4):

"start": "PORT=3006 react-scripts start"

or (may be) more general solution by @IsaacPak

"start": "export PORT=3006 react-scripts start"

Windows @JacobEnsor solution

"start": "set PORT=3006 && react-scripts start"
Update due to the popularity of my answer: Currently I prefer to use environment variables saved in .env file(useful to store sets of variables for different deploy configurations in a convenient and readable form). Don't forget to add *.env into .gitignore if you're still storing your secrets in .env files. Here is the explanation of why using environment variables is better in the most cases. Here is the explanation of why storing secrets in environment is bad idea.