You would need to give your devices different names in your Alexa app - lamp 1, lamp 2, bedroom, living room, etc.
Creating a Smart Home Device Group for each room may help you get closer to what you're looking for. An Echo device can only be added to one smart home group at a time, so when standing in the living room you could say "Turn on living room" or "Lights on" to control devices only in that group/room.
To create a smart home device groups in the Alexa app:
Go to the menu, and select the Devices icon .
Select the + icon, and then Add group.
Use pre-set group names, or create your own, and then select Next.
Select the devices you want to include and then Save.
Thanks for the reply. I already have groups set up but I can still control any device from both echos. I'll try renaming one plug to lamp 2 and see if it helps. Is there any way to make the grouped devices only controllable by the echo in the same group?