Yep, the router needs lookig at.
The router obtains a public IP address through its connection to the modem, but it is having a problem assigning private IP addresses to each of your PC's.
If you have software firewalls on each mahine, configure them to allow access to the IP range that the router uses to assign IP's to its LAN.....that could help.
Of course, there could be any number of internal settings on the router that could be at fault here. If this happens again, don't re-boot the router, log onto a command line and type "ipconfig /all", you should then see the IP address assigned to you by the router. If the address does not match the range that the router uses, then its the fault of the router.
I'm sure there is mre I can post here, but that is a start.
Hope that helps.
Welcome to the site as well.