Essentially, what you are saying is the same thing. You can connect a router to the cable connection and connect the 2 computers to that router. You could also get a wireless router, connect that to the cable, and either used network cables to the computers (exactly as above) or else use the router's wireless option and connect wirelessly. Of course, wired is the safest option as it won't have disconnections, bad speeds, etc..., but it's also neat because of the fact that you don't have wires running around, and if you have a notebook, you can always move around the house.
The price difference between a wired or a wireless router is almost negligible nowadays (at least where I live), so I'd go for a wireless router nonetheless. If the wireless connections prove to be inadequate, just wire everything up (wireless routers usually have 4 extra wired ports where you can hook up computers).
Oh, and always make sure you secure your wireless connection - you wouldn't want free joyriders on your internet connection!
Hope this helps mate!