Amazon cloud is pretty easy in general. I haven't used GoGrid.
Is network latency is a big issue with cloud computing?
Network latency was never an issue with cloud computing in my experience. Amazon cloud services is very ready for production use and is used widely, from small startups to big companies.
How is the performance as compared to a reserved virtual server provided by most ISPs?
That really depends on what kind of VPS service you opt for, specifically how much dedicated resource made available to your private server instance, how overloaded your (real) server is serving other clients (important for non-reserved resources). In general I have preferred dedicated servers over Amazon cloud for the services we provide.
If you are looking for a good VPS (which I used it for several years for my sites), I would recommend WestHost (aff. link).
Is it really cheaper as against say virtual private server (with similar configuration)?
Depends on your resource usage. Based on your expected usage you should really do some back of the hand calculation. If you expect large usage and yet spend a fixed amount go for dedicated server / virtual private server. If you are unsure about usage and don't like to spend upfront for a dedicated server / VPS, then go cloud.
Note: The above is an opinion piece and represents the opinion of the author only.