Liberty Landing is pretty much the going rate for NJ, around $4.50. Newport is another option and very convenient to the city by PATH. On the city side most are about $3 above the NJ side. You can move up to W 79th St for a little less. On the west side then you can move on up out of town and commute and save some.
On the east side you can cross to One 16 in Brooklyn for $4. Next are marinas on the East side of Brooklyn which is a long commute but a little less expensive. Then there are also marinas on the Long Island Sound, some convenient to trains but a long ride back to NYC.
A little lesson about NYC. It's expensive. Very. Everything. Go prepared for that and then get over it and just have fun. It's like no other place in the world. So much to see and enjoy and some of the sites such as Central Park are even free. But don't go to NYC to do nothing and save money. Go to have the best time, spending more than you want but not more than you can afford, and leave having had a great experience.
Everything there from Art to Opera to Broadway. Always the half price line for Broadway too. Deli's that are like no where else. I love seeing Deli's in the rest of the country thinking somehow they compare. They don't, although slowly the old famous great deli's closing. Carnegie Deli closed. Statue of Liberty, Empire State Building, Metropolitan, Museum of Modern Art so much to see. Just enjoy walking the streets. You can go watch people on 5th Avenue spending outrageous amounts in world famous stores and you don't have to spend. Must get to Soho while there.
You can have an incredible time if you can get over the pricing of food and lodging.
I also recommend circle boat tours of the city. Great tour boats with good narration. Of course I also list Yankee Stadium, Times Square, Madison Square Garden, and Penn Station as must sees.