Now that I get thinking about it, the beach/waterfront scene might be a little early in the year right now.
Disagree. Off-season is the best way to enjoy these areas. Very low douchebag factor this time of year - no traffic, no lines, always get a table, etc. Hotels, inns offer special pricing/getaway packages during this time.
I lived in Mystic for a few years. Nice little village to walk around with cute little shops, Mystic Seaport, Aquarium and of course the famous Mystic Pizza (pizza sucks - go to Angie's further down RT. 1). Daniel Packer Inn is an excellent place for a meal.
Agree with the other suggestions nearby - Foxwoods, Newport, Providence (Federal Hill - excellent Italian restaurants). If it were me, I'd do Mystic one day, get up drive up the coastal route through Rhode Island to Newport - have lunch, walk around, a few cocktails, head over to Providence for a nice Italian meal/pub crawl that night or to Foxwoods for a little gambling, dinner and a night's stay. Get up and head back to NJ.
Talk to your bartenders and waitresses for tips on filling in the blanks.