The museum holds decent artefacts and also houses the mangled remains of a whale skeleton. The museum also provides a history of the town and a list of names of all the Danes who manned the fort. It also has an exhaustive list of which ships landed at the port of entry. A pretty decent affair.
Karaikal also has decent rooms / stay options if Tharangambadi is filled up or doesnt suit ones budget. Karaikal has a decent beach and is probably the only beach that I can remember of in the recent times where one can ACTUALLY find sea shells on the beach... which is quite fitting and amazing considering the state of affairs in Marina and Elliots. I find more plastic and dirty items there than I can find shells. Karaikal also has a very big walkway of rocks which leads one some 1.1-1.3kms into the beach which makes it all the more interesting and thrilling. Its quite normal and common to see crabs, scorpions and a few land/sea animals frequent these rocks.
Update: There is a bypass from Chidambaram to Karaikal which is a better road to choose rather than driving into Chidambaram. The roads leading into Chidambaram are quite decrepit and worth giving a pass.