I second what sildra said. They will teach what you don't know, and there will be many other students in the same boat as you. There will be people who are better prepared mathematically, it's true. But it's not something for you to stress over. Covariant and contravariant matrices aren't too difficult to grasp, and in my experience, I got so much practice pushing subscripts around in Einstein notation in grad school that it didn't matter that I'd never really used it as an undergrad. (And trust me, the Einstein convention is NOT annoying! It's darned useful! Who wants to write out that many sums?)

I had my undergrad mechanics course out of Marion Thornton, and I found my classical mechanics preparation to be much better than many of my classmates (we covered Hamiltonians in that class, many of my classmates hadn't seen a hamiltonian, and the Hamitonian equations of motion, before grad school (at least in the context of mechanics). I don't know how far you got at Swat, but I wouldn't be to worried about it.)