vim/vi inverted replace

Replace any character that isn't the ^ character with nothing:

%s/[^^]//g