A critter is considered, by Merriam-Webster Dictionary, as simply a "creature".
A varmint, on the other hand, is considered "1: an animal considered a pest; specifically : one classed as vermin and unprotected by game law"
So basically, a varmint is a sub-set of a critter that's not tasty enough to eat or hunt into oblivion. Sort of like the difference between a plant and a weed:
A plant is something that you want growing in your garden, versus a weed, which is something you DON'T want growing in your garden.
It all gets a bit fuzzy when you have folks who like raising rabbits (versus gardeners who despise the ones munching on their gardens), or folks who eat dandelion greens (versus those who kill every fuzzy yellow head they find in their lawn). It all comes down to a matter of perspective, doesn't it?
[photo: Thug2 the Rooster checking out the new girls.... "hellloooo, ladieeeeez!"]