Edge as in 'rim' comes to English through the Germanic route and ultimately from the Proto-Indo-European ak 'be sharp' or perhaps 'point'. Note that Germanic trail also sees forms such as the Old Frisian egg, Old Saxon eggia, and Old Norse egg. Remember these early forms when we look at the other use of 'edge' below.
Rim is derived from Old Norse rimi where it was used to refer to 'a raised strip of land or ridge'. That it came to be used in this sense is down to it being used in the Old English saerima or 'seashore'.
Border has only been used in this sense since the early fifteenth century, it came to English with Old French bordeure 'seam, edge of a shield'.
Boundary puts the suffix '-ary' on the word which should be seen as meaning 'limit'.
Fringe also came to English from French, there frenge was used to refer to 'thread, strand' as much as 'edge'.
Margin was first used in English to refer to the edge of a body of water (and still is today). It came from Old French margin and ultimately from Proto-Indo-European merg 'border'.
Lip has only been used in this sense for the last two centuries at most. Clearly from the body part it has hardly changed in millennia and has never had any other meaning, thus the etymology is difficult to see.
Brim, as with margin, started as a reference to where land met water, this time more often used to refer to the riverbank. It is derived from Proto-Indo-European bhrem 'point, spike, edge', where it is easy to see why it came to be used when referring to hats.
Brink has near identical historical usage and meanings, with the exception of Proto-Indo-European bhreng meaning 'to protect' as well as 'edge'.
Verge comes from the Old French verge meaning 'twig, branch' and also 'measuring rod' and 'rod of office' and also 'penis'.
Perimeter is a compound of peri- 'around, about' and -meter from metron 'measure'.
Circumference is another compound word, from circum- 'around' and ferre 'to carry' and also 'to bear children'.
Periphery is first seen in the fourteenth century, where periferie means 'atmosphere around the earth'. It is a compound of peri- (as in perimeter) and the same suffix found in cirumference.
Edge again, this time as in 'advantage'. is from the verb 'to egg on'. Now, if you've ever wondered why we 'egg' someone on, it's because 'egg' is the correct early form and only from the early 17th century did the word become 'edge'. There was no confusion between the two as these would be used in quite different contexts. Thus to 'egg' someone on is the same as edging forward or advancing slowly. (I won't bother with the etymology of 'egg' as in chicken here, or we'll be here all day.)
Advantage is another first sen in the fourteenth century, and comes from the Latin abante 'from before' and Proto-Indo-European ant 'front, forehead'.
Lead is a Germanic word, all forms suggesting 'to travel' and all can be traced back to Proto-Indo-European leit 'to go forth'.
Superiority is from 'superior' and from 'super', 'above, upper' and from Proto-Indo-European uper 'over'.
Dominance is a derivative of dominant, coming to English from Old French dominant, from Latin dominari 'to rule', and ultimately from Proto-Indo-European dem 'home, household'.
Supremacy is derived from supreme, from Latin supremus 'highest' and from that same Proto-Indo-European root uper seen in 'superior'. (Well who else did you think I was going to suggest fitted this description?)
