I will show how the pervasive notion of self-similarity in geometry (so-called "fractals") have natural counterparts in Group theory, and in the theory of (Associative or Lie) algebras. I will in particular illustrate by examples how these notions led to the construction of radically new objects, that answered or refuted long-standing questions and conjectures in Algebra.