Those are done with animated GIF images. They're basically just multiple images (frames) stored within a single image file. There are a bunch of tutorials on the web about making them. Here's one I found:
Making Animated GIFsThe other way to do animations on the web is with Flash. These can be far more sophisticated and can have synchronized sound. Macromedia makes the software needed to make these. I don't know if there is any free software to make these. I'd start with GIFs first and move on to Flash if you find it's something you want to learn more about.