There are some distinct benefits, and also some definite downsides to keeping mice as pets. They are easy to care for, not long lived, and easily contained in a small space. However, they are also nervous, not long lived, and not very interactive in many cases.
Mice can live quite contently in a 10-gallon aquarium with a mesh top, or a cage of similar size. They need a food dish, a water bottle, somewhere to hide, and perhaps a wheel to run on and / or a chewing block. Litter of some sort must be placed in the bottom of the cage to soak up urine, and usually only needs changing about once a week.
Mouse urine is quite pungent, especially that of mature males. This can be tempered a bit by keeping only females, or by spot-cleaning the cage on a daily or every-other-day basis. But mice generally smell like something, so if you hate the smell, perhaps another pet or no pet would be better for you.
The lifespan of a mouse is not very long, usually only about a year to a year and a half. This can be difficult if they become beloved pets, but it also means they are less of a commitment. This makes them good pets for children that are old enough to be responsible, but might lose interest in a year or so.
Food and litter for a mouse or two is not very expensive. They are tiny and do not eat much, and do not need much litter for their cages, either. However, veterinary care for your mouse will not be much less expensive than for a cat or small dog, especially since you would need to seek out an exotics specialist.
A solitary mouse is not likely to be happy, because these are deeply social creatures. However, they are timid enough that it may be hard to get a mouse with a companion to bond with you. Two babies or already-friendly adults are generally best, because they will bond with you, and yet still appreciate each other's company. Always choose two of the same gender.
Interaction with a mouse is limited. People can hold them and sometimes let them run around desktops or other secure areas, but care must be taken that they do not escape. And older mice that are not already tame can be difficult to handle at all. However, people who mostly enjoy watching their pets run around and play will thoroughly enjoy having a mouse or two.