We’re not going to sugar-coat it. Unlike changing your major as an undergrad, changing graduate programs can be a challenge—and it’s really not encouraged.
That said, why spend two or more years of your life pursuing a graduate degree that you’re not passionate about? The whole point of a graduate education is learning more about the one subject that excites and inspires you. If you’re not in a program that does that for you (or perhaps you’ve realized your skill set doesn’t quite match the program), and you’ve realized that another master’s or doctoral degree path does, then you should consider switching graduate programs.