You have to do both, and you have to do them with discipline. I thought I could get by on reduced cardio but it just doesn't work. I do cardio seven days a week in addition to lifting 3-4 times a week.
The biggest single factor is how you eat though. You need it all, but the biggest thing...