In the US, only around 40% of the incarcerated are locked up for violent crimes and that figure is artificially inflated, as lots of things classified as violent crime don't actually involve direct physical harm. In some states, purse-snatching, manufacturing methamphetamines, and stealing drugs are considered violent crimes. Burglary is generally considered a property crime, but an array of state and federal laws classify burglary as a violent crime in certain situations, such as when it occurs at night, in a residence, or with a weapon present. So even if the building was unoccupied, someone convicted of burglary could be punished for a violent crime and end up with a long prison sentence and “violent” record.
Another thing; drug possession arrests lead to criminal records which lead to reduced employment opportunities which lead economic anxiety which lead to a life of crime, which often necessitates violence. If you have no opportunities, you have nothing to lose, so the threat of going to prison doesn't deter people.